initially it does sound good, but then the Dev team would need to alter every dialog in the game on every NPC. This would be both time consuming and somewhat unnecessary. Scripted conversations as they are depict a simple flowchart from point A to point B, they do not offer a lot of options, and often force a character to say something they would not necessarily say to achieve an end result. It is easy enough to simply emote that you are pointing and gesturing when going through the dialogs.
Secondly, if using the Silent Spell feat, the voice component of the spell is already removed, you only see the visuals during casting, so the only way you'd be hearing the casting part of the spell is if the player was metagaming and not using the feat so they could cast more higher level spells. I know from experience, I played a mute sorceress a while back, and every spell I casted was Silenced and never produced sound.