It would be nice if world creators could define a "sound deadening" collider, and if a player hears another user's voice, but a raycast between those two users passes through that collider, the other user's voice is ducked/EQ-ed to account for the fact that there's a solid wall between them.

Alternately, it might be simpler for you to just give Udon local-only control of how each user sounds to the current user, so world creators could implement their own system of tracking who sounds clear to whom (i.e. by who is inside or outside of a trigger rather than raycasts through walls).