Add an option in the player select menu for toggling avatar blocking, beside mute and block, so they can be individually grey-man'd, or restored to visible if default blocking is on.
This would allow people to block out obnoxious avatars without having to completely block the player, as well as being able to see individuals they wish to see without expecting them to add you, while having default blocking enabled for performance reasons.