The friend grouping system is a nice start, but VRChat is a huge social platform, just like Steam or Discord.
Three groups of 32 is just not enough friend list management for me, there are LOTS of people I want to keep up with and like to visit.
I suggest more or less copying something tried and true that works... like Steam!
Steam allows you to create as many "groups" as you want (instead of just 3), and you can add as many friends as you want to any group.
You can also put one friend into multiple groups (which you can't in VRChat atm). There are cases for this!
To accommodate this change, because it'd get messy if you could create as many groups as you want, we should be able to minimize the UI for group lists even further by hiding thumbnails. An example image has been provided in the attachments.