1. Go to Settings, User Interface
  2. Set the background (in my case Kawaii Cupcakes)
  3. Restart VRChat
The background will get reset to the default one