VRChat SDK doesn't gather Materials from SetMaterial action of inactive GameObjects to DynamicMaterials of VRC_SceneDescriptor.
Then, SetMaterial fails in some cases.
Reproduce step:
  1. Crate an GameObject, add VRC_Trigger, add OnInteract trigger, add SetMaterial action, select a material which doen't used by other objects.
  2. Open VRC_SceneDescriptor inspector, set size of DynamicMaterials to zero.
  3. Build and test the scene. It works fine. And now DynamicMaterials contains the material selected in step 1.
  4. Inactivate GameObject crated step 1.
  5. Again, open VRC_SceneDescriptor inspector, set size of DynamicMaterials to zero.
  6. Build and test the scene. This time, DynamicMaterials dosn't contain the material.
  7. Create another object, add VRC_Trigger, add OnInteract trigger, add SetGameObjectActive to change inactivated object above to active.
  8. Build and test the scene. Interact the object of step 7, then the object of step 1 appears, Interact that, Then it fails to SetMaterial.
Workaround:
Duplicate object of step 1, set active, hide to somewhere.