Proposal for fixing Audio Filters (eg low-pass) support for AVPro
techanon
I believe it is time to get a solution to the most plaguing issue (in my opinion) of AVPro: Audio Filters
I do not think this is a "can't fix" situation. In this proposal I will go over the problem, considerations that have been discussed historically, and the solution I believe is feasible to implement with working sample code.
### Previous Cannys
1) https://feedback.vrchat.com/open-beta/p/986-avpro-player-ignores-lowpass-reverb-and-other-filters
### Crux of the issue
As previously discussed in above canny #1, the current way of handling the AVPro speaker component (type named
AudioOutput
) is that the component is implicitly created via AddComponent on the same game object. This has the critical drawback of being unable to respect any audio filters due to the DSP filter chain being _component order dependent_. The second half of this issue is that Unity has no runtime native way to change the ordering of components without fully reconstructing the references. This causes very obvious issues in regards to dependent scene references. (TCL's comment in canny #1 clarifies this as well)
### What has been considered
- Detecting and destroying/rebuilding known filter components after the implicit component is added.
This is bad because all scene references to those components would be lost (eg: UI Events or public Udon variables).
To avoid the lost components issue, a full scene search would be required in order to update the references which is costly and fragile.
- Placeholder components (shims) for each filter type that gets implicitly created after the implicit AudioOutput
This is bad because it does not allow users to reference the actual filter components in the inspector (namely an issue for UI Events)
- Allow adding the AudioOutput script manually in scene and have the speaker search for it before trying to implicitly add the component.
This is bad because it requires that the user import the AVProTrial package to be able to use built-in unity audio filters.
This needs to be compatible with situations where the user does not wish to import that package so the dependency is decoupled from the feature itself.
### Proposed Solution
A shim script that inherits from the AudioOutput class combined with a compiler flag for detecting if AVPro is present, and if not, then have a stub type of the _same namespace and type name_ which is _ONLY_ present in the sdk.
Then update the VRCAVProVideoSpeaker to check for the existence of the shim component before creating an implicit AudioOutput, using it instead if found on the game object.
This ensures that when the main shim script is loaded in editor, it is already a valid AudioOutput component and the component order is correctly retained so the Audio Filters will work correctly.
In the SDK, the script type will check for the existence of AVPro through a version define, and if NOT present, will enable the AudioOutput stub class of the same namespace.
An example of what it might be like: https://gist.github.com/techanon/41efc336604e148dde55862bff1778d9
I've tested this specific example in editor and it works there. Can't test in-client obviously.
Log In
TheMaskedMan00
Tell the developers this:
When it comes to audio filter components, they should look one parent above to see if there is an audio source/listener, and just affect that one. That way things like AVPro won’t just break because it adds the audio output on runtime.
Basically the audio source checks the components on its gameobject, and then looks at its first layer child(ren). And the children components will always go AFTER the ones on the main object. That way we can still order the components as we please, not break any references, remove the need to create stub or shim components, and it’s just more convenient.
Although this seems more like something Unity would have to implement themselves (I’m not 100% sure though)
Example
SpeakerL(gameobject[transform, vrc avpro speaker, audio source, vrc spatial audio source])
—Filter(gameobject, Child of SpeakerL[transform, audio reverb filter, audio low pass filter])
SpeakerL
—Filter
Then the order of components would be evaluated like this:
SpeakerL:
transform
Vrc avpro speaker
Audio source
Vrc spatial audio source
[Components added at runtime]
{Done, check children}
Filter (child of SpeakerL):
transform
Audio reverb filter
Audio low pass filter
{Done, no more children}
And this will run whenever a new component is added the main (parent) object.
techanon
TheMaskedMan00
This would be an implementation requirement of unity itself, but tightly coupling game objects like that is very fragile as game objects are much more loosely related internally than components are. Making a DSP chain dependent on the transform component is not a good idea imo. (the transform component is what holds the parent/child "relationships" between game objects). Not to mention it's not something VRChat would be able to reasonably do themselves since it'd require modification of the fundamental unity audio system design.
Really what Unity needs to do is enable the non-destructive reordering of components within a gameobject. That would solve this _entire_ problem. But that would also require moving to a newer major version of unity as well which won't happen for a while.
shrug
TheMaskedMan00
techanon
Even if we could re order objects, the speaker would still break as it is added as the last component, we would more benefit if when adding a component, we could specify what index the component should be, and be able to access information regarding what index components are already in, similar to transforms sibling index
Maybe an overload for AddComponent(int componentindex)
techanon
TheMaskedMan00
I agree about the specifying the index to add it at.
But in regards to the reordering of components, it would be VRChat that would use it. They would add the component and then reorder it so it's above all the filters. It would effectively be the same as explicitly adding the component at a the index specified, as you mentioned.
But it's all just wishful thinking at this point as it requires unity itself to implement internal changes to the engine. My initial proposal is a method by which VRChat wouldn't have to wait for Unity to add a feature to accomplish it.
Janoooba
Not only is this feature important for just vibes, but it's actually functionally important for the club scene, which is a huge part of VRChat's community and livelihood.
The workarounds currently employed are extremely fragile and require real life equipment and the abuse of multi-channel audio. It's just not feasible for the average world creator.
This needs to be addressed!
S
Sproutzee
Boost 👍
TheMaskedMan00
Bump.
I thought i was going crazy until I was pointed here. I've already spent hours trying to get these audio filters working for my world, only to now realize its currently not possible.