Front Camera Face Tracking for Android
TypicalZedF
Enabling face tracking on Android through the front camera is entirely feasible. The existing SDK already includes support for Eye Tracking, making it a compelling opportunity to leverage this capability, especially given the presence of the requisite sensors on most Android devices.
Log In
mackandelius
Couldn't you already do this if the face tracking app supported OSC and sent the correct messages (and worked in the background)?
Just find it very unlikely that VRChat would natively support face tracking on android, since they'd have to make their own face tracking software for it.
TypicalZedF
While it's theoretically possible, there's a notable challenge when it comes to mobile implementation of OSC (Open Sound Control). The majority of mobile OSC applications aren't designed to seamlessly run in the background on your phone; their primary purpose is to remain actively open while you enjoy content on another device, such as your Quest.
It's improbable that VRChat would develop their own face tracking software for Android in this context. The requirement for native support for face tracking on Android would entail significant effort, including the development of proprietary face tracking software, which seems unlikely for VRChat to undertake.
For a relevant comparison, you can take a look at Roblox's implementation of face tracking in the following video: https://youtu.be/LJWBZtjgBgI?t=21 . This might provide some insight into how face tracking could potentially work in the context of mobile applications.
mackandelius
TypicalZedF: OSC wise I expect apps to appear to address those issues, just like we have a few FBT apps on phones nowadays.
And to the later portions, I expected you to provide a counter-argument for how it could feasibly work, using Roblox as an example can work as a proof of concept, but they've got so many more resources than VRChat that it can only work as a rough proof of concept.
So I guess I will. There are a few open source face tracking programs, in theory VRChat could use one of those, however I find it unlikely that they would until after they start supporting face tracking and not just eye tracking.
TypicalZedF
mackandelius: Those FBT apps don't run in the background. They remain open with your phone on a stand somewhere pointed at your body while you use another device. Such as your quest. So I'm confused what you mean by "like a few FBT apps on phones nowadays"
mackandelius
TypicalZedF: My meaning was that VRChat adding OSC trackers made people develop dedicated apps for it.
While it isn't as obvious that it is possible, eventually there will be dedicated face tracking apps for android VRChat.
While people developing such apps will definitely have to fight against the garbage collector, if it works well enough then there will be options.