Baballonia Eye/Face Tracking#

This guide will help you set up Baballonia Eye/Face Tracking under Linux.

Requirements#

  • Baballonia and VRCFT.Avalonia
  • A supported eye or face tracking device.
  • Windows 10/11 dual-boot availability or separate Windows 10/11 machine to train your model if your device requires it.

You will need to boot into Windows and use Baballonia there in order to train any and all models as needed so that you can then transfer them over to your Linux filesystem. They are typically stored at %APPDATA%/Roaming/ProjectBabble/Models on Windows and can be loaded from wherever you like via the Babbalonia GUI.

Then you can start the Linux native Baballonia.Desktop and tell it to load your pre-trained model.

ℹ️ Note

GPU Acceleration currently does not work on the Linux native build of Baballonia, so you can disable it for now.

You’ll also need to either disable the one-euro filter option or ensure that values are set for each of its parameters, else Baballonia will not send out usable data.

Baballonia should show your tracking device in the camera dropdown. You may need to adjust cropping and brightness for the cameras as necessary.

VRChat#

Ensure Baballonia is set to provide ‘Eye’ or ‘Both’ tracking to VRCFT, alongside the ‘Native OSC’ option if desired, and then launch VRCFT.Avalonia. If successful, you should see some small transfers of data, assuming the VRCFT-Babble module is loaded in VRCFT.

For eye tracking to function correctly within VRCFT using the Babble module, you will need to manually edit its configuration file.

After VRCFT has been launched at least once and installed Babble module, navigate to ~/.config/VRCFaceTracking/CustomLibs/{uuid of babble module}. Inside this directory, locate and open BabbleConfig.json and change the line "IsEyeSupported": false to "IsEyeSupported": true.

Assuming all goes well, you should be able to see OSC data coming in with it enabled after launching VRChat under Proton. Avatar parameters should load into VRCFT.Avalonia and you should see both send/receive values for data rates at this point alongside live results within VRChat itself.

Resonite#

You can enable Resonite’s native Babble support by adding the -ForceBabble flag to its startup options. Until Resonite issue #5533 is fixed, the native implementation only supports lower face tracking. To enable eye tracking, install the ResoniteBaballoniaEyeTracking mod. Once installed, you only need to start Baballonia, then Resonite, and make sure the OSC ports match (under Devices → Project Babble).