SPHERE: Automatic Music Upmixing via Audio Language Model Post-Training with Spatial Heuristics Rewards

Zixun (Nicolas) Guo1,2, Calvin Murdock2, Sanjeel Parekh2, W. Owen Brimijoin2, Simon Dixon1, Joshua Reiss1, Ishwarya Ananthabhotla2

1Queen Mary University of London    2Meta Reality Labs
arXiv BibTeX
🎧 Headphones required — the 3D spatial effect renders only over headphones for binaural mixes.

Overview

Automatic music upmixing is a task where a system receives mono music stems and produces a spatial music mix. We approach this task through audio language model (ALM) post-training, where ALMs predict spatial mixing parameters (azimuth, elevation, gain) for each mono stem which adheres to verfiable spatial mixing heuristics.

Task overview
Figure 1. Automatic music binaural upmixing via an audio language model. Given the mono stems and mixing instructions (text), the ALM predicts per-stem azimuth, elevation, and gain; a binaural spatializer renders the immersive headphone mix.

Binaural Audio Example

Binaural audio reproduces 3D sound localization (e.g., azimuth, elevation, and distance) around the listener's head. It is designed to produce a more realistic spatial impression over headphones. The three clips below demonstrate the differences. 🎧 Put your headphones on and play them in order.

1

Dry audio

front LR
The audio is perceived at the center of the listener’s head.
2

Spatialized

front LR
The same audio is perceived at a fixed position in front of the listener, rather than within the head.
3

Moving around you

front LR
Now the audio travels from your left to your right.

Automatic Music Upmixing Outputs

Input mono stems are raw mono stems that the model receive. Then compare a stereo reference and binaural upmixes from the student model, the teacher model, and our post-trained SFT and SFT+RL models.

What to listen for:
  1. Width and immersion — are sources spread across the panorama rather than collapsed to the center?
  2. Level balance — is the overall mix well-balanced?
  3. Instrumental separation: assess how easily each instrument can be distinguished within the mix.
Segment 1 / 11

BibTeX

@article{guo2026sphere,
  title   = {SPHERE: Automatic Music Upmixing via Audio Language Model Post-Training with Spatial Heuristics Rewards},
  author  = {Guo, Zixun and Murdock, Calvin and Parekh, Sanjeel and Brimijoin, W. Owen and Dixon, Simon and Reiss, Joshua and Ananthabhotla, Ishwarya},
  journal = {arXiv preprint},
  url     = {https://arxiv.org/html/2608.30559},
  year    = {2026}
}