We present SceneBind, an omni-modal representation of realistic scenes with joint semantic and 3D spatial understanding across Vision, Audio and Language. Existing omni-modal encoders excel at instance-level semantics, or what is present, but often lack explicit spatial structure, or where it is. SceneBind addresses this gap by representing each scene as a semantic-spatial entity, combining a global semantic embedding with object-centric semantic-spatial slots. This representation explicitly captures object-level semantics, spatial attributes, and uncertainty.
We further propose SceneBind Matching, a semantic-spatial matching scheme that integrates global scene similarity with object alignment, supporting cross-modal scene retrieval and object grounding. SceneBind is compatible with large-scale pretrained semantic encoders, adds lightweight spatial modeling with only a few additional tokens, and enables strong zero-shot transfer to downstream audio-visual localization.
Global measures whole-scene semantic similarity, Object measures semantic-spatial slot alignment, and Joint is the final retrieval score combining both signals.
A wide shot of a beach with waves crashing on the shore, buildings in the background, and a fishing rod in the foreground.
A woman plays the saxophone while a man plays the electric guitar on a stage with a red curtain backdrop and a banner.
A first-person perspective of walking along a paved sidewalk next to a stone-walled river on a cloudy day.
A bustling public square in front of a large church with bells ringing and a fountain with an obelisk.
1
2
1
2
1
2
SceneBind transfers to egocentric audio-visual localization without finetuning. It first proposes candidate sounding regions with global semantic attention, then refines the prediction using slot-level spatial consistency.
The sample viewer contains the full selected qualitative set, including query modality, retrieved top results, ranks, object slots, and audio playback.
@misc{chen2026scenebindbindingvisionaudio,
title={SceneBind: Binding What and Where Across Vision, Audio and Language},
author={Mingfei Chen and Zijun Cui and Ruoke Zhang and Hyeonggon Ryu and Eli Shlizerman},
year={2026},
eprint={2607.15265},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2607.15265},
}