Omnidirectional, or 360°, images pose a distinct challenge for models of human attention. The viewer is placed at the center of a full sphere, free to look in any direction, and the equirectangular projections we store them in introduce geometric distortion that standard vision models were never designed to handle. SalyPath360, presented at Electronic Imaging 2022, is a framework that jointly predicts saliency and scanpaths for this setting.
The context is that most attention research assumes a flat image viewed within a fixed frame. In 360° content, viewing behavior is different: exploration unfolds over a wide field, geometry is warped near the poles, and the sequence in which a viewer inspects the scene matters as much as which regions are salient overall.
Our idea is to treat saliency and scanpath prediction as complementary halves of a single framework rather than separate tasks. A saliency map captures where attention is likely to be drawn across the sphere, while a scanpath captures the temporal order of fixations, the actual trajectory of gaze. Predicting them together, with the omnidirectional geometry accounted for, gives a more coherent picture of how people explore immersive imagery.
This matters for virtual reality, streaming, and adaptive rendering, where anticipating both what and in what order a viewer will look can guide compression, foveated rendering, and content design.
Links
See the paper for the full methodology and results.