Predicting where people look in an image is a deceptively hard problem: attention is shaped by both fine local detail and broad scene structure. In SATSal, we introduce a saliency-prediction architecture that weaves self-attention through multiple levels of a deep network, letting the model reason about visual importance at several scales at once. This is one of the works I am most attached to, and it appeared in IEEE Access.
The challenge with conventional saliency models is that convolutional features are inherently local. They capture texture and edges well, but they struggle to relate distant regions of an image, which is often exactly what determines whether something draws the eye. A salient object may only become salient in relation to its surroundings.
Our core idea is to insert self-attention at multiple levels of the feature hierarchy rather than only at the top. Early layers refine low-level cues, while deeper layers integrate more abstract, global context. By attending within and across these levels, the network can align local saliency estimates with the larger scene, producing predictions that better reflect human fixation behavior.
Why does this matter? Saliency models feed into image compression, cropping, quality assessment, and interfaces that adapt to human attention. A model that captures both local and global structure gives those downstream systems a more faithful account of where human attention actually goes.
Links
See the paper for the full methodology and results.