Thrilled to announce that our paper on self-supervised scanpath prediction for painting images has been published at a CVPR 2022 Workshop!
The Paper
“Self Supervised Scanpath Prediction Framework for Painting Images” addresses a unique challenge: how do we predict where people look when viewing art, when collecting eye-tracking data for every painting style is impractical?
The Challenge
Paintings present unique challenges for visual attention models:
- Diverse artistic styles and techniques
- Limited labeled eye-tracking data for artwork
- Different viewing patterns compared to natural images
Our Approach
We developed a self-supervised learning framework that can learn to predict scanpaths without requiring extensive labeled data. This is particularly valuable for cultural heritage applications where data collection is expensive.
Key Contributions
- Self-supervised pre-training strategy for scanpath prediction
- Domain-specific adaptations for artistic images
- Evaluation on painting datasets showing strong results
The Code
The implementation is open-sourced as SSLArtScanpath, a Barlow Twins–style self-supervised pipeline for learning scanpaths on paintings without labeled fixations.
Links
This work was a collaboration with colleagues from multiple institutions, and I’m grateful for the opportunity to present at CVPR, one of the top computer vision conferences.