Global visual salience of competing stimuli

## Global visual salience of competing stimuli

Ricardo Ramos Gameiro & Alessandro Grillini & Peter König

.center[
<a href="https://mila.quebec/"><img src="../assets/images/slides/logos/mila-beige.png" alt="Mila" style="height: 4em"></a>
    
<a href="https://www.uni-osnabrueck.de/"><img src="../assets/images/slides/logos/uni-osnabrueck.png" alt="Universität Osnabrück" style="height: 4em"></a>
    
<img src="../assets/images/slides/logos/eu.png" alt="European Union" style="height: 4em">
    
<a href="https://www.nextgenvis.eu/"><img src="../assets/images/slides/logos/ngv.png" alt="NextGenVis" style="height: 4em"></a>
]

.footer[[alexhernandezgarcia.github.io](https://alexhernandezgarcia.github.io/) | [alex.hernandez-garcia@mila.quebec](mailto:alex.hernandez-garcia@mila.quebec) | [@alexhdezgcia](https://twitter.com/alexhdezgcia)] [![:scale 1em](/assets/images/slides/misc/twitter.png)](https://twitter.com/alexhdezgcia)

---

## Motivation

.center[Accurate computational models of visual salience are already available .cite[(Kümmerer et al. 2016)]]

.references[Kümmerer et al. [DeepGaze II: Reading fixations from deep features trained on object recognition](https://arxiv.org/abs/1610.01563). arXiv 1610.01563, 2016]

---

## Motivation

.center[Accurate computational models of visual salience are already available .cite[(Kümmerer et al. 2016)]]

.references[Kümmerer et al. [DeepGaze II: Reading fixations from deep features trained on object recognition](https://arxiv.org/abs/1610.01563). arXiv 1610.01563, 2016]

---

## Motivation

.center[Accurate computational models of visual salience are already available .cite[(Kümmerer et al. 2016)]]

.references[Kümmerer et al. [DeepGaze II: Reading fixations from deep features trained on object recognition](https://arxiv.org/abs/1610.01563). arXiv 1610.01563, 2016]

---

## Motivation

.center[Accurate computational models of visual salience are already available .cite[(Kümmerer et al. 2016)]]

.references[Kümmerer et al. [DeepGaze II: Reading fixations from deep features trained on object recognition](https://arxiv.org/abs/1610.01563). arXiv 1610.01563, 2016]

---

## Motivation

---

## Motivation

---

## Motivation

---

## Outline
### And main findings - spoiler!

1. [Data collection (eye tracking)](#datacollection)
--

2. [Computational model](#computationalmodel)
--

3. [Results](#results)
    * a. It is possible to measure the global salience of natural images
    * b. The global salience of images cannot be explained by their local salience properties
    * c. There is a strong spatial bias towards the left side
    * d. Task and familiarity with one image do not influence the first fixation
    * e. Images with larger global salience are explored for longer time

---

## Data collection
### An eye-tracking experiment

* 200 images
* 6 categories
* 49 participants
* 200 trials per participant
* 3 seconds per trial
* Record first fixation after stimulus onset

---

## Computational model
### Logistic regression

**Goal**: predict the location of the .highlight1[first fixation] (left or right) given a pair of images from the data.

#### Logistic regression for pairwise comparisons

* $u$: image on the left
* $v$: image on the right

$$
h_w(u,v) = p(u > v) = \frac{e^{w_u}}{e^{w_u} + e^{w_v}} = \frac{e^{w_u - w_v}}{1 - e^{w_u - w_v}}
$$

.conclusion[If the model fits the data, each learnt coefficient $w_i$ should reflect the likelihood that image $i$ is fixated first.]

---

## Computational model
### Design matrix

We have a data set of 49 participants $\times$ 200 trials ($N=9800$), where we know which two images of $M=200$ were presented, $x_u$ and $x_v$, and which image was fixated first.

---

## Computational model
### Design matrix

We have a data set of 49 participants $\times$ 200 trials ($N=9800$), where we know which two images of $M=200$ were presented, $x_u$ and $x_v$, and which image was fixated first.

* $t$: task
* $f$: _familiarity_
* $s$: subject ($K=49$)

---

## Computational model
### Validation results

.context[Logistic regression model trained to predict the location of the first fixation given two images.]

.conclusion[Since the model successfully learnt the behavioural patterns from the experimental data with very low overfitting and low variance, the learnt coefficients can be meaningfully used for further analysis.]

---

## Results
### a. Global salience of natural images

.context[The model learnt one coafficient for each image, which can be interpreted as its **global salience**.]

---

## Results
### a. Global salience of natural images

.context[The model learnt one coafficient for each image, which can be interpreted as its **global salience**.]

---