@article{startsev2022evaluating,
  title={Evaluating Eye Movement Event Detection: A Review of the State of the Art},
  author={Startsev, Mikhail and Zemblys, Raimondas},
  journal={Behavior Research Methods},
  pages={1--62},
  year={2022},
  publisher={Springer}
}

Full paper is available via https://link.springer.com/article/10.3758/s13428-021-01763-7.

ETRA2022 tutorial slides are available via https://emdetectionmetrics.page.link/etra2022. IPython Notebook demo available here: demo/ETRA2022 Demo.ipynb.

ETRA2024 tutorial slides are available here. IPython Notebook demo available here: demo/ETRA2024 Demo.ipynb

python run_calc.py

1. Download Hollywood2EM (Agtzidis et al., 2020) dataset from https://gin.g-node.org/ioannis.agtzidis/hollywood2_em

2. Run the following to convert dataset to the required format:

python misc/scripts/run_data_parse.py -root DATASET_ROOT -dataset hollywood2em --coder expert 
python misc/scripts/run_data_parse.py -root DATASET_ROOT -dataset hollywood2em --coder alg 

DATASET_ROOT is the directory where dataset was downloaded

3. Run evaluation script:

python run_calc.py  -job assets/job_hollywood2.json

The result file will be save to ./results/job_hollywood2.csv. This can take around 3 hours or more, depending on the computer

4. Run data analysis script:

python misc/scripts/analyse_results.py

Resulting plots will be saved to ./results/job_hollywood2/result-plots

To get matcher example plots run:

python run_calc.py -job assets/job_plot.json

Resulting plots will be saved to ./results/match-plots

List of publicly available annotated datasets to illustrate the variety of readily available material for algorithm development and evaluation. Duration reflects the amount of unique eye-tracking data (uniqueness judged based on file names); duration in parentheses -- the total amount of available annotated data (including undefined samples and taking into account several available annotations for a single recording). Sample distributions do not list proportion of undefined samples and samples annotated as noise, blinks and similar. Note that datasets might have different definitions of fixations, saccades, and other events.