README.md 3.81 KB
Newer Older
Xucong Zhang's avatar
Xucong Zhang committed
1
2
3
4
5
6
# OpenGaze: Open Source Toolkit for Camera-Based Gaze Estimation and Interaction

Appearance-based gaze estimation methods that only require an off-the-shelf camera have significantly improved and promise a wide range of new applications in gaze-based interaction and attentive user interfaces. However, these methods are not yet widely used in the human-computer interaction (HCI) community.

To democratize their use in HCI, we present OpenGaze, the first software toolkit that is specifically developed for gaze interface designers. OpenGaze is open source and aims to implement state-of-the-art methods for camera-based gaze estimation and interaction.

Andreas Bulling's avatar
Andreas Bulling committed
7
<a href="https://perceptual.mpi-inf.mpg.de/" target="_blank"><img src="https://github.molgen.mpg.de/perceptual/opengaze_old/blob/master/imgs/logo_pui.png" height="80"></a><img src="https://github.molgen.mpg.de/perceptual/opengaze_old/blob/master/imgs/logo_osaka-u.png" height="80">
Xucong Zhang's avatar
Xucong Zhang committed
8
9
10
11
12
13

## Functionality

The toolkit is capable of performing the following gaze-related tasks:

* **Gaze Estimation**
Xucong Zhang's avatar
Xucong Zhang committed
14
Estimate and show a user's gaze on a screen in real time.
Xucong Zhang's avatar
Xucong Zhang committed
15

Xucong Zhang's avatar
Xucong Zhang committed
16
[![Demo](https://img.youtube.com/vi/aenp4ZWjBZo/0.jpg)](https://youtu.be/aenp4ZWjBZo "Gaze Estimation")
Xucong Zhang's avatar
Xucong Zhang committed
17
18
19
<p>&nbsp;</p>

* **Gaze Visualization**
Xucong Zhang's avatar
Xucong Zhang committed
20
Plot gaze direction in images.
Xucong Zhang's avatar
Xucong Zhang committed
21

Xucong Zhang's avatar
Xucong Zhang committed
22
[![Demo](https://img.youtube.com/vi/9Lujg3beiYI/0.jpg)](https://youtu.be/9Lujg3beiYI "Gaze Visualization")
Xucong Zhang's avatar
Xucong Zhang committed
23
24
25
<p>&nbsp;</p>

* **Personal Calibration**
Xucong Zhang's avatar
Xucong Zhang committed
26
Perform personal calibration and remap the gaze target on a screen.
Xucong Zhang's avatar
Xucong Zhang committed
27

Xucong Zhang's avatar
Xucong Zhang committed
28
[![Demo](https://img.youtube.com/vi/BjhZcRw4N-w/0.jpg)](https://youtu.be/BjhZcRw4N-w "Personal Calibration")
Xucong Zhang's avatar
Xucong Zhang committed
29
30
31
32
33
34
35
36
37
38
39
<p>&nbsp;</p>

## Installation
[Unix Installation](https://github.molgen.mpg.de/perceptual/opengaze/wiki/Unix-Installation)

## Use
[Command line arguments](https://github.molgen.mpg.de/perceptual/opengaze/wiki/Command-line-arguments)

## Citation
If you use any of the resources provided on this page in any of your publications, please cite the following paper:

Xucong Zhang's avatar
Xucong Zhang committed
40
**Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications** <br/>
Xucong Zhang's avatar
Xucong Zhang committed
41
42
43
Xucong Zhang, Yusuke Sugano, Andreas Bulling<br/>
Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 2019<br/>

Xucong Zhang's avatar
Xucong Zhang committed
44
45
46
47
48
49
50
51
@inproceedings{zhang19_chi,<br/>
title = {Evaluation of Appearance-Based Methods for Gaze-Based Applications},<br/>
author = {Xucong Zhang and Yusuke Sugano and Andreas Bulling},<br/>
doi = {10.1145/3290605.3300646},<br/>
year = {2019},<br/>
booktitle = {Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)},<br/>
abstract = {Appearance-based gaze estimation methods that only require an off-the-shelf camera have significantly improved but they are still not yet widely used in the human-computer interaction (HCI) community. This is partly because it remains unclear how they perform compared to model-based approaches as well as dominant, special-purpose eye tracking equipment. To address this limitation, we evaluate the performance of state-of-the-art appearance-based gaze estimation for interaction scenarios with and without personal calibration, indoors and outdoors, for different sensing distances, as well as for users with and without glasses. We discuss the obtained findings and their implications for the most important gaze-based applications, namely explicit eye input, attentive user interfaces, gaze-based user modelling, and passive eye monitoring. To democratise the use of appearance-based gaze estimation and interaction in HCI, we finally present OpenGaze (www.opengaze.org), the first software toolkit for appearance-based gaze estimation and interaction.}<br/>
}
Xucong Zhang's avatar
Xucong Zhang committed
52
53
54
55
56
57
58
59

## License

The license agreement can be found in Copyright.txt

You have to respect boost, OpenFace and OpenCV licenses.

Furthermore, you have to respect the licenses of the datasets used for [model training](:https://github.molgen.mpg.de/perceptual/opengaze/wiki/Model-training).