DissLiteratur/storage/JFHULEXJ/.zotero-ft-cache

Journal of Physics: Conference Series

PAPER • OPEN ACCESS
A Review of Main Eye Movement Tracking Methods
To cite this article: Zhaowei Li et al 2021 J. Phys.: Conf. Ser. 1802 042066
View the article online for updates and enhancements.

You may also like
- Eye tracking technology in sports-related concussion: a systematic review and meta-analysis N Snegireva, W Derman, J Patricios et al.
- Real-time inference of word relevance from electroencephalogram and eye gaze M A Wenzel, M Bogojeski and B Blankertz
- Blink duration is increased in concussed youth athletes: a validity study using eye tracking in male youth and adult athletes of selected contact sports Nadja Snegireva, Wayne Derman, Jon Patricios et al.

This content was downloaded from IP address 87.146.225.238 on 12/09/2023 at 12:56

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

A Review of Main Eye Movement Tracking Methods

Zhaowei Li1, *, Peiyuan Guo2 and Chen Song3
1School of Engineering, Washington University, Seattle, US 2School of Computing, Beijing University of Posts and Telecommunications, Beijing, China 3School of Computing, Southwest University of Science and Technology, Mianyang, China
*Corresponding author: lzw365@cs.washington.edu
Abstract. The study of eye movement has been last for centuries. Over the past decade, scientists have developed many new eye-tracking methods. During that same period, technological innovations have made eye-tracking units more affordable. The article provides a systematic review on some main eye-tracking methods. This paper also includes the factors that may influence the eye-gaze tracking and the application of gaze estimation in many areas. In this article, we provide information on eye tracking in hopes of improving the quality and reporting of eye-tracking research in communication.

1. Introduction Vision is one of our important feelings, and our perception of the outside world is mainly achieved by our eyes. Eye-tracking is a technology that can understand people's gaze position and how people observe. Through technologies, the fixation point of the eye can be located, and tracking that point can obtain information, which can reveal our behaviors, even our thought. Generally, it is to track the movement of eyes and understand what a person is looking at.
There is a lot we can do with eye-tracking technology. About 80% of external information is received by the vision [1]. Eye movement can also reveal one's thoughts and mental activities. It can be said that studying a person's eye movement can understand his way of thinking. Due to the characteristics of the physiological structure of the human eye, eye tracking can be used to understand which elements attract people's attention, where people focus on, and what elements they are used to ignoring. Through eye tracking, we can understand ourselves and human beings better.
As we know, human vision is divided into two parts: a small central area with extremely high resolution, namely the central fossa, and most other areas with lower resolution. The main function of eye movement is to move the information we are interested in the area of the central fossa and to allocate our attention resources to the information that is useful and attracts us. So, when we design eye-tracking devices, we only need to pay attention to what people observe in the central fossa area.
The main ways of eye movement are gaze, saccade, following smooth movement, blinking, and vestibular eye reflex [2-5]. People obtain information mainly by gazing, to present a clear image on the central fossa. When our focus changes, the central fossa's eye movement as the focus moves from one point to another is called a saccade. The process of the saccade is very short, so no more effective visual information will be obtained in this process. Hence, gazing and saccade is the focus of various methods. The principles of eye tracking technology are mainly based on the physical structure of the eye.

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution

of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

Published under licence by IOP Publishing Ltd

1

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

Nowadays, it will incorporate the use of other technologies, such as optics, microcontrollers, computers, neural network, etc., which improves accuracy and practicality of eye tracking. Modern eye trackers use a more ingenious method. For example, the pupil-corneal reflex tracking method, the most commonly used one, uses camera to obtain pictures or videos of eye. Then algorithms are applied to extract the coordinates of the pupil center and the corneal reflection spot, which are used to calculate the fixation point. This method is a good example of the combination of the optical structure principle of the human eye and computer technology. There are many other methods, and they have their own advantages and disadvantages, they will be discussed in the methodology section.
Common indicators for eye-tracking technology are heat map (distribution of fixation points), fixation position, interesting area, pupil diameter, fixation time, fixation sequence, saccade distance, saccade path, saccade duration, etc. Different indicators can provide different information [6].
To design eye-tracking devices more effectively using eye-tracking technology, we need to consider many factors such as head posture, blinking, the clothing is worn, Midas contact problem [7-8], Multichannel problem [9-11].
In the review, we will introduce some methods that underlie the eye-tracking technology, including their principles, advantages, disadvantages and outstanding contributions from many researchers. It can provide an overall picture in this field, which is useful for the beginners. And then, the application scenarios have also been reviewed. As for the future development trend, we believe that with the development of science and technology, eye movement recording technology is continuously improved and updated to make it more convenient, accurate and fast, especially in combination with computer technology. In the future, eye tracking will be applicable in more fields.
2. Method Overview The eye-tracking can be traced back to the 19th century. Early eye movement research and reading research are closely related. At that time, people mainly recorded through observation. Later, with the development of technology, mechanical recording methods appeared. Then other methods are also developed, such as electrooculography, video and photographic, corneal reflection and so on. These methods will be demonstrated below.
2.1. Electrooculography (EOG)

Figure 1. Front Panel of the System(a), Looking left(b), Looking right(c), Looking Left(d) [13].
2

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

In 1849, Du Bois-Reymond found that there is a relationship between eye movements and the electrode potential on the surface of the human skin [12]. This potential relationship is caused by the potential difference between the cornea and the retina of the eye. When the eyeball continues to move, the potential difference will continue to change, and the electrooculogram signal will also continue to change, forming a changing curve on the time axis. This curve is called electrooculography (EOG). The EOG method was proposed by Fenn and Hursh in 1934 [13], as shown in Figure1. Eye movement can produce bioelectricity. The cornea is positively charged, and the retina is negatively charged. There is a certain potential difference (corneo-retinal potential difference), about 0.4-10mV [14], and sensitive current can be recorded [15]. When the eye gaze does not occur in front of the eye movement, a stable baseline potential can be recorded. When the eye moves in the horizontal direction, the potential horizontal difference of the eye will change; and when the eye moves in the vertical direction, the potential vertical difference will change. Placing two pairs of skin surface electrodes on the left and right, upper and lower sides of the eye can cause weak electrical signals in the direction of the eyeball change, and the position information of the eyeball movement can be obtained after amplification. Because there is a linear relationship between the rotation of the eye and the change in potential, these recorded data can simply determine the movement of the eye. Miles studied the effect of different conditions on the corneal-retinal potential difference and found that the potential difference caused by light adaptation increased, while the potential difference caused by dark adaptation decreased. The recorded potential change is usually less than 1 mV [16]. Researchers used this method to perform reading fatigue experiments. Participants read continuously for 6 hours and recorded the horizontal and vertical movement of the eye during a reading with electrodes. All eye movements are recorded on an ink recorder [17]. In 1955, Woodworth recorded the movement in the horizontal direction [18]. A better EOG method was proposed by Ford, White and Lich-tenstein, and this method can simultaneously record the horizontal and vertical movement of the eye [19]. The biggest advantage of this method is that the movement of the head does not affect the recording of eye movements, and there is no direct contact with the eyes during the recording. The shortcoming is that there are many factors that affect accuracy, including eyeball rotation and movement, electrode placement, etc. Each person's potential difference is different, and due to the poor conductivity of the dead skin layer, which should be removed with sandpaper before the test. What this method actually records is not the movement of the eyeball, but the relative value change of the potential difference, so further conversion is needed to obtain the actual situation of the eyeball movement. Shackel suggested that this recording method has three problems: First, the EOG signal is too weak, which causes difficulties in the recording; Second, the frequency range of the skin potential is the same as the frequency range of the EOG signal; Third, the poor contact between the electrode and the skin will affect the accuracy of results [20].
Generally speaking, the current recording method is relatively good, relatively cheap, high time resolution and can work in real-time. However, the accuracy is low, and the error may exceed 1 °.
2.2. Video and Photographic Method The video and photographic method mainly use the camera to record the light reflected by the cornea to record the eye movements, and the head is usually fixed. When using this method, the key is to find a reference point, and this reference point and the continuous movement of the eye are photographed together for analysis. Dodge and Cline used a camera to record eye movements for the first time. They used the camera to take a series of eye movement photos and analyzed reflections of the eyes in the image to obtain the line of gaze, to understand the eye movement of examinees [21]. Judd, McAllister and Steele continuously shot the examinee’s eyes and face (9 frames per second) in the experiment. They first painted a small spot of white pigment on the cornea of the examinee. Examinees need to fix their heads. There are two small bright balls on the stent, which are used as the reference point for photography. The photographs of the relative position between the reference point and the white point of the cornea are recorded to analyze the eye movement of the examinee [22]. Some researchers photographed the examinee’s eye movement behind a translucent mirror, and the examinee looked at the object through the mirror. In this method, the instrument is placed behind the mirror, so the examinee

3

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

is not aware of it, which avoids the distraction caused by the instrument [23]. Some researchers spotted a 0.1 mm drop of mercury on the edge of the cornea and a drop of mercury of the same size on the forehead. During the experiment, the examinee's head was fixed, and the mercury on the forehead was used as a reference point. The images of relative mercury movement were recorded with photographs [24]. Some researchers recorded 26 times magnified scleral blood vessel image, illuminating the blood vessel with UV light for easier recording [25]. J. F. Mackworth and N. H. Mackworth invented a television eye-tracking method. They used a TV camera to shoot objects, and at the same time import it to two TV screens for projection. The examinee observed the scene on the first screen. His cornea of eyes is illuminated by light. The reflection of the cornea was photographed by another TV camera and introduced into the second screen. So, on the second screen, there will be light spots of eye movements and objects. The head of the examinee is fixed during the experiment. This device records a minimum eye movement of 1°, which is not suitable for recording smaller eye movements [26]. Shackel installed a camera on top of the examinee's head so that the examinee can observe objects in any direction freely. Attach an electrode next to one of the eyes of the examinee, record the potential changes during eye movements by electrooculography, and present it on the oscilloscope; use a second TV camera to attract the oscilloscope's light spot movement to show the objects on TV screen. Therefore, there are both the observed scene on the screen and the movement track of the light spot representing the eye movement [27]. AR Shakhnovich and VR Shakhnovich described another device in which the images seen by two examinees are projected into the plane of the aperture of a photo kymograph. A compensating prism rotated in front of this aperture and displaced the image in a direction perpendicular to the aperture. In this case, start to scan the pupil and store its diameter on the film. Both the vertical and horizontal diameters of each pupil can be projected in the plane of the aperture of the optic system. Both horizontal eye movement and vertical eye movement are recorded on film [28]. Mackworth and Thomas also used the method of installing a camera overhead, but this time the examinee had a special dual-lens film camera, one lens used to observe the scene in front, and the other lens to observe the left side of the examinee by refraction. In this way, there are both the observed objects and the movement trajectory of the left eye [29]. Llewellyn-Thomas et al. improved the eye movement recording device of Mackworth. Corneal reflection controls the movement of the light spot on the TV screen, which is the point of gaze. The movement of the light spot is recorded by a teletype machine controlled by a group of photocells. When the examinee changed the gaze area, the stimulus in the area was a blackout, creating an artificial central scotoma. The sensitivity and accuracy of this device are not high [30].
The advantage of this method is that the operation is relatively simple and accurate for measuring large eye movements. However, because the eye movements are recorded in photos or images, it is timeconsuming for data processing, labor-intensive, and error-prone.
2.3. The Corneal Reflection Method The corneal reflection method utilizes the characteristics of corneal reflection. If the eyeball is regarded as a standard sphere rotating with the center of the circle, the reflection point is fixed. However, the cornea protrudes from the surface of the eyeball, so when the eyeball moves, the reflected light will be emitted at a varying angle. The virtual image formed on the cornea surface moves due to the movement of the eyeball. Therefore, when an infrared LED light source and a camera fixed in front of the subject's head are placed in front of the human eye, the light reflected by the cornea is transmitted to the camera through the beam splitting device in front of the eye and some mirrors and lenses. The same device is placed in front of the other eye. The position of the cornea reflecting light is determined by the image fixed on the camera screen in front of the head and some corresponding algorithms. This allows eye movements to be analyzed by recording corneal reflections. Dodge and Cline were the earliest people who used corneal reflection to record eye movements [21]. Dodge recorded changes in corneal reflected light on a falling photographic plate. He studied eye movements during gaze, following, and reading [31]. Someone recorded the movement of corneal reflection on a stationary photographic plate. Under the condition of excluding all other light sources, the corneal reflection was extracted and the eye movements were recorded by photography. Stratton used this method to study the eye movements of

4

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

the subjects when observing complex geometric figures [32]. The corneal reflection method has since been improved, and many specific instruments have appeared. An instrument specializing in eye movement during gaze, they designed a helmet-type eye tracker to study gaze. The eye tracker was designed as the shape of a hat made of alabaster. The light source and associated optical instruments are above the hat, which is fixed on the head of the subject. The light source and corneal reflections were captured simultaneously by a movie camera with a frequency of 60 frames per second [33]. Lord and Wright also developed a photoelectric method for recording corneal reflex motion. During the experiment, the subjects lay on their backs with their heads tied to a special headrest. The subjects bite a plate fixed on the headrest. The cornea is irradiated with ultraviolet light with a wavelength of 365, and the reflected light falls on a translucent aluminized mirror. The light beam falling on the mirror is divided into two parts. One is on a vertical screen and the other is on a horizontally placed screen. There is a photocell behind each screen to monitor the corneal reflection's changes in the vertical direction and horizontal direction. The current from the photoelectric cells was fed into a DC amplifier, and then into a cathode-ray oscilloscope. The eye movement can be inferred from the oscilloscope [34]. JF Mackworth and NH Mackworth use television techniques to record eye movements. The television camera captures the light reflected by the cornea, and the movement trajectory of the light spot is enlarged by 100 times. With proper calibration, the accuracy can be between one and two degrees [35]. Later, Johanson and Backlund installed an eye tracker based on the principle of corneal reflection on the head of the subject. The instrument was very light and did not cause the subject to feel uncomfortable [36].
The corneal reflection method is an important method of eye movement recording. The advantages of this method are not invasive, so it is comfortable for the users to use. The disadvantage is that it is not accurate enough to study large eye movements. If the eyes are wet, the results will be inaccurate.
2.4. Pupil-corneal reflex tracking

Figure 2. Example of relative position of pupil and corneal reflex [36].

Figure 3. The relative position of pupil and corneal reflex. By shining infrared light on the eye, the infrared light source can improve the contrast between the edge of the pupil and the iris, as well as create a reflection on the inner and outer surfaces of the cornea and lens. When light from the outside enters the pupil of the eye, the surface of the cornea reflects and forms
5

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

light. When the eye moves, although the Angle of light entering the eye changes, the position formed by the light spot does not change and can be used as a fixed point. On the contrary, the center of the pupil will always follow the direction of gaze. The vector made up of the center of the pupil and the bright spot of the cornea changes with the eye movement and can be used to analyze the direction of vision. The previously mentioned Dodge and Cline also used this method. A video camera or other optical receiver is used to record and receive infrared rays reflected back from the eyes. The analysis and processing of video data are done by computers using software and programs. The optical elements of the system have a fixed distance from the subject's eyes, and the reflected image is recorded by a camera. The data obtained by the camera is processed by computer or microprocessor, and the pupil and cornea in the eyeball image are extracted by using the principle of bright pupil and dark pupil. Corneal reflex was used to correct the relative position between the camera and the eyeball. Then the corneal reflex point data is used as the base point of the relative position of the eye camera and eyeball, and the gaze point in the screen space is calculated according to the position coordinate of the center of the pupil (the position of the eye). In 1998, Morimoto et al. installed two sets of infrared lights near the camera, one on the near axis and the other on the far axis. They used video acquisition of the odd and even field circuit to control the two groups of infrared lights, to obtain bright and dark pupil images. By placing the infrared (IR) light source near the camera's optical axis, the image with a bright pupil can be obtained by using the reverse reflection characteristic of the eye [37]. Based on pupil and corneal reflection of video recording of the eye-movement apparatus, mainly have the SR Canada Eyelink Ⅱ type and 2000 Eyelink eye movement device, a Swedish Toii company T60 and T120 eye-movement apparatus and so on. According to the way of use can be divided into helmet type, telemetry type and head fixed. Helmet means to attach a camera or video camera to the head or helmet. Telemetry means the installation of a camera or video camera at a distance from a person. Head fixation is the fixation of the head before use to ensure accuracy.
This method is relatively accurate and has no interference to people, and is easy to master and use [38]. It is by far the most widely used method, which also takes advantage of the corneal protrusion.
2.5. Dual-Purkinje-Image Method

Figure 4. The four Purkinje image formed by the eye.
The Czech physiologist Purkinje published his second book on the study of sensation in 1925. The book mentions a study in which visual adaptation is the result of initial changes in the lens of the eye. Purkinje measured changes in the glow reflections from the cornea, the anterior crystal surface, and the posterior crystal surface as subjects focused their focus on near and far objects.
Purkinje image is an image formed by the reflection of several optical interfaces of the eye. The image reflected from the anterior surface of the cornea is the first Purkinje image, while the image reflected from the posterior surface of the cornea is weaker and is called the second Purkinje image. The two are almost identical. The image reflected from the anterior surface of the lens is called a third Purkinje image. The image reflected from the posterior surface of the lens is called the fourth Purkinje image. However, the brightness of the fourth Purkinje image was 1% less than that of the first Purkinje image because the refractive index change at the back of the lens was much less than that at the air-
6

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

corneal interface. If the eye is shifted (for example, the head is moved laterally), the first and fourth pinfield images move together for exactly the same distance. However, if the eye rotates, the two images will move a different distance because the curvature centers of the two mirrors are not the same, thus changing the distance between them. Thus, the separation of the two images produces a measure of angular orientation. By monitoring the spatial separation of the two images, eye rotation can be accurately measured without confusion caused by the translation. Similarly, eye translation can be accurately measured without confusion due to eye rotation. Regardless of how the pupil moves in the orbit, the Purkinje image is relatively fixed to the eyeball. The first and fourth pin-field images can be measured by dual-purkinje eye tracker, which is one of the most accurate trackers [39]. Generation -v eye movement tracker uses this method, and it is a combination of an early dual-purkinje-image eye tracker (used to measure the horizontal and vertical movement of the eyes) and an optermeter (used to measure the refractive power of the eye). It not only reduces the volume of the eye tracker to 1/5 of the original volume, but also reduces system noise to 20-sec of arc RMS, which affects the final resolution of the eye tracker [40].
Although DPI (dual-Purkinje image) eye movement instrument accuracy is high and does not need to have any contact with the eyes, it still needs the head to be fixed. The second and third, fourth and Purkinje-image are very weak. The requirements for laboratory light and equipment are very high, so the price is high, and this method is sometimes not universal.
2.6. Eye Movement Tracking Method based on Micro Electro Mechanical Systems (MEMS) With the emergence and development of the micro-electromechanical system (MEMS) in the 1970s, many technologies have been combined with micro-computer technology to bring about new vitality, and eye movement tracking technology is no exception [41]. Microcomputer technology, as its name implies, uses microelectronics technology, integrated circuit technology and processing to produce nanoscale machinery so that the original mechanical components greatly reduced. The microcomputer technology has been combined with drivers and sensors to produce a variety of MEMS drivers and sensors. They are small, energy-efficient [42], and conductive to large-scale production. Thus, they enhance the prevalence of eye-tracking technology and the improvement of its practicality. In the following, we will introduce two representative applications of mems eye-tracking technology.
A patented MEMS eye-tracking system from Apple consists of a MEMS driver, a tiny camera, and a processor. It uses two MEMS drivers to control the camera's viewing direction so that it is always aimed at the eye to capture an eye image. One MEMS driver is connected to the rear end of the camera, which is responsible for the horizontal movement, and the other is connected to the upper or lower end of the camera to control the movement in the vertical direction [43].

Figure 5. MEMS-based eye tracking principle [44]. The University of Waterloo in Canada has opted for a technology-based entirely on MEMS devices, which convert larger cameras into smaller ones. It uses corneal reflex technology. It emits low-emission
7

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

light through a mechanical device to a 300nm scanner with a MEMS drive, which is then reflected the cornea and then reflected back to a light-emitting diode (led) that converts light signals into electrical signals. The electrical signal increases with the increase of the optical signal. The scanner-controlled beam tracks the point on the cornea where the photodiode can receive the maximum signal, thereby realizing the eye movement tracking of the system (original words) [45].
We can see that the computer technology and the traditional corneal reflection method, film and television method combined with each other, make full use of the advantages of the computer technology, resulting in more portable and more practical, and more convenient manufacture of eye-tracking device.
2.7. Eye movement tracking method based on Convolutional Neural Networks (CNN)

Figure 6. CNN function flow diagram [46].
As a kind of Feedforward Neural Networks with deep structure and convolution computation, convolutional Neural Networks is a kind of deep learning algorithm with great influence. The convolutional neural network simulates the construction of biological visual perception mechanism, which can be used for supervised learning and unsupervised learning. The sharing of convolution kernel parameters in the hidden layer and the sparsity of inter-layer connections enable the convolutional neural network to feature the grid-like topology with a small amount of computation. It has the characteristics of local perception, weight sharing and multi-convolutional kernel. CNN does not need to manually extract features as before, and it has a certain degree of invariance for common image deformation. All these features provide great advantages for the convolutional neural network model in image recognition. In 1987, the first Convolutional Neural Network was born, a Time Delay Neural Network (TDNN) proposed by Alexander Waibel et al., and was applied to speech recognition [47]. Since then, a convolutional neural network has been widely used in computer vision and natural language processing, and has demonstrated its advantages in image recognition processing, which is exactly what eye movement recognition and tracking technology needs.
Deep learning needs a large number of data sets as the foundation to play its role. Therefore, rapid and efficient eye recognition technology is also the key to the development of this eye-movement tracking technology. We take the commonly used Viola and Jones AdaBoost face detection algorithm as an example. They use integral graph to reduce the calculation of haar-like features (for identifying the digital features of objects in the image, the difference between the pixel sum of black matrix and the pixel sum of white matrix in two or more matrices in the image). It is also an iterative classification algorithm by dividing the original training set into many small training sets. Many different weak classifiers are obtained by adjusting the sample weights, and several strong trainers are formed by proportionally combining each other to improve the accuracy and speed.
8

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

With the original image as the input, the convolutional neural network automatically learns significant features to express the image after certain pre-processing (the convolutional neural network method focuses on the learning of the filter, so there are fewer pre-processing steps).The recognition process is basically that the eye movement data collected will be divided into two groups: training set and test set. After normalization of the two groups of data, the training set learns the training, automatically learns the features, and then performs feature dimensionality reduction to establish a neural network model, and carries out recognition test on the test set, and finally outputs the recognized results [48]. In the design of convolutional neural network algorithm, the structure design of the network is very important. Improper design will lead to non-convergence. At the same time, as far as possible in order to reduce the interference of noise on the error of the result, we need to choose a suitable vector (impact error convergence speed), the activation function (to deal with nonlinear separable problem, affect the convergence and performance), convolution kernel size and number (should not be too big also should not be too small), the convolution number (words will affect classification result is too small, too big words can make the image information loss).However, in the actual scene, the processing of the images is not easy. There are many external factors that will affect the accuracy of eye movement recognition, such as the intensity of light and the uncertainty of distance and position in the image acquisition process.
Eye movement tracking technology based on CNN has been further developed with the rapid development of mobile devices. A large number of smart phones are used, the improvement of camera technology in mobile devices and the relatively fixed position of cameras can all help to provide rich eye movement data [49]. With the popularization of high technology levels, convolutional neural network can be popularized [50], which all provide convenience for the popularization of eye movement tracking technology based on CNN. It can be seen that the eye movement recognition technology based on CNN has the advantages of reidentification performance and Robustness. Currently, there are many products based on this technology.

2.8. The summary of other methods Besides the methods mentioned above, there are also some other methods that used in the eye-tracking field, whose characteristics, including the advantages and drawback, have been listed in Table 1.

Table 1. The summary of advantages and drawback of other eye-tracking method.

Method observation mechanical recording method Scleral Search Coil method Electrooculography (EOG) Mirror recording method Video and photographic method The Corneal reflection method
Limbus tracking
Pupil-corneal reflex tracking Dual-Purkinje-image method
Eye movement tracking method based on MEMS
Eye movement tracking method based on CNN (Convolutional Neural Networks)

Advantages easy operation
relatively high accuracy high time resolution
relatively high accuracy relatively high accuracy measuring large eye
movements non-invasive
easy to process electrical signal output high accuracy, non-invasive
very high accuracy
low power consumption, small volume, low cost, high production rate, good for mass production, easy to integrate. Better recognition performance and robustness.

Drawback low accuracy sophisticated operation, low accuracy, highly invasive highly invasive, expensive low accuracy highly invasive
sophisticated data processing
low accuracy low accuracy, especially in vertical
direction
need to fix head position, high requirement for equipment and
environmental lightings.
sophisticated design
need large amount of data, complex adjustment of parameters, "black box"
characteristics, high cost.

9

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

3. Related Work and Applications In recent years, with the development of hardware and software, eye-tracking technology has been combined with many other technologies, and applied to many fields, such as psychology, VR/AR, human-computer interaction, etc. Its inherent potential is to attract large technology companies such as Google, Apple, Samsung, Oculus, etc. At the same time, some mature companies such as Tobii and SIM have appeared in the application of eye-tracking technology [51]. The volume of the eye tracker is continuously decreasing, and the degree of freedom of the subject's body or head is also continuously enlarged during the test. It also becomes safer and reduces direct eye contact. This shows that eyetracking technology has a very bright prospect in the future technology market. Next, we will give a more comprehensive introduction to the related work and applications of eye-tracking.
3.1. Medical Field Eye-tracking can help doctors diagnose many eye-related diseases, such as autism, concussion and so on [52]. Eye-tracking is also used in many related research fields in psychology and neuroscience to study the relationship between visual and behavioral feedback based on information processing [53].
3.2. Commercial Field Eye-tracking can help ad designers discover which elements attract consumers? What is the focus of consumer attention? Therefore, eye tracking is widely used in: packaging design research, advertising research, web design, improving web page loading speed (first loading may be of interest to users) and other fields.
Eye-tracking can also improve work efficiency, analyze the factors that may affect work efficiency in the work environment from the perspective of workers, find distractions and invalid work procedures during operation, and optimize mechanical equipment and work environment. By studying the eye movements of the top people in the industry at work, you can improve the training program and apply it to sports, competitive games and many other fields.
The gaze is the main way for customers to interact with products and services. Through eye tracking, these interactive data can be obtained directly from the user's eyes, analyze the data, and convert the results into quantitative data forms. Unlike traditional market research, eye tracking will have a bigger market.
3.3. VR Field The combination of eye-tracking and VR technology can be achieved through visual-based interaction [54]. The world is selectively focused by the human eye. The focused center is clear and the center is gradually blurred outward. Therefore, the gaze point rendering technology, which focuses on the image rendering of the eye condensation center, can reduce the load of the graphics card and the CPU, increase the processing speed, more in line with the observed characteristics of the human eye, and make the user more immersed in it [55].
3.4. Game Field In the field of games, eye tracking is convenient for game developers to collect data. Compared with the traditional information collection method, as long as some people are equipped with eye trackers, they can achieve the same or even better results. This simplifies the game's UI interface, provides space for other game elements and enriches the player experience. In the future, how to communicate and cooperate with the NPCs in the game through the eyes and how to aim directly through the eyes [56] in the shooting games is a very worthy research direction.
3.5. Study Field The dissipation of eyes is a universal phenomenon. If the phenomenon can be avoided during learning, the efficiency of a class can be greatly improved. Collecting class data of students through eye trackers

10

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

and analyzing the results can effectively design, evaluate and improve the course. It can also be used as a monitoring method for remote classes. In addition, eye tracking can also improve reading skills [57].
3.6. The field of human-computer interaction Nowadays, eye-tracking technology provides us with the possibility of new human-computer interaction [58]. More and more eye tracking technology is applied to mobile phones and computers. Vision is one of the most commonly used senses in humans, and its gaze is one step ahead of any physical movement. This new type of human-computer interaction will certainly have a huge impact on the future of computers.
4. Conclusion In this review, we provided a systematic review of some main eye-tracking methods and their applications. And we hope it will be useful to researchers and reviewers to make decisions about eyetracking research practice and its subsequent evaluation. Clearly, eye-tracking methods will continue to gain prominence among many fields, such as vision studies and cognitive science. We believe there will be more creative eye-tracking technology discoveries and applications in the future.
References [1] Lai, Meng-Lung, et al. "A review of using eye-tracking technology in exploring learning from
2000 to 2012." Educational research review 10 (2013): 90-115. [2] Jacob, Robert JK, and Keith S. Karn. "Eye tracking in human-computer interaction and usability
research: Ready to deliver the promises." The mind's eye. North-Holland, 2003. 573-605. [3] Salvucci, Dario D., and Joseph H. Goldberg. "Identifying fixations and saccades in eye-tracking
protocols." Proceedings of the 2000 symposium on Eye-tracking research & applications. 2000. [4] Larsson, Linnéa, et al. "Detection of fixations and smooth pursuit movements in high-speed eyetracking data." Biomedical Signal Processing and Control 18 (2015): 145-152. [5] Pander, Tomasz, et al. "An automatic saccadic eye movement detection in an optokinetic nystagmus signal." Biomedical Engineering/Biomedizinische Technik 59.6 (2014): 529-543. [6] Mele, Maria Laura, and Stefano Federici. "Gaze and eye-tracking solutions for psychological research." Cognitive processing 13.1 (2012): 261-265. [7] Haans, Antal, and Wijnand A. IJsselsteijn. "The virtual midas touch: helping behavior after a mediated social touch." IEEE Transactions on Haptics 2.3 (2009): 136-140. [8] Istance, Howell, et al. "Snap clutch, a moded approach to solving the Midas touch problem." Proceedings of the 2008 symposium on Eye tracking research & applications. 2008. [9] Chen, Mingtse, Anil K. Annadata, and Kuang Huang. "User interface for multi-channel communication." U.S. Patent No. 8,601,492. 3 Dec. 2013. [10] Agatz, Niels AH, Moritz Fleischmann, and Jo AEE Van Nunen. "E-fulfillment and multi-channel distribution–A review." European journal of operational research 187.2 (2008): 339-356. [11] Zhu, Zhiwei, and Qiang Ji. "Eye and gaze tracking for interactive graphic display." Machine Vision and Applications 15.3 (2004): 139-148. [12] Du Bois-Reymond, Emil. Untersuchungen uber thierische Elektricitat. 1849. [13] Das, S., Swar, S.K., Laha, S., Mahindar, S., Halder, S., Koushik, Hati, & Deb, S. (2016). Design approach of Eye Tracking and Mind Operated Motorized System. [14] Mowrer, O. H., Theodore C. Ruch, and N. E. Miller. "The corneo-retinal potential difference as the basis of the galvanometric method of recording eye movements." American Journal of Physiology-Legacy Content 114.2 (1935): 423-428. [15] Kohlrausch, Arnt. "Elektrische Erscheinungen am Auge." Receptionsorgane II. Springer, Berlin, Heidelberg, 1931. 1393-1496. [16] Kris, Christine. "Corneo-fundal potential variations during light and dark adaptation." Nature 182.4641 (1958): 1027-1028.

11

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

[17] Carmichael, Leonard, and Walter F. Dearborn. "Reading and visual fatigue." (1947). [18] Woodworth, Robert Sessions, Bernard Barber, and Harold Schlosberg. Experimental psychology.
Oxford and IBH Publishing, 1954. [19] Ford, A., C. T. White, and M. Lichtenstein. "Analysis of eye movements during free
search." JOSA 49.3 (1959): 287-292. [20] Shackel, B. "Eye movement recording by electro-oculography, in "Manual of Psychophysical
Methods" (PH Vewnables, I. Martin)." (1967): 229-334. [21] Dodge, Raymond, and Thomas Sparks Cline. "The angle velocity of eye
movements." Psychological Review 8.2 (1901): 145. [22] Judd, Charles H., Cloyd N. McAllister, and W. M. Steele. "General introduction to a series of
studies of eye movements by means of kinetoscopic photographs." Psychological Review Monographs 7.1 (1905): 1-16. [23] Karslake, James Spier. "The Purdue eye-camera: a practical apparatus for studying the attention value of advertisements." Journal of Applied Psychology 24.4 (1940): 417. [24] Barlow, Horace B. "Eye movements during fixation." The Journal of Physiology 116.3 (1952): 290-306.. [25] Higgins, George C., and Keith F. Stultz. "Frequency and amplitude of ocular tremor." JOSA 43.12 (1953): 1136-1140. [26] Mackworth, Jane F., and N. H. Mackworth. "Eye fixations recorded on changing visual scenes by the television eye-marker." JOSA 48.7 (1958): 439-445. [27] Shackel, B. "Note on mobile eye viewpoint recording." JOSA 50.8 (1960): 763-768. [28] Shakhnovich, A. R., and VR SHAKHNOVICH. "Scanning pupillograph." Voprosy neirokhirurgii 25 (1961): 57-59. [29] Mackworth, Norman H., and Edward Llewellyn Thomas. "Head-mounted eye-marker camera." JOSA 52.6 (1962): 713-716. [30] Thomas, E. Llewellyn. "Movements of the eye." Scientific American 219.2 (1968): 88-95. [31] Dodge, Raymond. "An experimental study of visual fixation." The Psychological Review: Monograph Supplements 8.4 (1907) [32] Stratton, George Malcolm. "Symmetry, linear illusions, and the movements of the eye." Psychological Review 13.2 (1906): 82. [33] Hartridge, Hamilton, and L. C. Thomson. "Methods of investigating eye movements." The British journal of ophthalmology 32.9 (1948): 581. [34] Lord, Mary P., and W. D. Wright. "Eye movements during monocular fixation." Nature 162.4105 (1948): 25-26. [35] Mackworth, Jane F., and N. H. Mackworth. "Eye fixations recorded on changing visual scenes by the television eye-marker." JOSA 48.7 (1958): 439-445. [36] Johansson, Gunnar, and Fredrik Backlund. "A versatile eye‐movement recorder." Scandinavian journal of psychology 1.1 (1960): 181-186. [37] Morimoto, Carlos H., et al. "Frame-rate pupil detector and gaze tracker." Proceedings of the IEEE ICCV. Vol. 99. 1999. [38] Jacob, Robert JK, and Keith S. Karn. "Eye tracking in human-computer interaction and usability research: Ready to deliver the promises." The mind's eye. North-Holland, 2003. 573-605. [39] Cornsweet, Tom N., and Hewitt D. Crane. "Accurate two-dimensional eye tracker using first and fourth Purkinje images." JOSA 63.8 (1973): 921-928. [40] Crane, Hewitt D., and Carroll M. Steele. "Generation-V dual-Purkinje-image eyetracker." Applied optics 24.4 (1985): 527-537. [41] Sarkar, N., et al. "A large angle, low voltage, small footprint micromirror for eye tracking and near-eye display applications." 2015 Transducers-2015 18th International Conference on Solid-State Sensors, Actuators and Microsystems (TRANSDUCERS). IEEE, 2015. [42] Turner, Kimberly L., et al. "Five parametric resonances in a microelectromechanical system." Nature 396.6707 (1998): 149-152.

12

CDMMS 2020 Journal of Physics: Conference Series

IOP Publishing 1802 (2021) 042066 doi:10.1088/1742-6596/1802/4/042066

[43] Cleveland, Dixon. "Systems and methods for miniaturizing eyetracking systems." U.S. Patent No. 9,386,921. 12 Jul. 2016.
[44] Poole, Alex, and Linden J. Ball. "Eye tracking in HCI and usability research." Encyclopedia of human computer interaction. IGI Global, 2006. 211-219.
[45] Chen, Z., Fu, H., Lo, W., & Chi, Z. (2018). Strabismus Recognition Using Eye-Tracking Data and Convolutional Neural Networks. Journal of Healthcare Engineering, 2018, 1-9.
[46] Weninger, Felix, et al. "Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR." International Conference on Latent Variable Analysis and Signal Separation. Springer, Cham, 2015.
[47] Zhang, Kaihao, et al. "Facial expression recognition based on deep evolutional spatial-temporal networks." IEEE Transactions on Image Processing 26.9 (2017): 4193-4203..
[48] Liu, Guojin, et al. "Passenger flow estimation based on convolutional neural network in public transportation system." Knowledge-Based Systems 123 (2017): 102-115.
[49] Krafka, Kyle, et al. "Eye tracking for everyone." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[50] Hansen, Dan Witzner. "Eye gaze tracking." U.S. Patent No. 9,398,848. 26 Jul. 2016. [51] Al-Rahayfeh, Amer, and Miad Faezipour. "Eye tracking and head movement detection: A state-
of-art survey." IEEE journal of translational engineering in health and medicine 1 (2013): 2100212-2100212.. [52] Boraston, Zillah, and Sarah‐Jayne Blakemore. "The application of eye‐tracking technology in the study of autism." The Journal of physiology 581.3 (2007): 893-898. [53] Samadani, Uzma, et al. "Eye tracking detects disconjugate eye movements associated with structural traumatic brain injury and concussion." Journal of neurotrauma 32.8 (2015): 548556. [54] Larsen, Eric, et al. "Methods and systems for user interaction within virtual reality scene using head mounted display." U.S. Patent No. 10,073,516. 11 Sep. 2018. [55] Ohshima, Toshikazu, Hiroyuki Yamamoto, and Hideyuki Tamura. "Gaze-directed adaptive rendering for interacting with virtual space." Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium. IEEE, 1996. [56] Smith, J. David, and TC Nicholas Graham. "Use of eye movements for video game control." Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology. 2006. [57] Hutt, Stephen, et al. "The Eyes Have It: Gaze-Based Detection of Mind Wandering during Learning with an Intelligent Tutoring System." International Educational Data Mining Society (2016). [58] Jacob, Robert JK, and Keith S. Karn. "Eye tracking in human-computer interaction and usability research: Ready to deliver the promises." The mind's eye. North-Holland, 2003. 573-605.

13