638 lines
43 KiB
Plaintext
638 lines
43 KiB
Plaintext
Volume 64, Issue 1, 2020
|
||
Journal of Scientific Research
|
||
Institute of Science, Banaras Hindu University, Varanasi, India.
|
||
|
||
Potential Eye Tracking Metrics and Indicators to Measure Cognitive Load in
|
||
Human-Computer Interaction Research
|
||
|
||
Antony William Joseph1*, Ramaswamy Murugesh1
|
||
|
||
1Department of Computer Application, Madurai Kamaraj University, Madurai. antony.william@gmail.com, mrswami123@gmail.com
|
||
|
||
Abstract: Eye tracking has a deep-rooted history in the field of medical and psychological research study as a method for investigating human visual behavior. It is a reliable and nonintrusive technology which opened up innovative approaches of solving problems or getting insightful data, but also emerging as a new filed of research. Due to recent technological advances, eye tracking technology has revolutionized human technology interaction. Even though, many practical issues exist concerning the use of eye tracking as an input device in human-computer interaction (HCI), this method is progressively used in user research, usability evaluation, user interface design, user experience design and in many other everyday applications and products. The objective of this research paper is to present potential indicators and their significance to measure cognitive load within the frame of human-computer interaction. We discussedvarious eye movement metrics and indicators which can be employed to measure cognitive load in the context of HCI research.
|
||
Index Terms: eye tracking, eye movement, blink, pupil size, saccades, fixations, gaze, user research, cognitive load, visual attention, eye tracking metrics, eye tracking indicators
|
||
I. INTRODUCTION
|
||
People constantly explore their natural world by moving their eyes. They look around rapidly with less conscious effort but with regard to the highly structured task, the research study has provedthat individuals look at what they are working on. The eyes do not wander randomly. Eye contact and gaze direction are essential indicators in human communication, for example, in regulating interaction and turn taking, establishing socioemotional connection, or indicating the target of our visual interest (Kleinke, 1986). The eye is the main gateway to the
|
||
|
||
brain (Brigham et al. 2001; Ellis et al., 1998). Gaze activities reveals cognitive process and provides insights about thought process and intentions(Land &Furneaux, 1997).
|
||
Eye tracking method is useful in manydomain, from psychological study and medical diagnostic to usability studies and interactive, gaze-controlled applications. The focus is to provide a meaningful starting point for researchers, designers of interactive technology and assistive technology professionals wishing to gain deeper insight into eye tracking metrics and their significance.
|
||
Primarily, eye activities were primarily studied by physiological evaluation and observation. Basic eye movements were classified and duration estimated for precise measurement of eye movements. The first generation of eye tracking devices was uncomfortable. The revolution and advancement of the eye tracking technology paved way for thedevelopment of the first ‘‘non-invasive’’ eye tracking device in the early 1900s (Wade &Tatler 2005) that was established on the principle of photography and light reflected from the cornea. It can be considered as the first ancestor of the current widely used videobased, corneal reflection eye tracking systems. The innovation of discreet camera based methods (Morimoto & Mimica 2005) and explosion of computing ability enriched eye tracking information, allowing the gaze as a main control mechanism for disabled people (Ten Kate et al., 1979). This has become a motivational factor to extend eye tracking techniques over a large area, which are discussed later.
|
||
Tracking people’s eye movements can benefit HCI researchers understand the factors that increase the cognitive load at a given point of time. Eye movements can also be recorded and used as control signals to empower users to interact with computer interfaces directly without using mouse or keyboard input or touch. In the coming chapters we given an overview of eye tracking technology, eye tracking in HCI
|
||
|
||
DOI: http://dx.doi.org/10.37398/JSR.2020.640137
|
||
|
||
268
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
research, cognitive load, potential eye tracking indicators to measure cognitive load and methodologies used to implement gaze interaction.
|
||
II. HISTORY OF EYE TRACKING IN HUMAN-COMPUTER INTERACTION
|
||
Research study shows that various methods have been commonly used to track eye movements 100 years ago (Javal, 1878/1879). The early non-invasive eye tracking method, using light reflected from the cornea was established by Dodge and Cline (1991). It recorded only horizontal eye position on a photographic plate when the participant’s head is motionless. Later in 1905 Judd, McAllister and Steel used motion picture photography to capture the eye movements in two dimensions. In the first half of the twentieth century, researchers made progressive advances in recording eye movements by linking the corneal reflection and motion picture techniques (Mackworth&Mackworth, 1958). In 1930, Miles Tinker and his team started to apply photographic method to study the eye movements in reading (Tinker, 1963) and studied the reading pattern and eye movement speed.. In 1947 motion picture cameras were used to investigate the movements of pilots’ eyesin the cockpit controls with regard to landing airplane (Fitts, Jones & Milton, 1950). This study represents earliest research study of usability engineering to study the users interacting with products and improve the product design.
|
||
The first head-mounted eye tracker was discovered by Hartridge and Thompson in 1948. Though it was a crude product in its form, it served to free the participants from tight constrains on head movements. Further it was developed by Shackel (1960) and Mackworth and Thomas (1962) and made it less obtrusive. In 1958, Mackworth and Mackworth developed a system to capture eye movements on the changing visual seen viewed by the participant and this led to the relevance of eye tracking to Human-computer interaction. Further the eye tracking technology advanced in 1970s and this paved the way for psychological theory to link eye tracking data to cognitiveprocess period (Monty & Senders, 1976; Senders, Fisher & Monty, 1978; Fisher, Monty & Senders, 1981).
|
||
In 1970s, significant efforts were made to technical enhancements to bring more accuracy and precision. The arrival of minicomputer provided necessary base for high-speed data processing and this led to the use of eye tracking data in realtime as a means of Human-computer interaction (Anliker, 1976). In 1970 psychologists who researched eye movements and fixations tried to evade cognitive issues such as learning, memory, workload and attention (Kowler, 1990). This scenario slowly changed when engineers developed better eye tracking system, psychologists started to investigate the relationshipbetween fixations and cognitive activity (Just & Carpenter, 1976a, 1976b).
|
||
Each decade, eye tracking has solved consistently new
|
||
|
||
problems (Senders, 2000). As the personal computers were proliferated, the eye tracking technology pioneered its study to address the issues in the domain of human-computer interaction and began to examine how users navigate (Card, 1984; Hendrickson, 1989; Altonen, Hyrskykari and Raiha, 1998; Byrne et al., 1999).During the 1980s user-computer communication was also emerged with the combination of eye tracking data (Bolt, 1981, 1982; Levine, 1984; Glenn et al., 1986; Ware andMikaelian, 1987).
|
||
Invasive methods were used initially, involving direct mechanical contact with the eye to detect the eye movements. An another technique also used, like bigger contact lenses to cover the cornea and sclera with a metal coil implanted on the edge for measuring the variations in an electromagnetic field while moving the metal coil along with eyes (Duchowski, 2003). Most modern eye tracking in human-computer interaction has shown a significant growth in exploring the usability of computer user interfaces. As the technology and technology enabled services were advancing and proliferating, eye tracking became a viable means to study the factors affecting human interaction and measure the cognitive load while performing tasks.
|
||
|
||
III. EYE TRACKING TECHNOLOGY
|
||
|
||
Eye tracking technology comes with an eye tracker to measure
|
||
|
||
the eye activity. It tracks our eye movement and help the
|
||
|
||
researcher to understand the attention area of the visual field.
|
||
|
||
The idea is simple but the method and analysis can be quite
|
||
|
||
complex. In general, eye tracking information is gathered by
|
||
|
||
means of either screen based or wearable eye tracker connected
|
||
|
||
to a computer. Many of the eye tracking systems available today
|
||
|
||
have two common elements: a light source and a camera.The
|
||
|
||
infrared, light source is targeted toward the eye. Reflection of
|
||
|
||
the light source together with noticeable ocular features for
|
||
|
||
instance the pupil are tracked by the eye tracking camera. Most
|
||
|
||
of the eye trackers existing today measure point-of-regard by the
|
||
|
||
“corneal-reflection/pupil-center”
|
||
|
||
method
|
||
|
||
(Goldberg
|
||
|
||
&Wichansky, 2003). The eye tracking information is used to
|
||
|
||
analyzethe movement of the eye and finally the direction of the
|
||
|
||
gaze. Information such as blink frequency, changes in the pupil
|
||
|
||
diameter and other information related to the eye movements are
|
||
|
||
also identified by the eye tracker.
|
||
|
||
Screen based eye trackers is a device which captures gaze data
|
||
|
||
for research into behavior and eye movements, from fixation-
|
||
|
||
based studies into micro-saccades as shown in Fig. 1. Wearable
|
||
|
||
eye tracker is a device which captures natural viewing behavior
|
||
|
||
in any real-world environment as shown in Fig. 2. This helps to
|
||
|
||
understand how a person is looking at in real time as they move
|
||
|
||
freely in any real-world setting and how people interact with the
|
||
|
||
environment. Wearable eye tracking system opens up a wide
|
||
|
||
range of opportunities in studying user behaviors.
|
||
|
||
Each eye data is interpreted into pixel coordinates position.
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
269
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
This kind of analysis will determine which feature is seen or what captures the attention, how eye moves and what content is overlooked. Graphs such as heat map and gaze plots are often produced to visualize the findings. Other than the visual attention, the eye data can be used to measure cognitive workload of a participant. The most commonly used metrics is Index of Cognitive Activity (ICA). Today with the advancement of eye tracking technology, anything with a visual element can be eye tracked and today it has become significantly important in research and design. The fields such as advertising, entertainment, packaging and interface design are widely benefitted from eye tracking from studying the user behavior. Modern day eye trackers provide possibilities to integrate additional biometric devices such as EEG, GSR or EKG to make the conclusion more concrete. The usage of eye tracking technology has extended to the sectors, like automotive, medical and defense to make the user safer. The growing newer areas like, advertising, entertainment, packaging and web design have
|
||
Fig. 1.Screen based eye tracking components of Tobii eye tracking Image Source: https://www.tobiipro.com/learn-and-support/learn/eye-
|
||
tracking-essentials/how-do-tobii-eye-trackers-work/
|
||
gained substantial insights from examining the visual behaviour
|
||
Fig. 2.Time versus location (communication fault) Image Source: https://www.tobiipro.com/learn-and-support/learn/eye-
|
||
tracking-essentials/how-do-tobii-eye-trackers-work/
|
||
of the end user.
|
||
|
||
IV. EYE TRACKING IN HUMAN-COMPUTER INTERACTION
|
||
Eye tracking is a method of tracking eye movementsdenoting to the point of user’s visual gaze focused on the scene. Eye tracking concept has a long history even before computer interfaces (Fitts, Jones & Milton, 1950) and their findings are still useful today. As the technology advanced to the new highest of user interface and user experience, human-computer interaction is gaining more importance each day. Proliferation of computing devices such as smartphones, tablets and laptops with internet of things (IoT) justifies the need for better user experiences. Eye tracking can assist significantly in the evaluation of usability aspect of a product, app, website, or any interface in general. Eye tracking techniques gives invaluable data to understand the users mental model and the problems users face while doing a task.
|
||
Persons eye gaze is presumed to indicate the thought in relation to cognitive process (Just and Carpenter, 1976). This means that the eye movement can give dynamic evidence to person’s attention in relation to a visual display.Calculating other features of eye movements like fixations can also uncover the amount of processing being applied to objects at the pointof-regard. HCI researcher defines the area of interest over some parts of display to infer useful information from the eye movement. This will enable the researcher can objectively evaluate the interface with eye tracking metrics and improve the human interaction (Goldberg &Kotval, 1999).
|
||
Eye movements provide many aspects of human cognition and this has become the scientific research tool in HCI and related disciplines like human factors and cognitive ergonomics. HCI and usability research studies have significantly researched the nature and effectiveness of information search methods on menu based user interfaces (Altonen, Hyrskykari, &Räihä, 1998; Byrne, Anderson, Douglas, &Matessa, 1999; Hendrickson, 1989) and also studied the characteristics that are associated with effective usability (Cowen, Ball, &Delin, 2002; Goldberg, Stimson, Lewenstein, Scott, &Wichansky, 2002; Poole, Ball, & Philips, 2004). Additionally, eye tracking technologies were broadly used in applied human factors research in air-trafficcontrol training (Hauland, 2003) to design user-friendly cockpit controls to reduce the error and in medical procedures to improve doctor’s performance (Law, Atkins, Kirkpatrick, & Lomax, 2004; Mello-Thoms, Nodine, &Kundel, 2002). Eye tracking technology is also used in the areas of market research and advertisements to attract greatest attention (Lohse, 1997).
|
||
V. POTENTIAL EYE TRACKING METRICS AND INDICATORS TO MEASURE COGNITIVE LOAD
|
||
This chapter summarizes the common eye tracking indicators to measure cognitive load in the HCI research. We discuss eye tracking metrics in order to investigate users’ cognitive process and cognitive load. Eye tracking measures like task performance, visualization, blink, pupil dilation, saccade,
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
270
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
fixation and scan path are discussed with other variables and their indicators. This can help many HCI researchers who use eye trackers to measure cognitive load. Each of the metrics can apply various measuring algorithms depending on the nature and context of the research.
|
||
|
||
A. Cognitive Task Analysis Metrics and Indicators
|
||
|
||
Cognitive task analysis (CTA) performed to measure the
|
||
|
||
expected performance standard on a goal oriented task and the
|
||
|
||
aim is to record the cognitive knowledge used by them (Clark &
|
||
|
||
Estes, 1999). It helps the researcher to capture accurate and
|
||
|
||
complete description of cognitive processes and decisions. There
|
||
|
||
are many number of CTA methods currently in use due to the
|
||
|
||
diverse path that the development of CTA has taken but one can
|
||
|
||
choose any method fitting into the context. One can design
|
||
|
||
Table I. Cognitive task analysis metrics and indicators
|
||
|
||
Metrics
|
||
|
||
Indicators
|
||
|
||
Time on a task
|
||
|
||
Time taken to complete the task indicates
|
||
|
||
(simple to complex) more cognitive load.
|
||
|
||
Steps to complete Number of steps taken to complete the
|
||
|
||
the task
|
||
|
||
task indicates more cognitive load.
|
||
|
||
simple to complex tasks or only complex tasks or only simple tasks to measure the cognitive load of a novice to expert users. The time taken to complete the task and number of steps taken to complete the task will indicate the cognitive load. Table 1 gives details of cognitive task analysis metrics.
|
||
|
||
B. Eye Tracking Visualization Metrics and Indicators
|
||
In commercial eye tracking systems, there are two kinds of visualization that is heat map and gaze plot. The heat map is graphical representation of data clustered data and their values represented in colour. Heat map is calculated by number of fixations and duration of fixation. A gaze plot is a map which shows movement sequence of fixations on a visual display in the
|
||
|
||
Table II. Visualization Metrics and Indicators
|
||
|
||
Metrics
|
||
|
||
Indicators
|
||
|
||
Heat map
|
||
|
||
It shows the number of times attention to a particular part of display.
|
||
|
||
Gaze plot
|
||
|
||
More steps to perform a task indicates more cognitive load.
|
||
|
||
order they occur. It is simply the user’s eye path represented by circles as fixations and circle areas are proportional to fixation length. These two metrics can help the researcher to understand how user’s mind works while performing a task. Table II gives details of cognitive task analysis metrics.
|
||
|
||
C. Fixation Metrics and Indicators
|
||
Fixations are the most important element that eye tracking researchers analyze to make inferences about cognitive process. It is the stage where eye essentially stops scanning the scene and sticks to the central foveal vision in place so that the visual system can focus about what is being looked at. Gaze pints are the direct spatial spots of visual axis falling on the stimulus. They have (x,y) coordinate relating to its measurement. This is
|
||
|
||
an output generated by the eye tracking hardware. If theeye tracking device is operating at 60 Hz, a gaze point will be generated typically every 16.7 milliseconds. Similarly, at 300 Hz, gaze points are spaced a mere 3 milliseconds apart. The number fixation indicates number of times the user looked at a specific area. Game points indicate what the eyes are looking at. Gaze points per second are collected based on the sampling rate
|
||
|
||
Table III. Fixation metrics and indicators
|
||
|
||
Metrics
|
||
|
||
Indicators
|
||
|
||
A longer fixation duration describes issues
|
||
|
||
Duration of
|
||
|
||
related to extracting information, or it
|
||
|
||
fixation
|
||
|
||
indicates that the object is more appealing
|
||
|
||
(Just & Carpenter, 1976).
|
||
|
||
Fixation Count
|
||
|
||
The number of fixations on media indicates efficiency in information search.
|
||
|
||
Total number of Total number of fixations on media indicates
|
||
|
||
fixation
|
||
|
||
efficiency in information search
|
||
|
||
Fixation spatial density
|
||
|
||
Fixations concentrated in a small area indicate focused and efficient searching (Cowen et al., 2002).
|
||
|
||
Fixations per area of interest
|
||
|
||
More fixations on a specified area signify that it is more perceptible, or more significant to the viewer than other areas (Poole et al., 2004).
|
||
|
||
On-target Fixation
|
||
|
||
Fixations on-target divided by total number of fixations. A lower ratio indicates lower search efficiency.
|
||
|
||
Time to first fixation ontarget
|
||
|
||
The time-to-first fixation for each area of interest on all Media and indicates that an object or area has a better attention-getting properties (Byrne et al., 1999).
|
||
|
||
Average fixation count/duration
|
||
|
||
The average duration of the fixations within each area of interest on all Media. It shows how long the average fixation lasted for .
|
||
|
||
It is the number of gaze points inside a
|
||
|
||
Fixation density
|
||
|
||
fixation event divided by the smallest area to record all gaze events (Brookings, J. B.,
|
||
|
||
Wilson, G. F., & Swain, C. R., 1996).
|
||
|
||
Fixation rate
|
||
|
||
The frequency of looking at some point per second. It represent attention.
|
||
|
||
Time spent in the same position and area. It
|
||
|
||
Dwell time
|
||
|
||
indicates measuring the time the user remains
|
||
|
||
at a search result after a click.
|
||
|
||
Sum of all fixation durations within a
|
||
|
||
Gaze
|
||
|
||
specified area. It is best utilized to evaluate attention distributed among targets (Mello-
|
||
|
||
Thoms et al., 2004; Hauland, 2003).
|
||
|
||
A series of close gaze points or gaze cluster
|
||
|
||
Gaze samples in space and time indicates a period where the
|
||
|
||
eyes are locked towards an object.
|
||
|
||
Attention
|
||
|
||
Measures the dynamics of visual attention
|
||
|
||
Switching
|
||
|
||
using the total number of shifts between a set
|
||
|
||
Frequency
|
||
|
||
of area of interest per minute.
|
||
|
||
of the eye tracker. If gaze points are very close in time or space, it denotes that where the eyes are locked and it is an excellent measure for visual attention. Gaze direction is the indication for the interface element for the currently cognitive activity (Darrell S. Rudmann, George W. McConkie, &Xianjun Sam Zheng, 2003).
|
||
Fixations have two features from gaze points. The first feature
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
271
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
is that fixations have duration besides a spatial (x, y) location and start and end timestamps. The second is fixations are directly measurable using gaze point-to-fixation conversion algorithm or fixation filter. It can be explained easily depending on the context. In a web page task, higher fixation on a particular area can indicate more interest in the target or the target may be complex (Jacob &Karn, 2003; Just & Carpenter, 1976). But in the case of search task, higher number of single fixations indicate greater uncertainty (Jacob &Karn, 2003). Similarly, the fixation duration is linked to theprocessing time related to the target being fixed (Just & Carpenter, 1976). It is an effort made to restructure these significant eye movements as meaningful as possible in terms of attention, visibility, mental processing and understanding.Areas of interest (AOIs) is very important aspect of stimulus. This is more contextual and can be defined either before the experiment or after the experiment. The metrics applied to AOIs are related to transition that is number of transitions between two AOIs. The transition can be looked into in terms of dwell time and fixations within AOIs. Metrics closely related to fixations are listed with their indicators in the Table III.
|
||
|
||
D. Saccade Metrics and Indicators
|
||
Saccade is another eye tracking measure used to move the fovea quickly from one to another point of interest. This indicates to a shift between two fixations. Saccade can be triggered voluntarily or involuntarily.The length of a saccade and its amplitude are sequentially associated. When the eye voluntarily moves from one fixation to another and takes 30-80 milliseconds to complete. One can measure the velocity and length of saccades to investigate the cognitive load. The higher
|
||
|
||
Metrics Length of saccade
|
||
Number of saccades
|
||
Saccadic amplitude
|
||
Saccadic regression
|
||
Saccade duration
|
||
|
||
Table IV. Saccade metrics and indicators Indicators
|
||
Duration of changing the point of fixation and this indicates visual attention and cognitive relevance. Number of involuntary, abrupt, rapid, small movements of both eyes simultaneously in changing the point of fixation. (Goldberg &Kotval, (1999). It is the distance travelled by a saccade from onset to offset. Larger saccades indicate more meaningful cues (Goldberg et al., 2002). Events that move in the opposite direction to the text. Regressions indicate the presence of less meaningful cues. (Sibert et al., 2000). The time duration of a rapid movement of the eye between two fixation points. More saccades indicate more searching.
|
||
|
||
the load, the longer the saccades (Victor Manuel, Victor M. García-Barrios, Christian Gütl, Alexandra Preis, Keith Andrews, Maja Pivec, Felix Mödritscher& Christian Trummer, 2004). During saccades, no encoding happens and so cannot reveal the complexity of an interface. But, regressive saccades can provide clue for measuring processing difficulty (Rayner &Pollatsek,
|
||
|
||
1989). Table IV shows, metrics of saccade and their indicators in detail.
|
||
E. Scanpath Metrics and Indicators Scanpath is another indicator which embodies the spatial structure of eye movements performed by a participant during task. It unfolds visual attention indicating visual context attended. There is no clear method to average the scanpath but the difficulty of averaging the scanpath is due to spatial and temporal causes. The gaze plot that represents saccades and fixations with straight lines and circles respectively is used as a common method for scanpath visualization. Though there are many algorithms to discover relationship in different scanpaths, Scan path Trend Analysis (STA) uncovers the most appropriate path of multiple users on visual screen. Table V shows, metrics and indicators of scanpath to measure the cognitive load in eye tracking study.
|
||
F. Pupil Dilation Metrics and Indicators Pupil dilation is a significant indicator of cognitive load in eye
|
||
|
||
Table V. Scanpath metrics and indicators
|
||
|
||
Metrics
|
||
|
||
Indicators
|
||
|
||
Duration of a complete sequence of
|
||
|
||
Scan path duration
|
||
|
||
fixations and interconnecting saccades. A long lasting scan path signifies lesser
|
||
|
||
effectiveness (Goldberg &Kotval, 1999).
|
||
|
||
A lengthier scanpath denotes less
|
||
|
||
Scan path Length efficient Searching (Goldberg et al.,
|
||
|
||
2002).
|
||
|
||
Spatial Density
|
||
|
||
Lesser spatial density signifies more direct Search (Goldberg &Kotval, 1999).
|
||
|
||
The path in which a whole order of
|
||
|
||
Scanpath Direction
|
||
|
||
fixations and interlinked saccades occur. This can indicate participant’s search
|
||
|
||
strategy (Altonen et al., 1998).
|
||
|
||
Deviance from a normal scanpath denote
|
||
|
||
Scanpath Regularity search issues related to interface
|
||
|
||
(Goldberg &Kotval, 1999).
|
||
|
||
This is a comparison between time spent
|
||
|
||
Saccade/Fixation Ratio
|
||
|
||
on searching (saccade) and time spent on processing (fixations). Here, higher ratio implies more processing or less
|
||
|
||
searching.
|
||
|
||
tracking research study. It is an involuntary reflex and the pupil can vary in diameter from 1/16 inch to more than 1/3 inch. Psychologists have acknowledged that changes in pupil dilation indicates effortful cognitive processing. Past research also confirms that the pupils dilate when the difficulty of the task and their cognitive effort to solve it increases. Video cameras and infrared illuminators are used in the eye tracking technology to record the eye position, cornea reflections and the pupil dilation. Using eye tracker, the researcher can examine the response of pupil dilation in relation to the information on the screen and the behavioral choices during the experiment.
|
||
Human pupils dilate for numerous reasons, including memory, cognitive struggle, excitement, pain and so on (Beatty, 1982).
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
272
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
When a user performs cognitively demanding tasks, pupils dilate 1-2 seconds in response to mental workload of the task (Beatty, 1982) and constrict gradually after the task is finished (Kahneman& Beatty, 1966; Hess, 1972). To measure changes in cognitive load under a range of tasks, the average percentage change in pupil size can be analyzed. One has to be very careful
|
||
|
||
Table VI. Pupil dilation metrics and indicators
|
||
|
||
Metrics
|
||
|
||
Indicators
|
||
|
||
The size of the pupil in response to the media. Pupil
|
||
|
||
size could be used as a manifestation of cognitive
|
||
|
||
workload (Marshall, 2000; Pomplun&Sunkara,
|
||
|
||
2003).
|
||
|
||
Pupil size
|
||
|
||
It is an involuntary reflex and the pupil can range in
|
||
|
||
diameter from 1/16 inch (1.5mm) to 1/3 (8mm).
|
||
|
||
Changes in pupil dilation accompany effortful
|
||
|
||
cognitive process.
|
||
|
||
to distinguish the exact cause that activated the pupillary response because there are various causes which can triggers pupillary response, like light reflex, changes in illuminous and error due to gaze angles. Table VI shows metrics and indicators related topupil dilation.
|
||
|
||
G. Blinking Metrics and Indicators Blinking activity is another metric to another layer of information for measuring cognitive load. Blinking is mostly an involuntary act during which the eyelid covers the pupil and cornea from the light reflect resulting in raw data points missing
|
||
|
||
Table VII. Blinking metrics and indicators
|
||
|
||
Metrics
|
||
|
||
Indicators
|
||
|
||
The blink frequency per minute. The blink rate and
|
||
|
||
pupil size could be considered as a manifestation of
|
||
|
||
Blink rate cognitive workload (Bruneau, Sasse, & McCarthy,
|
||
|
||
2002; Brookings; Wilson, & Swain, 1996;
|
||
|
||
Marshall, 2000; Pomplun&Sunkara, 2003).
|
||
|
||
Blink
|
||
|
||
Closure time duration of a blink in ms. Provides
|
||
|
||
duration information about cognitive load and attention.
|
||
|
||
Velocity of blinking
|
||
|
||
Velocity attained while eye lid closing and opening. Gives information about cognitive load and attention.
|
||
|
||
Latency in the saccadic system is around 200 ms,
|
||
|
||
Blink latency
|
||
|
||
and corresponds to the time from the onset of the stimulus to the initiation of the eye movement. Provides information about cognitive load and
|
||
|
||
attention.
|
||
|
||
the x, y coordinates information. For analysis, fixation filter can be used to remove the missing points and infer the information correctly into fixations.
|
||
The blink rate and latency will provide information for deeper understanding about the state of the attention of the participant. High blink rate and low blink rate has been found to indicate high mental effort (Siyuan Chen, Julien Epps, Natalie Ruiz & Fang Chen, 2011). Similarly, research shows that increase and decrease in the blinking velocity and a decreasing degree of the eyelids’ openness are sings of increasing tiredness.
|
||
To detect eye blink, the researcher needs to look at the center
|
||
|
||
point of the pupil. When the eye is open, there will be white pixels on the image which will help to calculate easily the center point of the pupil. If eye is closed, there will not white pixels of pupil on the image. A count variable is created to calculate the center point. The variable is taken as ‘1’ when the eye is open and the variable is taken as ‘0’ when the eye is closed. These variables will help to detect the eye blink metrics and their indicators as given in the Table VII.
|
||
CONCLUSION Eye tracking technology is an important device to measure and analyses cognitive load in the context of HCI and visual computing. It appears as a new and promising area of work. We reviewed the usage of eye tracking system in HCI research and discussed the relevance of eye tracking metrics and indicators for measuring cognitive load. Our approach was to look at all possible eye tracking metrics and indicators holistically so that they can be possibly used in any type of HCI research in terms of measuring cognitive load. The focus is to provide a meaningful reference for HCI researchers so that they can identify metrics and make their own recipe for the study of attentional, cognitive states or process.As the eye tracking technology becomes increasingly more intelligent, less invasive and easier to use, the use of eye movement tracking technique going to increase manifold.
|
||
ACKNOWLEDGEMENT
|
||
The authors are grateful to Madurai Kamaraj University, National Institute of Design for their support. We are also thankful to Tobii Eye Tracking for their support and guidance.
|
||
REFERENCES
|
||
Wilfley, D. (1989). Interpersonal analyses of bulimia: Normal-weight and obese. Unpublished doctoral dissertation, University of Missouri, Columbia. Altonen, A. Hyrskykari, A. &Räihä, K. (1998). 101 Spots, or how do users read menus? In Proceedings of CHI 98 Human Factors in Computing Systems, ACM Press, 132-139.
|
||
Anliker, J. (1976). Eye movements- On-line measurement, anal-ysis, and control. Eye movements and psychological processes, pp.185– 199.
|
||
Beatty, J. (1982). Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources. Psychological Bulletin, 91(2), 276-292.
|
||
Brookings, J. B., Wilson, G. F., & Swain, C. R. (1996). Psychophysiological responses to changes in workload during simulated air traffic control. Biological Psychology, 42, 361-377.
|
||
Byrne, M. D., Anderson, J. R., Douglas, S., &Matessa, M. (1999). Eye tracking the visual search of clickdown menus. Proceedings of CHI’99 (pp. 402-409). NY: ACM Press.
|
||
Bolt, R. A. (1981). Gaze-orchestrated dynamic windows. Computer Graphics, 15, 109–119.
|
||
Bolt, R. A. (1982). Eyes at the interface. In: Proceedings of the ACM Human Factors in Computer Systems Conference (pp. 360–362).
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
273
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
Brigham FJ, Zaimi E, Matkins JJ et al. (2001). The eyes may have it: reconsidering eye-movement research in human cognition. In: Scruggs TE, Mastropieri MA (eds) Technologicalapplications. Advances in learning and behavioral disabilities, vol 15. Emerald Group Publishing Limited, Bingley, pp 39–59.
|
||
Bruneau, D., Sasse, M. A., & McCarthy, J. D. (2002). The eyes never lie: The use of eye tracking data in HCI research. In Proceedings of the CHI’02 Workshop on Physiological Computing. NY: ACM Press.
|
||
Byrne, M. D., Anderson, J. R., Douglas, S., &Matessa, M. (1999). Eye tracking the visual search of click-down menus. In: Proceedings of CHI 99 (pp. 402–409). NY: ACM Press.
|
||
Card, S. K. (1984). Visual search of computer command menus. In: H. Bouma and D. G. Bouwhuis (eds), Attention and Performance X,Control of Language Processes. Hillsdale, NJ: Lawrence Erlbaum Associates.
|
||
Clark, R. E. and Estes, F. (1999). The development of authentic educational technologies. Educational Technology, 39(2), 5-16.
|
||
Cowen, L., Ball, L. J., &Delin, J. (2002). An eye-movement analysis of web-page usability. In X. Faulkner, J. Finlay, & F. Détienne (Eds.), People and Computers XVI—Memorable yet Invisible: Proceedings of HCI 2002 (pp. 317-335). London: Springer-Verlag Ltd.
|
||
Darrell S. Rudmann, George W. McConkie and Xianjun Sam Zheng. (2003). Eye tracking in cognitive state dedection for HCI. In: roceedings of the 5th international conference on Multimodal interfaces (pp. 159-163). Vancouver, British Columbia, Canada.
|
||
Duchowski, A. T. (2003). Eye tracking methodology: Theory and practice. London: Springer-Verlag Ltd.
|
||
Ellis S, Cadera R, Misner J et al. (1998). Windows to the soul? What eye movements tell us about software usability. In: Proceedings of 7th annual conference of the usability professionals association, Washington, pp 151–178.
|
||
Eye-voice-controlled interface. In: Proceedings of the 30th Annual Meeting of the Human Factors Society (pp. 322–326). Santa Monica: Human Factors Society.
|
||
EYETRACKING, INC, 'What is eye tracking?’, 2011. [Online]. Available: http://www.eyetracking.com/About-Us/What-Is-EyeTracking. [Accessed: 01- Feb- 2019].
|
||
Fitts, P. M., Jones, R. E., & Milton, J. L. (1950). Eye movements of aircraft pilots during instrument-landing approaches. Aeronautical Engineering Review 9(2), 24–29.
|
||
Fisher, D. F., Monty, R. A., & Senders, J. W. (eds) (1981). Eye Movements: Cognition and Visual Perception. Hillsdale, NJ: Lawrence Erlbaum.
|
||
Glenn, F., Iavecchia, H., Ross, L., Stokes, J., Weiland, W., Weiss, D., &Zaklad, A. (1986).
|
||
Goldberg, J. H., Stimson, M. J., Lewenstein, M., Scott, N., &Wichansky, A. M. (2002). Eye tracking in web search tasks: Design implications. In Proceedings of the Eye Tracking Research and Applications Symposium 2002 (pp. 51-58). NY: ACM Press.
|
||
Goldberg, J.H. &Kotval, X.P. (1998). Eye movement-based evaluation of the computer interface. In S.K. Kumar (Ed.), Advances in Occupational Ergonomics and Safety. (pp. 529-532). Amsterdam: ISO Press.
|
||
Goldberg, H. J., &Kotval, X. P. (1999). Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics, 24, 631-645.
|
||
|
||
Goldberg, H. J., &Wichansky, A. M. (2003). Eye tracking in usability
|
||
|
||
evaluation: A practitioner’s guide. In J. Hyönä, R. Radach, & H.
|
||
|
||
Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye
|
||
|
||
movement research (pp. 493-516). Amsterdam: Elsevier.
|
||
|
||
HarsimratDeo. (2015). International Journal of Electrical, Electronics
|
||
|
||
and
|
||
|
||
Computer
|
||
|
||
Systems
|
||
|
||
3
|
||
|
||
(10),
|
||
|
||
pp. 49-55.
|
||
|
||
Hartridge, H., & Thompson, L. C. (1948). Methods of investigating eye
|
||
|
||
movements, British Journal of Ophthalmology, 32, 581–591.
|
||
|
||
Hauland, G. (2003). Measuring team situation awareness by means of
|
||
|
||
eye movement data. In Proceedings of HCI International 2003: Vol
|
||
|
||
3 (pp. 230-234). Mahwah, NJ: Lawrence Erlbaum Associates.
|
||
|
||
Hendrickson, J. J. (1989). Performance, preference, and visual scan
|
||
|
||
patterns on a menu-based system: Implications for interface design.
|
||
|
||
In: Proceedings of the ACM CHI’89 Human Factors in Computing
|
||
|
||
Systems Conference (pp. 217–222). ACM Press.
|
||
|
||
Hess, E. H. (1972). Pupillometrics. In N. S. Greenfield & R. A.
|
||
|
||
Sternbach (Eds.), Handbook of Psychophysiology (pp. 491-531):
|
||
|
||
Holt, Rinehart & Winston.
|
||
|
||
Hendrickson, J.J. (1989). Performance, preference, and visual scan
|
||
|
||
patterns on a menu-based system: implications for interface design
|
||
|
||
(pp. 217-222). In Proceedings of the ACM CHI’89 Human Factors
|
||
|
||
in Computing Systems Conference. ACM Press. In: E. Kowler (ed.),
|
||
|
||
Eye Movements and their Role in Visual and Cognitive Processes.
|
||
|
||
Amsterdam: Elsevier Science.
|
||
|
||
Javal, E. (1878). Essai sur la physiologie de la lecture.
|
||
|
||
Annalesd’Oculistique, 79, 97–117, 155–167, 240–274; 80 (1879),
|
||
|
||
61–73, 72–81, 157–162, 159–170, 242–253.
|
||
|
||
Jacob, R. J. K., &Karn, K. S. (2003). Eye tracking in Human-Computer
|
||
|
||
Interaction and usability research: Ready to deliver the promises, In
|
||
|
||
J. Hyönä, R. Radach, & H. Deubel (Eds.), The mind's eye:
|
||
|
||
Cognitive and applied aspects of eye movement research (pp. 573-
|
||
|
||
605). Amsterdam: Elsevier.
|
||
|
||
JohannesaZagermann, Ulrike Pfeil, and Harald Reiterer, Measuring
|
||
|
||
Cognitive Load using Eye Tracking Technology in Visual
|
||
|
||
Computing, BELIV ’16, October 24 2016, Baltimore, MD, USA.
|
||
|
||
ACM Press.
|
||
|
||
Joseph Tao-yi Wang. (2009). Pupil Dilation and Eye-Tracking,
|
||
|
||
Department of Economics, Natinoal Taiwan University, Taiwan.
|
||
|
||
Just, M. A., & Carpenter, P. A. (1976a). Eye Fixations and Cognitive
|
||
|
||
Processes. Cognitive Psychology, 8, 441–480.
|
||
|
||
Just, M. A., & Carpenter, P. A. (1976b). The role of eye-fixation
|
||
|
||
research in cognitive psychology. Behavior Research Methods &
|
||
|
||
Instrumentation, 8, 139–143.
|
||
|
||
Kahneman, D. &Beatty, J. (1966). Pupil Diameter and Load on
|
||
|
||
Memory. Science, 154(3756), 1583-1585.
|
||
|
||
Kleinke CL. (1986). Gaze and eye contact: a research review. Psychol
|
||
|
||
Bull 100(1):78–100.
|
||
|
||
Kowler, E. (1990). The role of visual and cognitive processes in the
|
||
|
||
control of eye movement.
|
||
|
||
Land MF, Furneaux S. (1997). The knowledge base of the oculomotor
|
||
|
||
system. Philos Trans BiolSci 352(1358):1231–1239. October 24
|
||
|
||
2016, Baltimore, MD, USA. ACM Press.
|
||
|
||
Kahneman, D. &Beatty, J. (1966). Pupil Diameter and Load on
|
||
|
||
Memory. Science, 154(3756), 1583-1585.
|
||
|
||
Kleinke CL. (1986). Gaze and eye contact: a research review. Psychol
|
||
|
||
Bull 100(1):78–100.
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
274
|
||
|
||
Journal of Scientific Research, Volume 64, Issue 1, 2020
|
||
|
||
Kowler, E. (1990). The role of visual and cognitive processes in the control of eye movement.
|
||
Land MF, Furneaux S. (1997). The knowledge base of the oculomotor system. Philos Trans BiolSci 352(1358):1231–1239.
|
||
Law, B., Atkins, M. S., Kirkpatrick, A. E., & Lomax, A. J. (2004). Eye gaze patterns differentiate novice and experts in a virtual laparoscopic surgery training environment. In Proceedings of the Eye Tracking Research and Applications Symposium 2004 (pp. 4148). NY: ACM Press.
|
||
Levine, J. L. (1984). Performance of an eyetracker for office use. Computational Biology and Medicine, 14, 77–89.
|
||
Lohse, G. L. (1997). Consumer eye movement patterns on Yellow Pages advertising. Journal of Advertising, 26, 61-73.
|
||
Mackworth, J. F., &Mackworth, N. H. (1958). Eye fixations recorded on changing visual scenes by the television eye-marker. Journal of the Optical Society of America, 48, 439–445.
|
||
Mackworth, N. H., & Thomas, E. L. (1962). Head-mounted eye-marker camera. Journal of the Optical Society of America, 52, 713–716.
|
||
Marshall, S. (2000). Method and apparatus for eye tracking and monitoring pupil dilation to evaluate cognitive activity. U.S. Patent 6,090,051, July 2000.
|
||
Mello-Thoms, C., Nodine, C. F., &Kundel, H. L. (2004). What attracts the eye to the location of missed and reported breast cancers? In Proceedings of the Eye Tracking Research and Applications Symposium 2002 (pp. 111-117). NY: ACM Press.
|
||
Monty, R. A., & Senders, J. W. (eds) (1976). Eye Movements and Psychological Processes. Hillsdale, NJ: Lawrence Erlbaum.
|
||
Morimoto CH, Mimica MRM. (2005). Eye gaze tracking techniques for interactive applications. Comput Vis Image Underst 98(1): 4–24.
|
||
Oliver Hohlfeld, Andr´e Pomp, Jo´ A´ gilaBitsch Link, Dennis Guse, On the Applicability of Computer Vision based Gaze Tracking in Mobile Scenarios, Mobile HCI’ 15, August 25-28, 2015, Copenhagen, Denmark. ACM Press.
|
||
Pomplun, M., &Sunkara, S. (2003). Pupil dilation as an indicator of cognitive workload in Human-Computer Interaction. In Proceedings of HCI International 2003: Vol 3 (pp. 542-546). Mahwah, NJ: Lawrence Erlbaum Associates.
|
||
Poole, A.; Ball, Linden J. (2006). Eye tracking in HCI and usability research.Encyclopaedia of human-computer interaction.. ed. / C. Ghaoui. Pennsylvania : Idea Group Inc., 2006. p. 211-219.
|
||
Poole, A., Ball, L. J., & Phillips, P. (2004). In search of salience: A response time and eye movement analysis of bookmark recognition. In S. Fincher, P. Markopolous, D. Moore, & R. Ruddle (Eds.), People and Computers XVIII-Design for Life: Proceedings of HCI 2004. London: Springer-Verlag Ltd.
|
||
Rayner, K., &Pollatsek, A. (1989). The psychology of reading. Englewood Cliffs, NJ: Prentice Hall.
|
||
Robert J., K. Jacob & Keith S. Karn. (2003). Cognitive and Applied Aspects of Eye Movement Research (pp. 573-605). Hyona, Radach&Deubel (eds.) Oxford, England.
|
||
Senders, J. W. (2000). Four theoretical and practical questions. Keynote address presented at the Eye Tracking Research and Applications Symposium 2000. Abstract in: Proceedings of the Eye Tracking Research and Applications Symposium 2000 (p. 8). New York: Association for Computing Machinery.
|
||
Senders, J. W., Fisher, D. F., & Monty, R. A. (eds) (1978). Eye Movements and the Higher Psychological Functions. Hillsdale, NJ: Lawrence Erlbaum.
|
||
|
||
Shackel, B. (1960). Note on mobile eye viewpoint recording. Journal of the Optical Society of America, 59, 763–768.
|
||
Shojaeizadeh, M., Djamasbi, S., & Trapp, A. C. (2016, July). Density of Gaze Points Within a Fixation and Information Processing Behavior. InInternational Conference on Universal Ac-cessin Human-Computer Interaction(pp. 465-471). Springer International Publishing.
|
||
Sidra Naveed, BushraSikander, and Malik Sikander Hayat Khiyal. (2012). Eye Tracking System with Blink Detection. Journal of Computing, 4 (3), pp 50-60.
|
||
Sibert, J. L., Gokturk, M., &Lavine, R. A. (2000). The Reading Assistant: Eye gaze triggered auditory prompting for reading remediation. In Proceedings of the Thirteenth Annual ACM Symposium on User Interface Software and Technology (pp. 101107). NY: ACM Press.
|
||
Siyuan Chen, Julien Epps, Natalie Ruiz and Fang Chen. (2011). Eye activity as a measure of human mental effort in HCI. In: Proceedings of the 2011 International Conference on Intelligent User Interfaces. Palo Alto, CA, USA.
|
||
Ten Kate JH, Frietman EEE, Willems W et al. (1979). Eye-switch controlled communication aids. In: Proceedings of the 12th international conference on medical and biological engineering, Jerusalem, Israel, August 1979.
|
||
Tinker, M. A. (1963). Legibility of Print. Ames, Iowa: Iowa State University Press.
|
||
Tobii AB, 'Eye tracking Product’, 2011. [Online]. Available: https://www.tobiipro.com. [Accessed: 01- Feb- 2019]. [Accessed: 01- Feb- 2019].
|
||
Victor Manuel , Victor M. García-Barrios , Christian Gütl , Alexandra Preis , Keith Andrews , Maja Pivec , Felix Mödritscher and Christian Trummer (2004) AdELE: A Framework for Adaptive ELearning through Eye Tracking. Proceedings of IKNOW 2004; place, pp 609-616.
|
||
Wade NJ, TatlerBW .(2005). The moving tablet of the eye: the origins of modern eye movement research. Oxford University Press, Oxford.
|
||
Ware, C., &Mikaelian, H. T. (1987). An evaluation of an eye tracker as a device for computer input. In: Proceedings of the ACM CHI+GI’87 Human Factors in Computing Systems Conference (pp. 183–188). New York: ACM Press. ***
|
||
|
||
Institute of Science, BHU Varanasi, India
|
||
|
||
275
|
||
|