Files
DissLiteratur/storage/DYLJUY3X/.zotero-ft-cache
Johannes Paehr c4354c0441 init
2025-10-18 15:35:31 +02:00

1014 lines
44 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
Stuart S, Alcock L, Godfrey A, Lord S, Rochester L, Galna B. Accuracy and re-test reliability of mobile eye tracking in Parkinson's disease
and older adults. Medical Engineering & Physics 2016, 38(3), 308-315
Copyright:
© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license
DOI link to article: http://dx.doi.org/10.1016/j.medengphy.2015.12.001 Date deposited:
07/09/2016
Embargo release date:
16 January 2017
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International licence
Newcastle University ePrints - eprint.ncl.ac.uk
Medical Engineering and Physics
http://dx.doi.org/10.1016/j.medengphy.2015.12.001
Accuracy and re-test reliability of mobile eye-tracking in Parkinsons disease and older adults
Stuart, S.1, Alcock, L. 1, Godfrey, A.1, Lord, S.1, Rochester, L.1 and Galna, B.1
1Institute of Neuroscience / Newcastle University Institute for Ageing, Clinical Ageing Research Unit, Newcastle University, United Kingdom
Correspondence to: Sam Stuart Research assistant and PhD Candidate Institute of Neuroscience Newcastle University Clinical Ageing Research Unit Campus for Ageing and Vitality Newcastle upon Tyne NE4 5PL Tel: +44191 208 1242 Email: sam.stuart@newcastle.ac.uk
Word Count: Abstract (200): 199 Article (including abstract): 3,495 Figures: 2 Tables: 3 Supplementary Material: 1 References: 45 out of 45
1 Abstract
2 Mobile eye-tracking is important for understanding the role of vision during real-world tasks 3 in older adults (OA) and people with Parkinsons disease (PD). However, accuracy and 4 reliability of such devices have not been established in these populations. We used a novel 5 protocol to quantify accuracy and re-test reliability of a mobile eye-tracker in OA and PD.
6 A mobile eye-tracker (Dikablis, 50Hz) measured the saccade amplitudes of 20 OA and 14 7 PD on two occasions. Participants made saccades between targets placed 5°, 10° and 15° 8 apart. Impact of visual correction (glasses) on saccadic amplitude measurement was also 9 investigated in 10 OA.
10 Saccade amplitude accuracy (median bias) was -1.21° but a wide range of bias (-7.73° to 11 5.81°) was seen in OA and PD, with large vertical saccades (15°) being least accurate. 12 Reliability assessment showed a median difference between sessions of <1° for both 13 groups, with poor to good relative agreement (Spearman rho: 0.14 to 0.85). Greater 14 accuracy and reliability was observed in people without visual correction.
15 Saccade amplitude can be measured with variable accuracy and reliability using a mobile 16 eye-tracker in OA and PD. Human, technological and study-specific protocol factors may 17 introduce error and are discussed along with methodological recommendations.
18 Keywords: Parkinsons disease, mobile eye-tracking, accuracy, reliability, saccades, 19 walking
20 1. Introduction
21 Eye-tracking provides data regarding the acquisition of visual information, which is crucial for 22 the safe and effective performance of many real-world activities. Eye-tracking devices have 23 become increasingly popular for investigating visual deficits in people with Parkinsons 24 disease (PD) and older adults (OA) [1, 2]. Previous eye-tracking studies have typically 25 measured visual activity in static laboratory settings [3]. More recently, mobile eye-tracking 26 devices have allowed researchers to investigate the influence of both PD and ageing on 27 visual exploration during real-world activities such as walking and obstacle crossing [1, 2]. 28 Both mechanistic and clinical research requires accurate and reliable devices. However, a 29 recent review [1] highlighted that previous studies do not report the accuracy or reliability of 30 their eye-tracking devices. This is likely due to a lack of gold-standard device or protocol for 31 comparison. As such, there is sparse information regarding the psychometric properties of 32 mobile eye-tracking devices in people with PD and OA.
33 Previous studies [4-7] have evaluated reliability of static eye-tracking devices in various 34 clinical populations, measuring saccades for specific phenomena using highly specialised 35 protocols. For example, Farzin et al. (2011) [7] reported that their static eye-tracker (Tobii, 36 T120, 300Hz) was reliable in reporting number and duration of fixations, and pupillary 37 response during a seated picture-viewing protocol in Fragile-X syndrome patients and 38 controls. Similarly, other studies have assessed reliability of eye-movement characteristics 39 measured with static devices but focus on specific assessments such as anti- or pro40 saccade tests [4, 5, 8], and attribute reliability differences to disease-related influences 41 rather than the device [4]. Results of these highly specialised protocols are not easily 42 generalised, highlighting the need for a standardised protocol.
43 A previous study reported the accuracy of a desk-mounted Tobii eye-tracker (TX300, 300 44 Hz) was 0.5° [9] when participants walked on a treadmill and look at targets on a screen at 45 various locations. The static device had a high sampling-frequency (300Hz) and accounted
2
46 for head movement as long as participants stayed within 200cm of the screen. As such, the 47 results may not apply to head-mounted mobile eye-tracking devices which capture at lower 48 frequencies (i.e. 50-60Hz) but do not require movement to be restricted [10].
49 Our previous work [11] has shown that mobile eye-trackers can accurately detect saccades, 50 however little is known about the accuracy or reliability of specific saccade characteristics 51 (e.g. amplitude) recorded via mobile eye-trackers during static or dynamic tasks [1]. This is 52 important as such characteristics can inform disease-related impairment. This study aimed 53 to evaluate accuracy and re-test reliability of a mobile eye-tracker in measurement of 54 saccade amplitude in people with PD and OA when sitting, standing and walking. Due to the 55 lack of information we developed a simple protocol using visual targets placed at set 56 distances, which could be used to evaluate other devices and across different populations.
57 2. Materials and Methods
58 2.1 Participants
59 Fourteen people with PD were recruited through local Movement Disorders clinics along with 60 20 age-matched OA through advertisement within the local area.
61 Inclusion criteria for all participants were: ≥50 years, normal or corrected-to-normal vision 62 (<18/6 on the Snellen visual acuity), non-demented cognitive status (≥21 on the Montreal 63 cognitive assessment (MoCA) [12]), independently mobile indoors without a walking aid, 64 absence of any neurological problem (other than PD for that group) or severe co-morbidity 65 affecting gait.
66 PD specific inclusion criteria were; a diagnosis of idiopathic PD (by a consultant neurologist 67 with a special interest in movement disorders) and mild-moderately severe symptoms 68 (Hoehn and Yahr (H&Y) stage I-III). PD participants were excluded if they presented with 69 severe dyskinesia or experienced prolonged off periods. PD participants were tested on the 70 peak dose of their anti-Parkinsons medication.
3
71 2.2 Equipment
72
2.2.1 Dikablis Mobile Eye-tracker
73 A Dikablis (Ergoneers GmbH, Germany) mobile (head-mounted) infra-red eye-tracker 74 measured saccade amplitude (distance between two fixations), which has an adequate 75 sampling frequency (50Hz) to detect saccades [11, 13]. The Dikablis consisted of a light76 weight head-unit and transmitter (weight: 69g). The head-unit was double-sided taped to 77 each participants forehead to prevent slippage error. The dual-camera system consisted of 78 a monocular infra-red eye-camera to track pupil blackness and a fish-eye field-camera to 79 record the environment in front of the participant. The system was calibrated using the 80 manufacturers four-point procedure (Figure 1) for each participant before data acquisition. 81 Calibration created a shared coordinate system relating the position of the pupil captured by 82 the eye-camera with the gaze direction displayed on the field-of-view camera [11].
83
84 Figure 1 Calibration board and procedure. Participants were seated and had a chin rest in place, and were
85
then asked to move only their eyes to look at the targets on the board (65cm square) starting at the bottom left
86
target and continuing in a clockwise direction.
87
2.2.2 Monitoring Head Movement
88 Head and eye-movements are interdependent [14]. Head movement can impact saccade 89 amplitude measurement when the head is unconstrained [15]. Therefore, head movement 90 was recorded using a tri-axial accelerometer (Axivity AX3, York, 100Hz) fixed to the Dikablis 91 head-unit to examine whether head movement affected our findings.
4
92 2.3 Protocol
93 The study consisted of two sessions, one week apart. Accuracy was assessed using data 94 from session 1 and re-test reliability was assessed using data from both sessions. Prior to 95 testing, participants underwent demographic, clinical and cognitive assessments (MoCA and 96 Mini Mental State Examination (MMSE)).
97
2.3.1 Accuracy (session 1)
98 Accuracy of saccade amplitude was examined by tracking eye-movements as participants 99 looked between two targets placed at set distances (5°, 10° and 15°, Figure 2) in time with a 100 metronome (1 Hz) for 20seconds. A maximal target distance of 15° was chosen because 101 most naturally occurring saccades occur within this range [16]. Beyond 15°, co-ordinated 102 eye-head movement is required [17]. A brief (30second) rest was permitted after each trial to 103 avoid fatigue, as previous studies have reported that fatigue occurs after a sequence of 104 36seconds of eye-movements [18].
105 Eye-Movement Procedure:
106 Two highly salient targets (coloured red and yellow to attract visual attention) were placed on 107 a white board 200cm from the participant, with the central target at eye-level (Figure 2). 108 Participants were instructed to move their fixation from central to peripheral target (Figure 2). 109 Order was as follows:
110 1) Horizontally: 5°,10°,15° 111 2) Vertically: 5°,10°,15°
112 Tasks: 113 The eye-movement procedure was repeated during: 114 1) Static sitting (with chin rest; restricted head movement) 115 2) Static standing (asked to not move their head; self-restricted head movement)
5
116 3) Walking on a treadmill (Force Link, Netherlands) (head movement permitted). Treadmill
117
speed was set to 80% of that achieved during a 10m walk test carried out at the start of
118
each session. Researchers provided verbal feedback to ensure participants stayed 2m
119
from the testing board.
120
121
Figure 2 - Diagram illustrating the testing board used during sitting, standing and walking
122
2.3.2 Reliability
123 To assess re-test reliability, the same protocol described in section 2.3.1 was repeated 124 approximately one week later (Mean: 7, SD: 2 days). All testing conditions were kept as 125 consistent as possible, with trials conducted by the same researchers (SS, LA) using the 126 same procedure, instructions and testing sequences.
127
2.3.3 Older Adult without Visual Correction
128 To assess potential influence of visual correction (glasses or contact lenses) on accuracy 129 and reliability, data from OA participants who did not require visual correction (n=10) was re130 analysed (Table 3).
6
131 2.4 Data Processing and Analysis
132
2.4.1 Eye and Head Movement
133 Saccade amplitude and head movement were derived using a validated velocity-based 134 algorithm (MATLAB® 2012a, Mathworks, USA) [11], which accounts for small catch up 135 saccades that follow large saccades to locate a target (i.e. saccades occurring within 100ms 136 of a previous saccade are summed to provide total distance). To quantify head movement 137 impact on saccade amplitude, raw vertical and horizontal eye position data was compared to 138 medio-lateral and superior-inferior head accelerations using cross-correlations (peak139 correlation) as a measure of combined eye-head movement [19-22]. Head accelerations 140 were low-pass filtered using a 4th order 30Hz Butterworth filter [21, 23].
141
2.4.2 Statistical Analysis
142 Statistical analysis was performed using SPSS 21.0 (SPSS Inc., IL). Data were assessed for 143 normality using KolmogorovSmirnov tests. Between groups (PD and OA) comparison of 144 saccade amplitude was not performed as this was not the study focus.
145 As the majority of variables were non-normally distributed, we did not calculate intra-class 146 correlation. Instead, we describe accuracy in terms of bias and consistency of saccades. 147 Bias was determined by subtracting known target distance from median saccade amplitude 148 measured using the eye-tracker (median saccade amplitude target distance). Consistency 149 was calculated as the range (Maximum, Minimum) of error between measured and target 150 saccade amplitude across participants.
151 Re-test reliability was described using median and range of between-session difference 152 (median session 2 median session 1), and formally tested using Wilcoxon signed-rank 153 tests for each target amplitude. Relative agreement between sessions was assessed using 154 Spearmans rho correlations. Correlation coefficients were interpreted as follows:
7
155 excellent >0.90, good ≥0.75-0.89, fair ≥0.50-0.74, and poor <0.49 [24]. A threshold of p<0.05 156 guided interpretation.
157 3. Results
158 3.1 Demographics
159 Participant characteristics are described in Table 1. Several participants (OA n=2, PD n=1) 160 were unable to complete session 2 but their data was retained for the accuracy analysis. 161 There were no significant group differences in age, sex or education level. Participants wore 162 any visual correction they usually wore to walk during testing, with significantly more PD 163 participants wearing visual correction (p=0.03). The PD group had moderate motor 164 symptoms as assessed using the MDS-UPDRS-III and H&Y-scale.
165 Table 1 - Demographics
Characteristic Age (yrs), Sex, n (%)
Men Women Height (cm) Weight (kg)
Older adults (n=20) median (range) 68.5 (51, 86)
Parkinsons disease (n=14) median (range) 68.0 (61, 81)
12 (60%) 8 (40%) 170.5 (143, 184) 72.9 (58, 101)
9 (64%) 5 (36%) 168.5 (150, 183) 78.3 (51, 107)
p-value .88
.85 .85 .36
Glasses, n (%)
None
10 (50%)
2 (14.2%)
-
Bifocals
2 (10%)
4 (28.6%)
-
Varifocals
4 (20%)
4 (28.6%)
-
Contact lenses
3 (15%)
0 (0%)
-
Distance
1 (5%)
4 (28.6%)
-
Glasses Worn During Testing
10 (50%)
12 (86%)
.03*
MMSE
30 (26, 30)
29 (24, 30)
.26
MoCA
28 (21, 30)
27 (23, 30)
.42
Years of Education
13 (7, 20)
12 (10, 19)
.31
H & Y stage (n)
-
I (4), II (8), III (2)
-
UPDRS-III
-
34.5 (8, 63)
-
10m Walk (sec)
7.73 (5.97, 13.84)
8.14 (6.01, 13.73)
.55
Walk speed (km/hr)
4.67 (2.61, 6.05)
4.43 (2.63, 6.01)
.58
166 [MMSE: Mini-Mental State Examination; MoCA: Montreal Cognitive Assessment; UPDRS-III: Unified Parkinsons
167 disease Rating Scale motor symptoms, H & Y stage: Hoehn and Yahr stage *: p<.05]
8
168 3.2 Eye and Head Movement 169 Low cross-correlation coefficients indicated that head movement did not influence saccade 170 amplitude (r ranged from 0.01 to 0.12 for walking; see supplementary material 1). As such, 171 standing and walking head movement data was not included in further analyses. The poor 172 correlations were likely due to the maximum target distance of 15°, as saccades greater than 173 20° are needed to elicit combined eye-head movement [25, 26]. 174 3.3 Accuracy 175 Overall, saccade amplitude consistently increased with target distance (Table 2). In relation 176 to overall accuracy, bias of -1.23° and -1.17° was observed for PD and OA participants 177 respectively. However, poor consistency (large range of error between participants) was 178 observed within each group (PD: -7.48° to 5.18°; OA: -7.73° to 5.81°), which was dependent 179 upon target distance (5°, 10°, 15°) and direction (horizontal, vertical). Task (sitting, standing, 180 walking) did not significantly affect accuracy. 181 Table 2 shows that the magnitude of bias generally increased with the magnitude of eye182 movement (e.g. sitting 5°= -0.19°, 10°= -2.69°, 15°= -5.66°). Similarly, both groups tended to 183 undershoot targets set 10° (e.g. -2.63°) and 15° (e.g. -4.94°) apart, which was consistent for 184 all tasks. In addition, the range of error was greatest for larger saccades (e.g. 10°= -4.08° to 185 0.28° and 15°= -7.48° to 2.31°). 186 Bias was also related to saccade direction (horizontal, vertical), such that participants 187 undershot the target distance considerably more when performing vertical compared to 188 horizontal saccades.
9
189 190
Table 2 Accuracy (session 1) and re-test reliability (comparison between session 1 and session 2)
Accuracy (Session 1) (Saccade Amplitude (°))
Re-test Reliability (Session 2) (Saccade Amplitude (°))
Median
Task
Direction ° Median (Min, Max) Bias
Range of Error
Median (Min, Max)
Difference
Range of Difference
p-value Spearmans rho (p-value)
Older
Sitting
Horizontal 5
5.69 (4.84, 9.56) 0.69
-0.16, 4.56
5.96 (4.41, 8.08)
-0.03
-5.51, 2.20
0.98
0.42 (0.07)
Adults (n=20)
Vertical
10 10.23 (7.66, 13.18) 0.23
15 12.71 (9.87, 14.52) -2.29
5
4.88 (4.05, 7.00) -0.12
10 7.42 (6.20, 11.77) -2.58
15 9.55 (7.27, 13.70) -5.45
-2.34, 3.18 -0.13, 4.52 -0.95, 2.00 -3.80, 1.77 -7.73, -1.30
9.87 (8.59, 13.50) 13.28 (10.93, 14.71)
5.13 (4.05, 21.09) 7.74 (6.34, 20.90) 9.84 (7.85, 20.70)
-0.09 0.45 0.21 0.07 0.26
-8.28, 3.35
0.60
-11.76, 2.03
0.27
-7.00, 16.75
0.14
-6.52, 12.53
0.32
-8.37, 12.15
0.29
0.35 (0.14) 0.20 (0.42) 0.34 (0.16) 0.27 (0.27) 0.27 (0.38)
Median
-
-1.21
-7.73, 4.56
-
-
-
-
-
Standing Horizontal 5 6.16 (4.77, 10.81) 1.16
-0.23, 5.81
6.38 (4.98, 9.76)
-0.22
-6.23, 4.64
0.90
0.48 (0.30)
Vertical
10 10.01 (4.77, 10.81) 0.01 15 12.68 (10.51, 14.77) -2.32 5 5.15 (3.98, 10.38) 0.15 10 7.55 (5.81, 11.97) -2.45 15 10.17 (7.96, 12.00) -4.83
-5.23, 4.77 -4.49, -0.23 -1.02, 5.38 -4.19, 1.97 -7.04, -3.00
10.57 (8.48, 14.46) 13.22 (10.91, 13.99)
4.98 (4.05, 15.96) 7.58 (5.95, 19.03) 9.79 (7.11, 21.15)
0.39 0.06 -0.27 0.32 -0.36
-7.92, 2.62
0.55
-11.69, 2.83
0.81
-4.65, 11.13
0.35
-6.22, 11.32
0.11
-8.68, 9.16
0.89
0.36 (0.13) 0.21 (0.39) 0.30 (0.21) 0.61 (0.005) 0.66 (0.002)
Median
-
-1.16
-7.04, 5.81
-
-
-
-
-
Walking Horizontal 5
5.41 (4.68, 8.16) 0.41
-0.32, 3.16
5.81 (4.30, 9.60)
0.21
-5.59, 4.92
0.07
0.30 (0.28)
Vertical
10 9.59 (7.02, 14.48) -0.41
15 13.07 (9.55, 14.37) -1.93
5
4.93 (4.46, 7.24) -0.07
10 7.22 (5.52, 9.35) -2.78
15 10.21 (7.87, 12.01) -4.79
-2.98, 4.48 -5.45, -0.63 -0.54, 2.24 -4.28, -0.65 -7.13, -2.99
9.44 (7.33, 13.79) 11.96 (10.25, 13.41)
5.22 (4.17, 7.53) 7.43 (5.86, 9.12) 10.63 (7.93, 12.06)
-0.55 -0.95 -0.04 -0.09 0.10
-8.71, 3.05 -12.60, 3.51 4.90, 2.97 -6.67, 2.10 -8.22, 2.86
0.88 0.02* 0.34 1.00 0.32
0.26 (0.29) 0.14 (0.57) 0.53 (0.24) 0.45 (0.06) 0.75 (0.001)
Median
-
-1.17
-7.13, 4.48
-
-
-
-
-
Group Median
-
-1.17
-7.73, 5.81
-
0.02
-12.60, 12.53
-
-
Parkinsons Disease (n=14)
Sitting
Horizontal 5
5.81 (4.45, 6.74) 0.81
10 9.52 (7.02, 13.40) -0.48
15 12.31 (8.80, 14.98) -2.69
Vertical
5
4.81 (4.03, 6.26) -0.19
10 7.31 (6.01, 9.00) -2.69
15 9.34 (7.80, 11.70) -5.66
-0.55, 1.74
-2.98, 3.40 -6.20, -0.02 -0.97, 1.26 -3.99, -1.00 -7.20, -3.30
6.10 (4.99, 7.74)
9.80 (7.59, 12.69) 12.56 (10.24, 14.01)
4.76 (4.05, 6.87) 7.00 (6.04, 10.84) 9.25 (7.89, 11.19)
0.05
-0.25 -0.02 -0.29 -0.55 -0.31
-5.18, 3.19
0.27
-9.08, 2.88
0.89
-11.40, 2.42
0.91
-4.51, 2.12
0.36
-6.97, 2.62
0.69
-8.65, 1.23
0.46
0.17 (0.59)
0.51 (0.07) 0.37 (0.29) 0.14 (0.65) 0.64 (0.18) 0.67 (0.01)
Median
-
-1.59
-7.20, 3.40
-
-
-
-
-
Standing Horizontal 5 5.94 (4.81, 10.18) 0.94
-0.19, 5.18
6.05 (4.32, 7.59)
-0.13
-5.32, 1.37
0.73
0.76 (0.002)
Vertical
10 10.13 (8.20, 12.08) 0.13
15 12.20 (9.90, 13.62) -2.80
5
4.79 (4.25, 5.53) -0.21
10 8.02 (6.10, 12.25) -1.98
15 9.82 (7.54, 11.91) -5.18
-1.80, 2.08 -5.10, -1.38 -0.75, 0.53 -3.90, 2.25 -7.46, -3.09
10.28 (6.91, 13.50) 12.50 (10.13, 17.47)
4.56 (3.91, 11.08) 7.52 (6.08, 10.14) 9.11 (7.19, 12.54)
-0.21 0.45 -0.08 -0.41 -0.75
-9.53, 2.23
0.24
-10.63, 5.03
0.15
-4.58, 6.63
0.37
-6.63, 1.42
0.51
-8.65, 1.10
0.10
0.85 (0.000) 0.64 (0.02) 0.38 (0.20) 0.38 (0.20) 0.50 (0.08)
Median
-
-1.10
-7.46, 5.18
-
-
-
-
-
Walking Horizontal 5
5.62 (4.65, 9.90) 0.62
-0.35, 4.90
5.58 (4.95, 6.24)
-0.01
-5.15, 0.91
0.62
0.20 (0.51)
Vertical
10 9.70 (6.29, 12.94) -0.30
15 12.38 (8.53, 13.82) -2.62
5
4.80 (4.35, 6.98) -0.20
10 7.37 (5.92, 10.28) -2.63
15 10.06 (7.52, 12.31) -4.94
-3.71, 2.94 -6.47, -1.18 -0.65, 1.98 -4.08, 0.28 -7.48, 2.31
9.93 (7.99, 13.00) 12.92 (11.09, 15.67)
4.68 (4.32, 5.77) 6.95 (5.83, 16.30) 9.52 (7.28, 11.67)
0.15 0.23 -0.15 -0.11 -0.27
-8.82, 2.11
0.20
-11.40, 5.24
0.16
-4.45, 0.72
0.10
-6.63, 6.55
0.67
-8.68, 1.45
0.21
0.63 (0.02) 0.14 (0.65) 0.44 (0.13) 0.45 (0.13) 0.80 (0.001)
Median
-
-1.46
-7.48, 4.90
-
-
-
-
-
Group Median
-
-1.23
-7.48, 5.18
-
-0.14
-11.40, 5.24
-
-
Overall Median
-
-1.21
-7.73, 5.81
-
-0.09
-12.60, 16.75
-
-
[*Significance level p<0.05]
10
191 3.4 Reliability 192 Overall, median difference (session 2 session 1) in saccade amplitude was low in both 193 groups (PD; -0.14°, OA; 0.02°, Table 2). Similarly, median difference for individual tasks and 194 amplitudes (Table 2) was low (<1°). Only one variable (OA; walking, horizontal, 15°) showed 195 a significant difference between sessions (p=0.02) but the median difference was still low (196 0.95°). However, there was a wide range of difference between sessions across the 197 participants (-12.60° to 16.75°). Relative agreement varied greatly from poor to good (rho 198 range: 0.14, 0.85). Test condition did not have a consistent influence on bias or relative 199 agreement. In contrast, larger saccades were associated with a greater range of change 200 between sessions. 201 3.5 Influence of Visual Correction 202 Greater accuracy and re-test reliability results were found in the sub-set of OA with no vision 203 correction (Table 2 and 3). With regards to accuracy, median bias from target reduced from 204 1.17° to -1.15° and error was more consistent across the participants. Median difference in 205 saccadic amplitude between sessions (reliability) was similar but between-person range was 206 much smaller. Modest improvements were also seen in relative agreement between 207 sessions when considering people who did not use visual correction. 208 209
11
210 Table 3 Accuracy (Session 1) and re-test reliability (comparison of Session 1 and Session 2) of older adults with no vision correction (n=10)
Accuracy (Saccade amplitude (°))
Task Sitting
Standing
Walking
Direction
°
Horizontal
5
10
15
Vertical
5
10
15
Median
Horizontal
5
10
15
Vertical
5
10
15
Median
Horizontal
5
10
15
Vertical
5
10
15
Median
Overall Median
211 [*Significance level p<0.05]
Session 1 Median (Min, Max)
5.58 (4.84, 7.48) 9.86 (7.66, 12.35) 13.13 (9.87, 14.52) 4.75 (4.05, 5.35) 6.76 (6.20, 9.03) 9.14 (7.27, 10.88)
5.97 (4.77, 7.17) 10.01 (7.98, 14.42) 12.80 (10.85, 14.77) 4.76 (3.98, 6.10) 6.57 (5.81, 8.16) 9.55 (7.96, 11.12)
5.40 (4.80, 5.77) 9.93 (7.02, 14.30) 13.85 (10.46, 14.37) 4.81 (4.58, 7.24) 7.14 (4.58, 7.24) 9.97 (7.87, 10.89)
-
Bias 0.58 -0.14 -1.87 -0.25 -3.24 -5.86 -1.06 0.97 0.01 -2.20 -0.24 -3.43 -5.45 -1.22 0.40 -0.07 -1.15 -0.19 -2.86 -5.03 -0.67 -1.15
Range of Error -0.16, 2.48 -2.34, 2.35 -5.13, -0.48 -0.95, 0.35 -3.80, -0.97 -7.73, -4.12 -7.73, 2.48 -0.23, 2.17 -2.02, 4.42 -4.15, 4.77 -1.02, 1.10 -4.19, -1.84 -7.04, -3.88 -7.04, 4.77 -0.20, 0.77 -2.98, 4.30 -4.56, 4.37 -0.42, 2.24 -5.42, -2.76 -7.13, -4.11 -7.13, 4.37 -7.73, 4.77
Re-test Reliability (Saccade Amplitude (°))
Session 2 Median (Min, Max)
5.91 (5.21, 6.98) 9.48 (8.59, 13.50) 12.78 (10.93, 14.54) 4.88 (4.05, 5.42) 7.42 (6.40, 9.00) 9.70 (7.85, 11.44)
5.89 (4.98, 7.47) 10.41 (8.48, 12.61) 13.20 (10.91, 13.84) 4.92 (4.05, 5.57) 7.04 (5.95, 8.32) 8.82 (7.11, 10.43)
5.76 (4.30, 6.13) 8.86 (7.33, 13.23) 12.47 (10.82, 13.41) 5.24 (4.17, 6.11) 6.83 (5.86, 8.05) 9.21 (7.93, 11.08)
-
Median Difference 0.24 -0.09 0.27 0.04 0.43 0.64 0.23 0.20 -0.06 0.12 0.32 -0.48 0.09 -0.63 -1.19 0.19 -0.09 0.04 0.11
Range of Difference -0.52, 1.34 -2.87, 3.35 -2.10, 1.63 -0.83, 0.94 -2.30, 1.78 -1.04, 1.43 -0.56, 1.44 -2.59, 2.62 -1.42, 1.96 -1.06, 1.18 -1.27, 1.61 -2.89, 0.70 -4.80, 0.82 -8.37, 2.30 -10.49, 0.12 -4.90, 0.72 -6.29, 0.95 -8.01, 0.84 -10.49, 3.35
p-value 0.14 1.00 0.95 0.36 0.26 0.07 0.38 0.84 0.92 0.88 0.26 0.15 0.37 0.40 0.008* 0.40 0.35 1.00 -
Spearmans rho (p-value) 0.29 (0.42) 0.89 (0.05) 0.33 (0.35) 0.13 (0.73) 0.83 (0.08) 0.76 (0.01) -
0.77 (0.009) 0.32 (0.36) 0.20 (0.59) 0.17 (0.65) 0.53 (0.12) 0.43 (0.21)
0.40 (0.28) 0.23 (0.56) 0.43 (0.25) 0.44 (0.24) 0.42 (0.27) 0.74 (0.02)
-
12
212 4. Discussion
213 To our knowledge, this is the first study to examine accuracy and reliability of a mobile eye214 tracker in people with PD and OA. Results provide evidence that mobile eye-trackers can 215 measure saccade amplitude in people with PD and OA although the accuracy and reliability 216 depend on several factors. Findings contribute to the development of novel protocols for 217 establishing the psychometric properties of mobile eye-trackers.
218 4.1 Accuracy 219 Median saccade amplitude measured by the mobile eye-tracker, increased with increasing 220 target distance (Table 2). This indicates that the mobile eye-tracker can discern change in 221 saccade amplitude. However, the measured saccade amplitudes were smaller than target 222 distance (5°, 10°, 15°), especially for larger and vertical saccades. In addition, bias was 223 inconsistent across the participants, especially for larger saccades.
224 Although our previous work has shown mobile eye-trackers can accurately detect saccade 225 occurrence [11], this study indicates saccade amplitude may not be measured with the same 226 degree of certainty. This suggests that saccade detection outcomes (number or frequency) 227 are more robust than saccade amplitude. Regardless, overall median bias (-1.21°) and 228 consistency (-7.73° to 5.81°) is acceptable for certain protocols, such as dynamic protocols 229 involving saccade detection which often use a minimum threshold of ≥5° saccade amplitude 230 [2] to account for artefact error (e.g. vestibular-ocular-reflex) [11]. However, this degree of 231 accuracy may not be acceptable for protocols where precision of large saccade amplitude is 232 important.
233 4.2 Reliability 234 Re-test reliability varied across conditions and participants. Although median difference 235 between sessions was low (<1°), difference ranged from -12.60° to 16.75° across 236 participants. Similarly, relative agreement ranged from poor to good between conditions (rho;
13
237 0.14 to 0.85). Variable reliability indicates that saccade amplitude measurement may not be 238 stable over time and is likely due to several sources of error (see section 4.3). Until robust 239 protocols are developed which are stable over time, we cannot recommend saccade 240 amplitude as a reliable mobile eye-tracker outcome.
241 4.3 Potential Challenges and Recommendations 242 Error noted in both accuracy and reliability stems from technological, human and study243 protocol factors. A better understanding of these sources of error is important for future 244 protocols and devices.
245
4.3.1 Technology Factors
246 Manufacturer reported accuracy (0.5°) was not observed in this study. In contrast, a 247 preliminary study (four young adults) using a static eye-tracker (Tobii, TX300; 300Hz) during 248 treadmill walking reported eye-tracker accuracy was consistent with manufacturer 249 specifications (0.5°) regardless of target locations or saccade amplitude [9, 27]. Overlooking 250 the preliminary nature of the referenced study [9], inconsistency between the current study 251 and this previous report may be due to the lower sampling-frequency of the mobile eye252 tracker used in this study (50Hz) compared to the static device (300Hz) [10]. A sampling253 frequency of 50Hz enables saccade detection [13] but higher frequency (>200Hz) devices 254 may be more accurate at reporting specific saccade characteristics [1]. For example; a 255 sampling-frequency of 50Hz assumes that the eye is in a fixed location for 20ms (50Hz) 256 whereas a higher frequency system (1000Hz) assumes this for only 1ms, providing better 257 temporal accuracy and more eye-position data [10, 13]. Therefore, a mobility-accuracy 258 trade-off exists. Static higher sampling-frequency devices may offer improved accuracy and 259 reliability but in order to use them, studies must limit participant mobility during dynamic 260 tasks. That is, participants must walk on a treadmill and be at a set distance from visual 261 targets [9]. However, protocols which limit mobility can limit validity the characteristics
14
262 measured [28]. For example, restricted head movements during static protocols may 263 facilitate abnormal visual processing, seen through alterations in saccade responses [29].
264 Some bias may be due to eye curvature induced error [30]. The eye is a convex curved lens 265 with a horizontal movement range of ~100° and vertical range of ~90° [31]. Many eye266 trackers locate the pupil via the black pixels recorded by an infra-red eye-camera and uses 267 specific circular pupil shape parameters to derive the pupil centre. Depending upon the 268 location of the eye-camera in relation to the eye, the pupil shape will appear as an ellipse 269 and therefore the circular pupil shape parameters would lead to inaccurate tracking. This is 270 most relevant for large saccades, where the person is looking furthest from the camera. The 271 Dikablis eye-tracker demonstrated such an error by recording an undershoot for all targets 272 at 15° and may have contributed to the poorer accuracy in seen for 15° saccades. This error 273 could be controlled for in future technology with the use of convex cost function algorithms 274 [32] or corneal reflexion tracking [33], which would provide further means of tracking eye-in275 head movements [34] and control for pupil tracking errors [35].
276
4.3.2 Human Factors
277 4.3.2.1 Visual Correction and Obstruction of the Eye
278 Pupil tracking was likely impacted by a number of general eye-tracker issues, such as 279 inaccuracies due to poor calibration [36], long or drooping eye lashes/lids, infra-red refraction 280 due to visual correction ( glasses), hair obstruction and any slippage of the one-size-fits-all 281 eye-tracker from original placement when recording [13]. During data collection eye 282 lids/lashes and visual correction (particularly bi-focal glasses) were observed as main cause 283 of error, particularly for vertical saccades and large saccades of any direction. These 284 challenges are inherent to infra-red eye-tracking devices and although some can be 285 controlled within an experiment, many are dependent upon researcher ability to identify and 286 address these issues. For example, using double-sided tape to minimise device slippage
15
287 and requesting participants not wear eye make-up were ways which we found anecdotally 288 improved accuracy.
289 We assessed whether visual correction may have impacted accuracy and re-test reliability 290 by looking at a subset of 10 OA who wore no visual correction. Results showed that the 291 accuracy and reliability were better in individuals who did not use visual correction, likely due 292 to visual correction affecting pupil detection via infra-red refraction [13]. Unfortunately, 293 exclusion of participants with visual correction may not be appropriate when selecting 294 participants for research studies, particularly with groups likely to have increased use of 295 visual correction such as OA. Therefore, the negative effect of visual correction on eye296 tracker accuracy and reliability must be considered when designing robust protocols and is a 297 challenge which still needs to be addressed by manufacturers of the next generation of eye298 trackers.
299 4.3.2.2 Visual Attention 300 Participant saccades were voluntary and therefore involved selective visual attention which 301 is influenced by internal factors [37] and may have affected amplitude results. Factors such 302 as level of fatigue between sessions [38], ethnicity of participants [39], prior knowledge of 303 testing protocols (learning effect) [40], individual emotional state [41] and motivation [42] 304 could all have influenced saccade measures. Future studies could control for such factors by 305 investigating saccade latencies compared to auditory signal, or quantifying total saccade 306 number to compare to a set amount (i.e. 20 saccades within 20 seconds).
307 In addition, this study did not consider the inhibition-of-return mechanism whereby a person 308 orientates their attention to novel locations and stimuli, as our target appearance, location 309 and saliency [43] remained the same. Once a peripheral location is foveated (fixated) there 310 is a delayed response in returning attention to subsequent stimuli in the same location [44]. 311 Programming of the next saccade occurs even before the previous saccade is completed 312 [45], therefore introducing a time constraint (1 second) and using the same targets/locations
16
313 may have led to inaccuracies in saccade programming and execution. Therefore, some of 314 the error observed in this study may have been due to inaccurate saccades rather than error 315 introduced by the mobile eye-tracker.
316
4.3.3 Study Protocol Limitations
317 Future work should address the limitations of this study to establish a gold standard method 318 to be applied to differing devices and various populations. Novel peripheral targets in varying 319 locations which require reflexive (involuntary) saccades should be used, with variations on 320 saccadic timings. For example; a light board or computer-based programme where objects 321 or targets randomly appear (similar to that used by Serchi, Peruzzi [9] for their static eye322 tracker) could be used with mobile devices. Future studies could also examine the impact of 323 combined eye-head movement on saccade amplitude accuracy, particularly for larger 324 saccades (>20°) where coordinated eye-head movement is required.
325 5. Conclusion
326 The Dikablis mobile eye-tracker had variable accuracy and reliability when recording 327 saccade amplitude in people with PD and OA. Accuracy is acceptable for certain protocols 328 but more precision may be necessary when investigating specific saccade characteristics. 329 Error was induced via several technological, human and study-specific factors which need to 330 be addressed to achieve more robust testing protocols.
331 Acknowledgements
332 The authors acknowledge Aodhán Hickey (Research Technician, Newcastle University) for 333 his technical support. 334 Conflict of Interest 335 None.
17
336 Funding
337 This research is supported by the National Institute for Health Research (NIHR) Newcastle
338 Biomedical Research Unit (BRU) and centre (BRC) based at Newcastle-upon-Tyne
339 Hospitals NHS Foundation Trust and Newcastle University. The research was also
340 supported by NIHR Newcastle CRF Infrastructure funding. The views expressed are those
341 of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
342 Ethical Approval
343 Ethical approval was obtained via local research ethics committee (Newcastle and North-
344 Tyneside REC-2; 13/NE/0128). Participants provided written informed consent prior to
345 testing.
346 References
347 [1] Stuart S, Alcock L, Galna B, Lord S, Rochester L. The measurement of visual sampling 348 during real-world activity in Parkinson's disease and healthy controls: A structured literature 349 review. Journal of neuroscience methods. 2014;222:175-88. 350 [2] Galna B, Lord S, Daud D, Archibald N, Burn D, Rochester L. Visual sampling during 351 walking in people with Parkinson's disease and the influence of environment and dual-task. 352 Brain research. 2012;1473:35-43. 353 [3] Pelz JB, Canosa R. Oculomotor behavior and perceptual strategies in complex tasks. 354 Vision research. 2001;41:358796. 355 [4] Blekher T, Weaver MR, Cai X, Hui S, Marshall J, Jackson JG, et al. Test-retest reliability 356 of saccadic measures in subjects at risk for Huntington disease. Investigative ophthalmology 357 & visual science. 2009;50:5707-11. 358 [5] Klein C, Fischer B. Instrumental and test-retest reliability of saccadic measures. Biol 359 Psychol. 2005;68:201-13. 360 [6] Farris-Trimble A, McMurray B. Test-retest reliability of eye tracking in the visual world 361 paradigm for the study of real-time spoken word recognition. Journal of speech, language, 362 and hearing research : JSLHR. 2013;56:1328-45. 363 [7] Farzin F, Scaggs F, Hervey C, Berry-Kravis E, Hessl D. Reliability of eye tracking and 364 pupillometry measures in individuals with fragile X syndrome. Journal of autism and 365 developmental disorders. 2011;41:1515-22. 366 [8] Ettinger U, Kumari V, Crawford TJ, Davis RE, Sharma T, Corr PJ. Reliability of smooth 367 pursuit, fixation, and saccadic eye movements. Psychophysiology. 2003;40:620-8.
18
368 [9] Serchi V, Peruzzi A, Cereatti A, Della Croce U. Tracking gaze while walking on a 369 treadmill: Spatial accuracy and limits of use of a stationary remote eye-tracker. Engineering 370 in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the 371 IEEE2014. p. 3727-30.
372 [10] Andersson R, Nyström M, Holmqvist K. Sampling frequency and eye-tracking measures: 373 how speed affects durations, latencies, and more. Journal of Eye Movement Research. 374 2010;3:1-12.
375 [11] Stuart S, Galna B, Lord S, Rochester L, Godfrey A. Quantifying Saccades While 376 Walking: Validity of a Novel Velocity-Based Algorithm for Mobile Eye Tracking. Engineering 377 in Medicine and Biology Society, 2014 EMBS 2014 36th Annual International Conference of 378 the IEEE. Chicago, Illinois, USA: IEEE; 2014.
379 [12] G Bernardi M. Mild Cognitive Impairment is Under-Recognised in Newly Referred 380 Patients with Parkinsons Disease: Mini Mental State Examination (MMSE) Versus Montreal 381 Cognitive Assessment (Moca). Journal of Alzheimers Disease & Parkinsonism. 2012;01.
382 [13] Holmqvist K, Nystrom M. Eye tracking: A comprehensive guide to methods and 383 measures. Oxford, UK: Oxford University Press; 2011.
384 [14] Freedman EG. Interactions between eye and head control signals can account for 385 movement kinematics. Biol Cybern. 2001;84:453-62.
386 [15] Proudlock FA, Shekhar H, Gottlob I. Age-related changes in head and eye coordination. 387 Neurobiology of aging. 2004;25:1377-85.
388 [16] Bahill AT, Adler D, Stark L. Most naturally occurring human saccades have magnitudes 389 of 15 degrees or less. Investigative ophthalmology. 1975;14:468-9.
390 [17] Maurer C, Mergner T, Lucking CH, Becker W. Adaptive changes of saccadic eyehead 391 coordination resulting from altered head posture in torticollis spasmodicus. Brain. 392 2001;124:41326.
393 [18] Wilson SJ, Glue P, Ball D, Nutt DJ. Saccadic eye movement parameters in normal 394 subjects. Electroencephalography and clinical neurophysiology. 1992;86:69-74.
395 [19] Lee C. Eye and head coordination in reading: roles of head movement and cognitive 396 control. Vision research. 1999;39:3761-8.
397 [20] Pelz J, Hayhoe M, Loeber R. The coordination of eye, head, and hand movements in a 398 natural task. Experimental brain research. 2001;139:266-77.
399 [21] Kavanagh JJ, Barrett RS, Morrison S. Upper body accelerations during walking in 400 healthy young and elderly men. Gait & posture. 2004;20:291-8.
401 [22] Kavanagh JJ, Menz HB. Accelerometry: a technique for quantifying movement patterns 402 during walking. Gait & posture. 2008;28:1-15.
403 [23] Kavanagh JJ, Morrison S, Barrett RS. Coordination of head and trunk accelerations 404 during walking. Eur J Appl Physiol. 2005;94:468-75.
405 [24] Rosner BA. Fundamentals of Biostatistics. illustrated ed. Boston, USA: Thomson406 Brooks/Cole; 2006.
19
407 [25] Crawford JD, Martinez-Trujillo JC, Klier EM. Neural control of three-dimensional eye and 408 head movements. Current opinion in neurobiology. 2003;13:655-62.
409 [26] Gandhi NJ, Sparks DL. Experimental control of eye and head positions prior to head410 unrestrained gaze shifts in monkey. Vision research. 2001;41:3243-54.
411 [27] Serchi V, Peruzzi V, Cereatti A, Della Croce U. Performance of a remote eye-tracker in 412 measuring gaze during walking. 20th IMEKO TC4 International Symposium and 18th 413 International Workshop on ADC Modelling and Testing Research on Electric and Electronic 414 Measurement for the Economic Upturn. Benevento, Italy2014. p. 770-4.
415 [28] Nevalainen S, Sajaniemi J. Comparison of three eye tracking devices in psychology of 416 programming research. Proceedings of the 16th Annual Workshop of the Psychology of 417 Programming Interest Group, Carlow, Ireland2004. p. 151-8.
418 [29] van Stockum S, MacAskill MR, Myall D, Anderson TJ. A perceptual discrimination task 419 results in greater facilitation of voluntary saccades in Parkinson's disease patients. The 420 European journal of neuroscience. 2013;37:163-72.
421 [30] Zhiwei Z, Qiang J. Novel Eye Gaze Tracking Techniques Under Natural Head 422 Movement. Biomedical Engineering, IEEE Transactions on. 2007;54:2246-60.
423 [31] Botha CP, de Graaf T, Schutte S, Root R, Wielopolski P, van der Helm F, et al. MRI424 based Visualisations of Orbital Fat Deformation DUring Eye Motion. In: Linsen L, Hagen H, 425 Hamann B, editors. Visualization in Medicine and Life Sciences. Berlin Heidelberg: Springer426 Verlag; 2008.
427 [32] De Santis A, Iacoviello D. Robust real time eye tracking for computer interface for 428 disabled people. Computer Methods and Programs in Biomedicine. 2009;96:1-11.
429 [33] Mele ML, Federici S. A psychotechnological review on eye-tracking systems: towards 430 user experience. Disability and rehabilitation Assistive technology. 2012;7:261-81.
431 [34] Hennessey CA, Lawrence PD. Improving the accuracy and reliability of remote system432 calibration-free eye-gaze tracking. IEEE transactions on bio-medical engineering. 433 2009;56:1891-900.
434 [35] Li F, Munn S, Pelz J. A model-based approach to video-based eye tracking. Journal of 435 Modern Optics. 2008;55:503-31.
436 [36] Nystrom M, Andersson R, Holmqvist K, van de Weijer J. The influence of calibration 437 method and eye physiology on eyetracking data quality. Behavior research methods. 438 2013;45:272-88.
439 [37] Baluch F, Itti L. Mechanisms of top-down attention. Trends in neurosciences. 440 2011;34:210-24.
441 [38] Faber LG, Maurits NM, Lorist MM. Mental Fatigue Affects Visual Selective Attention. 442 PloS one. 2012;7:e48073.
443 [39] Blignaut P, Wium D. Eye-tracking data quality as affected by ethnicity and experimental 444 design. Behavior research methods. 2014;46:67-80.
445 [40] Kim S, Rehder B. How prior knowledge affects selective attention during category 446 learning: An eyetracking study. Mem Cogn. 2011;39:649-65.
20
447 [41] Oatley K, Parrott WG, Smith C, Watts F. Cognition and Emotion over twenty-five years. 448 Cognition and Emotion. 2011;25:1341-8. 449 [42] Kaplan RL, Van Damme I, Levine LJ. Motivation Matters: Differing Effects of Pre-Goal 450 and Post-Goal Emotions on Attention and Memory. Frontiers in psychology. 2012;3:404. 451 [43] t Hart BM, Schmidt HCEF, Klein-Harmeyer I, Einhäuser W. Attention in natural scenes: 452 contrast affects rapid visual processing and fixations alike2013. 453 [44] Klein RM. Inhibition of return. Trends in cognitive sciences. 2000;4:138-47. 454 [45] McPeek RM, Skavenski AA, Nakayama K. Concurrent processing of saccades in visual 455 search. Vision research. 2000;40:2499-516. 456
21