DissLiteratur/storage/A4L93SIV/.zotero-ft-cache

PHYSICAL REVIEW PHYSICS EDUCATION RESEARCH 15, 010126 (2019)

Visual cues improve students’ understanding of divergence and curl: Evidence from eye movements during reading and problem solving
Pascal Klein,1,* Jouni Viiri,2 and Jochen Kuhn1
1Department of Physics, Physics Education Research Group, Technische Universität Kaiserslautern, Erwin-Schrödinger-Str. 46, 67663 Kaiserslautern, Germany
2Department of Teacher Education, University of Jyväskylä, 40014 Jyväskylä, Finland
(Received 22 November 2018; published 9 May 2019)
The coordination of multiple external representations is important for learning, but yet a difficult task for students, requiring instructional support. The subject in this study covers a typical relation in physics between abstract mathematical equations (definitions of divergence and curl) and a visual representation (vector field plot). To support the connection across both representations, two instructions with written explanations, equations, and visual representations (differing only in the presence of visual cues) were designed and their impact on students’ performance was tested. We captured students’ eye movements while they processed the written instruction and solved subsequent coordination tasks. The results show that students instructed with visual cues (VC students) performed better, responded with higher confidence, experienced less mental effort, and rated the instructional quality better than students instructed without cues. Advanced eye-tracking data analysis methods reveal that cognitive integration processes appear in both groups at the same point in time but they are significantly more pronounced for VC students, reflecting a greater attempt to construct a coherent mental representation during the learning process. Furthermore, visual cues increase the fixation count and total fixation duration on relevant information. During problem solving, the saccadic eye movement pattern of VC students is similar to experts in this domain. The outcomes imply that visual cues can be beneficial in coordination tasks, even for students with high domain knowledge. The study strongly confirms an important multimedia design principle in instruction, that is, that highlighting conceptually relevant information shifts attention to relevant information and thus promotes learning and problem solving. Even more, visual cues can positively influence students’ perception of course materials.
DOI: 10.1103/PhysRevPhysEducRes.15.010126

I. INTRODUCTION
Visual representations have the potential to substantially promote the learning of abstract concepts [1]. However, students must learn about the representations before they can be used for learning or problem solving. This interdependence is known as the representational dilemma [2]. One example which has recently been studied in this context consists of the visual interpretation of twodimensional vector field plots with respect to divergence or curl [3–6]. Vector fields are mathematical structures that assign a vector to every point in space. They are important in many branches of physics, and oftentimes, they are expressed as vector field plots (see Fig. 1). Learning how divergence or curl relate to this representation contributes
*pklein@physik.uni-kl.de
Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

to students conceptual understanding about divergence itself, but also about important laws and theorems that involve divergence, e.g., Maxwell’s theorems or the laws of Gauss and Stokes. In a more general form, the ability to relate mathematical concepts to various representations contributes to representational competence, that is, being able to use multiple representations for understanding [7]. Learning processes with respect to multiple representations have also been described by Ainsworth [8], who argues that multiple representations can contribute to knowledge enhancement: the understanding that students gain by using multiple representations is deeper, more robust, and more flexible than if students use single representation, and research evidence during the past years supports this claim. However, the representational dilemma calls for learning about the representations first, before students can exploit their advantages [2].
For example, making connections between the mathematical definitions of divergence and curl and a vector field plot is a challenging task, even for graduate students [3], hence the acquisition of this skill requires instructional support. Instructions about how multiple representations

2469-9896=19=15(1)=010126(9)

010126-1

Published by the American Physical Society

KLEIN, VIIRI, and KUHN

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

FIG. 1. Left: Graphical representation of a two-dimensional vector field F⃗ ðr⃗ Þ with constant divergence. Students were asked whether divergence was zero or nonzero. Right: Visual cues highlighting the vector field decomposition and framing one row or column for evaluating ∂=∂x and ∂=∂y. For instance, the divergence can be estimated by judging the change of the black arrows in the black box and the red arrows in the red box, cf. Eq. (1). The fields are constructed to have constant zero or nonzero divergence everywhere.

refer to each other and what features contain conceptual information makes it possible for learners to use them for learning and problem solving [2]. The particular subject discussed here focuses on the development of visual understanding concerning the divergence operator in (Cartesian) differential form, i.e.,

curlF⃗ ðr⃗ Þ

¼

∇⃗

×

F⃗

≡

∂Fy ∂x

−



∂Fx ∂y

eˆ z ;

ð2Þ

to two-dimensional vector field plots (Fz ¼ 0) in a similar manner as judging the divergence. Equation (2) was given
to students.

divF⃗ ðr⃗ Þ

¼

∇⃗

·

F⃗

¼

∂Fx ∂x

þ

∂Fy ∂y

;

ð1Þ

where F⃗ ¼ ðFx; Fy; 0Þ, r⃗ ¼ ðx; y; zÞ, and ∇⃗ represents the nabla (del) operator. By definition, the divergence of a vector field can be evaluated qualitatively (zero or nonzero) by interpreting the partial vector derivatives [9]. The coordination between Eq. (1) and the graphical vector field plot requires inspection of the change of both field components in the x and y directions, respectively (Fig. 1). Prior research has shown that students’ performance regarding this problem was only marginal above the chance of guessing correct—even after explicit instruction [6]. Students mainly struggled with the correct interpretation of partial vector derivatives [6,10]. Based on previous work [6], we created an instruction with visual cues that emphasize important conceptual features in the vector field diagram which are crucial to connect Eq. (1) to the field plot. The visual cues were designed to trigger the coordination between text, equation, and the graphical representation. The aim of the current study is to identify how the presence of visual cues influenced students’ instruction processing and understanding, using eyetracking methodology and traditional assessment.
While using very explicit instructions, it might be possible that students will perform well by following the steps provided but lack improving a conceptual understanding of the algebraic equation and the field plot. Therefore, we established a transfer problem that should be solved without instruction, that is, application of the curl operator,

A. Theoretical background
The coordination of information provided in multiple representations with prior knowledge is described by the cognitive theory of multimedia learning (CTML) [11]. CTML identifies three distinct processes (selection, organization, and integration) involved in learning from multiple representations, that can all be facilitated by highlighting relevant information with visual cues [12,13]. Furthermore, several eye-tracking measures have been described in the literature indicating the presence and strength of the abovementioned processes [14]. First, selection can be described as the process of accessing pieces of sensory information from each representation. By highlighting relevant information in a visual representation, such as the vector decomposition, it is easier for the learner to attend to that information. Enhanced selection processes are indicated by a shorter time to first fixate on the relevant information. Second, organization describes structuring the selected information to build a coherent internal representation, involving, for example, comparisons and classifications. Highlighting relevant information emphasizes the structural relations and the identification of links between different representations. In our example, the x and y directions are highlighted with framing boxes, thus they can easily be related to the partial derivatives of the algebraic equation. Deeper knowledge organization is characterized by longer visit duration and a higher number of fixations. Last, integration can be considered as combining internal representations with activated prior knowledge (long-term memory). In particular, learners need to integrate elements

010126-2

VISUAL CUES IMPROVE STUDENTS’ …

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

within a single representation or across multiple representations, for example, coordinating graphical representations and equations with text to create a mental model to solve a problem. Integration processes are quantified by transitions between relevant elements [15,16].
In the present example we used colored elements to refer to corresponding parts in the equation and the field plot. Taken together, visual cues can help the learner attend to and notice conceptual relevant information in the problem which they may not have considered otherwise. As Madsen et al. point out, “providing visual cues by no means guarantees that the learner will reach a correct solution and understanding of the problem” [13]. However, visual cues might activate prior knowledge from long-term memory for problem solving and they could at least facilitate learning by favoring deeper engagement with the instructional strategy and helping the learner avoid superficial reading over it.
B. Research questions
The main research question of this study was “What are the differences between students instructed with visual cues and students instructed without visual cues in interpreting the divergence and curl of vector field plots?”. More specifically, the aim of this study was to evaluate the effects on (a) problem-solving scores (interpretation of divergence), (b) transfer scores (interpretation of curl), (c) confidence scores, (d) perceived task difficulty (mental stress) and perceived
quality of instruction, and what information can eye-tracking data reveal about (e) processing of the instruction in terms of visual aspects
(e.g., visit durations on different parts), (f) problem-solving strategies.
Based on the theoretical framework above, we hypothesize that problem-solving abilities will be improved by the cues shown in Fig. 1 since they make implicit causal relations or implicit functional relations more explicit. So we expect that instructions are more successful with visual cues than without. In other words, students instructed with visual cues (VC students) will perform better in subsequent problem-solving tasks than students instructed with no cues (NC students). In addition to students’ response correctness, we also assess their response confidence level and other measures to evaluate the effectiveness of the visual cues. Confidence refers to one’s belief in one’s own ability [17] and a positive correlation between confidence and performance has been reported in several studies [17–20].

ranging from 18–27 (mean 20.6 years) took part in the experiment. All of them attended an introductory electromagnetism course (total enrollment: 51) and had successfully completed two mechanics courses before (calculus-based mechanics and experimental physics). The lectures have introduced the concepts of divergence and curl traditionally without emphasizing visual interpretation. Participation was voluntary, took about 30 min, and was compensated with 10 EUR. All students had normal or correct-to-normal vision.
B. Study design and procedure
The experimental design is shown in Fig. 2. The students first completed a pretest, including conceptual questions about divergence and curl, questions about vector decomposition, and a standardized test for spatial abilities [21]. Before the experiment started, the students were asked to judge the divergence and curl of a vector field plot (VF0), presented on a computer screen. The answer to the initial question contained information about students’ performance and eye movements before any instruction. Then the students were randomly assigned to the NC condition (N ¼ 16) or VC condition (N ¼ 16). In period 1, each group of subjects was given the same instruction about the interpretation of divergence either with (VC) or without (NC) visual cues, as shown in Fig. 1. The students then applied the strategy to 8 vector field plots

II. METHODS
A. Participants A total of 32 (26 male) major physics students from the Technische Universität Kaiserslautern (TUK) with ages

FIG. 2. Study design. The VC or NC condition defines the only contrast between both groups. (Perceived) quality of instruction (QI), (Perceived) mental stress (MS).

010126-3

KLEIN, VIIRI, and KUHN

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

TABLE I. Elements of the experimental procedure and measures or variables [eye tracking (ET)].

Element name

Behavioral measures

ET

1: Pretest and

conceptual understanding

no

covariates

(divergence and curl), vector

decomposition, spatial ability

2: Initial problem

performance

yes

3: Instruction

ÁÁÁ

yes

4: Problem solving

performance, confidence

yes

5: Questionnaires

perceived quality of instruction, no

perceived mental stress

6: Definition of curl

ÁÁÁ

yes

7: Transfer

performance, confidence

yes

which were presented one after another. They were not allowed to take notes or use any aids. Whenever a student was ready to give an answer, they pressed a button and gave their answer (divergence is zero or nonzero) and rated their response confidence on a scale. The students were unable to skip back to the instruction page or the previous questions. After completing period 1, questions about the perceived difficulty of the learning materials (mental stress) and about the quality of the instruction were posed.
In period 2, the students were presented the definition of the curl operator and they were told to transfer the visual strategy to the curl equation. The same 8 vector field plots were used again in the same sequence as before. There were no time limitations and the students did not receive any feedback at any point during the experiment. The experimental procedure consisted of 7 elements which are summarized in Table I.
C. Materials and measures
All study material is included as Supplemental Material [22]. During the study, several behavioral measures were taken which are summarized in Table I. The reliability of all behavioral measures was at least acceptable (Cronbach’s α ¼ 0.74–0.94). Eye movements were recorded with a Tobii X3-120 stationary eye-tracking system [23], operating with a sampling frequency of 120 Hz.
D. Data analysis methods
Between-group comparisons of the dependent variables are accomplished by analysis of variance (ANOVA). In addition, a multivariate analysis of variance (MANOVA) was performed. MANOVA is a generalized form of ANOVA but it includes all outcome variables together (instead of performing several ANOVAs) and uses the covariance between the outcome variables in testing the statistical significance of the mean differences [24]. Thus, the procedure accounts for intercorrelations between the variables, and also accounts for the error inflation caused by multiple testing. From raw gaze data, fixations and

FIG. 3. Instruction page (with visual cues) with AOIs.

saccades were classified with an I-VT algorithm [25]. From these data, fixation or visit durations and number of fixations were obtained. To analyze the eye-tracking data during instruction reading (element 3), the following procedure was applied:
(1) Define specific rectangular areas of interest (AOIs) covering nonoverlapping areas on the instruction page, that are the problem definition, the strategy, the diagram, and the hint (see Fig. 3).
(2) For every participant, the time course of the reading process was divided into 10 equal time intervals.
(3) Fixation count and transitions between AOIs were determined for every time interval and each AOI.
(4) Calculation of error bands and data smoothing. The procedure results in a time series of visual attention for each AOI or transitions between them, referred to as gaze likelihood analysis or transition likelihood analysis here. Quantitative analysis of this data structure requires a repeated measures analysis of variance (ANOVA-rm) with the time interval as within-subject variable and the condition as the between-subject variable for each AOI. To analyze the eye-tracking data during problem solving and transfer (elements 4 and 7), absolute saccadic angles between subsequent fixations were analyzed [6].
III. RESULTS
A. Pretest and initial problem (elements 1 and 2)
The physics pretest contained three groups of items about vector decomposition, divergence concept, and curl concept; see Ref. [22]. Although the VC students achieved

TABLE II. Students’ prior achievement (mean and standard deviation). Statistics refer to ANOVA (degrees of freedom ¼ 1, 30).

Variable

VC

NC

Fp

Pretest overall

0.88 (0.10) 0.83 (0.09) 1.80 0.20

Vector decomposition 0.96 (0.14) 0.95 (0.15) 0.03 0.86

Divergence concept 0.74 (0.19) 0.70 (0.11) 0.35 0.56

Curl concept

0.76 (0.13) 0.68 (0.11) 3.23 0.09

Spatial ability

0.73 (0.19) 0.73 (0.15) 0.04 0.84

010126-4

VISUAL CUES IMPROVE STUDENTS’ …

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

FIG. 4. Distributions of visual attention (left) and saccadic angles (polar plot, right) regarding the initial problem.
slightly better scores than NC students (see Table II), the differences were not statistically significant. It is notable that the absolute scores are rather high, indicating sufficient prior knowledge of all students to understand the subsequent instruction. Furthermore, both groups of students scored similarly on the spatial ability test (Table II).
The initial vector field was presented with 4 alternatives (fzero; nonzerog × fdivergence; curlg) and a no-answer option which was chosen by 1 student. The correct alternative (zero divergence and nonzero curl) was recognized by 7 of 32 (22%) students (3 in later NC condition). Most students (47%) answered that the vector field had nonzero divergence and zero curl, and 12.5% of students chose the nonzero curl and nonzero divergence option. Students looked at the field plot for 15 s on average (with standard deviation SD ¼ 13.8 s), including 41.2 fixations (SD ¼ 29.9). The visual attention of the students was located on the center of the plot and the saccadic directions are broadly distributed over the angles with small peaks in 0°, 45°, 180°, and 270° directions (Fig. 4).
B. Visual processing of the instruction page (element 3)
Average total visit duration of the instruction page was 106.8 seconds (range 55.7–242.2 s, SD ¼ 35.9 s), with no significant differences between the groups, p ¼ 0.17. There was no significant correlation between time spent with the instruction and any of the outcome measures. To analyze how students distributed their attention across the whole instruction-reading process, and whether the patterns of attention allocation differed between both groups, the gaze likelihood analysis was applied as described in Sec. II D. Figure 5 indicates that students read the instruction from the top to the bottom as it was intended: The

definition received the most attention during the first time intervals, the hint during the last time intervals, and the strategy and diagram received most attention in intermediate time intervals. The statistical results of four repeated measures ANOVAS (2 × 10) are summarized in Table III.
The analysis reveals significant within-subject effects with large effect sizes for all AOIs [Fð9; 270Þ > 4.3; p < 0.001; η2 > 0.12]. Furthermore, a between-subjects effect concerning the visual attention on the graphical representation exists [Fð1;30Þ ¼ 3.9;p < 0.05;η2 ¼ 0.11]. Planned contrasts and descriptive data analysis revealed that the graphical representation received more attention from students instructed with highlights. Moreover, interactions between the group and the time course was significant for the definition AOI, Fð9; 270Þ ¼ 2.20; p < 0.05; η2 ¼ 0.07. To break down this two-way interaction, the interaction between time interval and group was analyzed for each time step in 9 ANOVAs. Repeated measures revealed that visual attention on the definition developed differently for students instructed with and without highlights between the first and the second time interval. That is, differences were only found in the beginning of the time course.
Next, transitions between the AOIs were counted for each student by generating a character string from the scan paths, where each character refers to a fixation in a specific AOI. After removing repetitions between two characters, the transitions between the elements were counted. Table IV presents the average transitions between the four elements per student, separated along the diagonal by the group condition. The direction of the transition was not taken into account, e.g., transitions from the definition to the strategy and from the strategy to the definition were aggregated. Obviously, we expect at least 1 transition between consecutive elements. The number of transitions involving the hint AOI does not exceed this number significantly, hence they can be neglected.
VC students performed more transitions overall than NC students (p < 0.001), particularly considering transitions between the strategy and diagram. The transitional behavior between these two elements was deeper analyzed by extracting the transition likelihoods over time. Figure 6 shows that students in both groups performed transitions at the same points in time, but the amount of transitions was thoroughly more pronounced for VC students. A 10 × 2

FIG. 5. Students’ attention allocation on the instruction page across 10 equal time intervals, separated by AOIs.
010126-5

KLEIN, VIIRI, and KUHN

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

TABLE III. Attention measures on each AOI per group and results of four repeated measures ANOVAs (10 × 2) with “condition” as between-subjects factor and “time interval” (TI) as within subjects factor.

Definition Strategy Diagram Hint

Total visit duration (s)

VC

NC

32.6 (12.9) 43.8 (12.2) 25.8 (32.6) 14.9 (5.7)

32.5 (9.3) 38.7 (15.9)
7.8 (5.2) 16.3 (5.1)

Total fixation count

VC

NC

125 (46.8) 175 (49.1) 75.0 (83.4) 58.8 (21.0)

118 (28.9) 151 (56.9) 23.2 (16.4) 62.4 (13.8)

Time interval

Fð9; 270Þ η2

90.0ÃÃÃ 0.75

19.2ÃÃÃ 0.39

4.3ÃÃÃ

0.12

128.0ÃÃÃ 0.81

Condition

Fð1; 30Þ η2

0.1

ÁÁÁ

0.3

ÁÁÁ

3.9Ã

0.11

2.0

ÁÁÁ

TI × condition

Fð9; 270Þ η2

2.2Ã

0.07

1.0

ÁÁÁ

1.4

ÁÁÁ

1.5

ÁÁÁ

TABLE IV. Average number of transitions between individual elements of the instruction page. The upper (lower) diagonal refers to students instructed without (with) highlights.

Definition Strategy Diagram Hint

Definition AOI

ÁÁÁ

Strategy AOI

4.75

Diagram AOI

6.38

Hint AOI

0.31

2.88

5.75

0.31

ÁÁÁ

4.94

1.06

9.00

ÁÁÁ

1.00

1.75

1.69

ÁÁÁ

ANOVA-rm yielded a significant between group effect, Fð1; 30Þ ¼ 5.8, p ¼ 0.023, η2p ¼ 0.16, but no interaction effect (times × condition), Fð9; 279Þ ¼ 0.4, p ¼ 0.96.
C. Instruction-based problem solving (divergence) and behavioral measures (period 1, elements 4 and 5)
Table V presents the average values of students’ performance, confidence, mental stress, and perceived quality of instruction measures by group. All values have a possible range between 0 and 1, where 1 represents perfect performance, highest confidence, high mental stress, and high quality of instruction, respectively. The criteria to treat the data sets as sufficiently normally distributed were met. Table VI in the Appendix presents the correlations between the measures. VC students achieved a higher

problem-solving score and replied with higher confidence than NC students, resulting in significant between-group effects. Eight students in VC condition and 2 students in NC condition performed perfectly on all eight vector fields and 5 students scored 4 points or less (2 in VC condition). From previous studies, it is known that the polar distribution of saccadic angles of expertlike performers is highly symmetric with sharp peaks in the horizontal and the vertical directions and the distribution of the poor performers is broader [6]. Figure 7 in the Appendix shows the distributions on the test and on the item level.
Moreover, VC students experienced less mental stress (medium-sized effect) and judged the quality of instruction better. Since the outcome variables are intercorrelated (see Table VI in the Appendix), a multivariate analysis of variance was applied. There was a statistically significant main effect on the group factor (NC and VC students) for all response variables in the model, Fð6; 25Þ ¼ 3.02, p ¼ 0.023, η2p ¼ 0.42.
D. Transfer task: Interpretation of curl (period 2, elements 6 and 7)
After the completion of period 1, students were shown the definition of the curl operator next to a vector field plot without any visual cues and without instructional text. In average, a student in VC condition fixated on the equation for 19.2 s in total (SD ¼ 14.0), whereas NC students spent 13.8 s on it (SD ¼ 11.6). The difference is not significant, Fð1; 30Þ ¼ 1.40; p ¼ 0.25. Likewise, the difference in

FIG. 6. Average transitions between the strategy and the diagram per student over time by group condition.

TABLE V. Descriptive data on all outcome variables (means and standard errors in parentheses) and ANOVA results.

Visual cues No cues

p

d

Period 1

Problem solving

0.82 (0.03) 0.71 (0.04) 0.04 0.37

Confidence

0.74 (0.02) 0.53 (0.02) <0.001 1.28

Questionnaires

Mental stress

0.27 (0.02) 0.38 (0.03) <0.001 0.70

Quality of instruction 0.81 (0.02) 0.61 (0.03) <0.001 1.22

Period 2

Transfer

0.76 (0.04) 0.61 (0.04) 0.01 0.46

Confidence

0.73 (0.02) 0.56 (0.02) <0.001 0.97

010126-6

VISUAL CUES IMPROVE STUDENTS’ …

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

visual attention concerning the vector field plot was not significant, even though VC students spent in average about twice the time on it than NC students (17.6 Æ 32.5 s vs 8.1 Æ 5.2 s).
Students’ achievement in period 1 (divergence) and period 2 (curl) was highly correlated, r ¼ 0.69, p < 0.001. VC students outperformed NC students in terms of correct responses and higher confidences, see Table V. Seven students solved all eight items correctly (5 from the VC group) and 15 students answered no more than 4 items correctly (11 from the NC group). Again, the distribution of saccadic eye movements replicates previous findings, see Fig. 8 in the Appendix. In total, 54% of items were judged correctly concerning the presence of both divergence and curl. The VC students outperformed the NC students regarding this matched item score significantly (64% vs 44%, p < 0.001).
IV. DISCUSSION AND CONCLUSIONS
Despite students’ high conceptual understanding of curl and divergence as measured by the pretest and almost perfect performance on vector-decomposition tasks prior to the experiment (see Table II), most of them (78%) failed on the initial problem VF0 without instructional support, confirming the results of previous studies in this field [3,4,10]. The most often chosen incorrect answers included a nonzero divergence statement (60%), meaning that students incorrectly believed that the vector field had a source or a sink. The tendency of students to give a false positive answer of this kind has also been observed in a previous eye-tracking study and was explained by the students’ carelessness about the distinction between the direction of the arrows and the direction of their change [6]. This error likely arises from confusion between a primary quantity (the arrow in this case) and its derivative (the spatial change of the arrow). This has also been documented in another context [26]. The eyetracking data from the initial problem shed light on student’s visual attention when they were confronted with the coordination task for the first time (Fig. 4). Students paid most attention to the center of the plot, i.e., the region where the direction of the arrows is changing. The estimation of the field arrows is difficult in the center of the plot since the arrows are rather small. Thus, students’ visual attention was guided by superficial characteristics of the representation rather than by conceptual thoughts and rigorous strategies. Additionally, most of the saccades followed the x direction, i.e., the direction of change. Taken together, the eye-tracking data support the explanation about students’ confusion given above.
After instruction with or without visual cues, students’ accuracy in judging the divergence and curl of vector field plots increased significantly compared to the initial problem. Hence, students successfully learned about the coordination between multiple representations with the instruction before they used them for problem solving.

Matched responses from both experimental periods yielded 54% accuracy, and single responses yielded 77% (divergence) and 69% accuracy (curl), respectively. These results underline the general need for instructional support concerning coordination of multiple representations in abstract concepts. Even though there is growing evidence that targeted instruction can improve the coordination of multiple representation [27,28], this competence has rarely been explicitly taught to students and tested as an intervention [29], so this study contributes to this line of research. The acquisition of this competence is costly because it is cognitively demanding and requires instructional support [2].
The instruction with visual cues included important information about vector decomposition and spatial information about partial derivatives—concepts that are crucial in the subject context. Even though the experimental condition included only this minor change (adding visual cues and emphasizing the text-graph connection), the overall effect on learning was immense. The betweengroup analysis provided in Table V showed that visual cues enhanced students visual understanding of divergence, that is, they were more likely to connect the equation to the field plot correctly and respond with higher confidence. The significant between-subject effects concerning the perception of task difficulty (mental stress) and instruction quality indicate that the visual hints, while increasing correct answers and response confidence on the divergence problem, furthermore facilitated students’ learning. Since the instruction and hints were given in an explicit manner, one could argue that students were just good at following the steps provided by text and they made no meaning out of the strategy as they were solving the problems. The positive results on the transfer tasks (curl concept) without instructional support provide reasons against this view and support the conclusion that both groups grew in their conceptual understanding of the fields. Again, the group that was instructed with visual cues before also performed better on the transfer tasks and responded with higher confidence. Hence, we conclude that visual cues indeed made students’ meaning for vector fields deeper. The results are in line with design principles of multimedia learning, that is, providing graphical representations with visual cues improves learning [30,31]. Positive correlations between response accuracy and confidence have been reported in several studies [17–20], and the data from this study support these findings in the context of instruction-based learning.
The analysis of eye-tracking data during the reading process yielded that VC students spent more time on the vector field plot (diagram AOI) than NC students. There were no significant differences concerning visual attention on other parts of the instruction page that can be considered as a successful manipulation check, since only changes to the field plot were made. Furthermore, the time series of visual attention on the definition AOI, the strategy AOI, and

010126-7

KLEIN, VIIRI, and KUHN

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

the hint AOI look very similar (no meaningful interaction effects), indicating similar reading speeds of students in both conditions. While it is entirely intuitive that visual cues generate more attention on the visual representations, it was also found that VC students put in more effort to integrate the information from the diagram with information from the equation and the text. Extending previous eye-tracking analysis methods, we applied the transition likelihood analysis, making the occurrence of transitions visible over time. The results show that integration processes appeared in both groups at the same point in time but they were more pronounced for the students in the VC condition.
The eye-movement data during problem solving were analyzed using the saccadic direction analysis. We replicated a previous finding [6] that the best-performing students concentrated on horizontal and vertical eye movements and ignored oblique saccades. This holds for both experimental periods (divergence and curl). By comparing this result to the saccadic direction distributions related to the initial problem, we find that the instruction helped students develop a rigorous procedure for interpreting and determining divergence and curl rather than thinking about the common meaning of the words or causing other imperfect associations [3,4,10].
As this study shows, making correct connections between mathematical concepts and graphical representations of vector fields is difficult for students, especially without instructional support. Given the importance of representational competence in physics learning and problem solving, we advocate incorporating qualitative exercises like those presented here into lectures or tutorials. The correct qualitative interpretation of the partial vector derivative is encouraged by using various examples and incorporation of visual cues, stressing the focus on horizontal and vertical directions. Once a complete visual understanding of the differential definitions is achieved, discussions of the local property of the operators can follow.

FIG. 7. Polar distributions of saccadic angles for experts and novice students including data from all divergence items (top), and including correct or incorrect responses to VF4 (bottom).

APPENDIX: ADDITIONAL DATA
The correlation matrix is shown in Table VI. The polar distributions of saccadic angles are shown in Figs. 7 and 8.

FIG. 8. Polar distributions of saccadic angles for experts and novice students including data from all curl items.

TABLE VI. Pearson correlation coefficients. Only significant correlations (p < 0.05) are shown.

12

3

4

5

6

1: Performance score

Á Á Á 0.57 not significant 0.47

0.69

0.44

(problem solving, divergence)

2: Confidence score (problem solving) Á Á Á Á Á Á

−0.71

0.54

0.46

0.68

3: Mental stress

ÁÁÁ ÁÁÁ

ÁÁÁ

−0.65 not significant

−0.47

4: Quality of instruction

ÁÁÁ ÁÁÁ

ÁÁÁ

Á Á Á not significant not significant

5: Performance score (transfer, curl) Á Á Á Á Á Á

ÁÁÁ

ÁÁÁ

ÁÁÁ

0.41

6: Confidence score (transfer)

ÁÁÁ ÁÁÁ

ÁÁÁ

ÁÁÁ

ÁÁÁ

ÁÁÁ

010126-8

VISUAL CUES IMPROVE STUDENTS’ …

PHYS. REV. PHYS. EDUC. RES. 15, 010126 (2019)

[1] D. E. Meltzer, Relation between students’ problem-solving performance and representational format, Am. J. Phys. 73, 463 (2005).
[2] M. A. Rau, Conditions for the effectiveness of multiple visual representations in enhancing stem learning, Educ. Psychol. Rev. 29, 717 (2016).
[3] C. Singh and A. Maries, Core graduate courses: A missed learning opportunity?, AIP Conf. Proc. 1513, 382 (2013).
[4] C. R. Baily, L. Bollen, A. Pattie, P. van Kampen, and M. De Cock, Student thinking about the divergence and curl in mathematics and physics contexts, Proceedings of the 2015 Physics Education Research Conference, College Park, MD (AIP, New York, 2015), pp. 51–54.
[5] L. Bollen, P. Van Kampen, C. Baily, and M. De Cock, Qualitative investigation into students’ use of divergence and curl in electromagnetism, Phys. Rev. Phys. Educ. Res. 12, 020134 (2016).
[6] P. Klein, J. Viiri, S. Mozaffari, A. Dengel, and J. Kuhn, Instruction-based clinical eye-tracking study on the visual interpretation of divergence: How do students look at vector field plots?, Phys. Rev. Phys. Educ. Res. 14, 010116 (2018).
[7] M. De Cock, Representation use and strategy choice in physics problem solving, Phys. Rev. ST Phys. Educ. Res. 8 (2012).
[8] S. E. Ainsworth, The functions of multiple representations, Comput. Educ. 33, 131 (1999).
[9] Integral representations of divergence are discussed in Ref. [6].
[10] R. E. Pepper, S. V. Chasteen, S. J. Pollock, and K. K. Perkins, Observations on student difficulties with mathematics in upper-division electricity and magnetism, Phys. Rev. ST Phys. Educ. Res. 8, 010111 (2012).
[11] R. E. Mayer, Multimedia Learning (Cambridge University Press, Cambridge, England, 2001).
[12] B. B. de Koning, H. K. Tabbers, R. M. J. P. Rikers, and F. Paas, Towards a framework for attention cueing in instructional animations: Guidelines for research and design, Educ. Psychol. 21, 113 (2009).
[13] A. Madsen, A. Rouinfar, A. M. Larson, L. C. Loschky, and N. S. Rebello, Can short duration visual cues influence students’ reasoning and eye movements in physics problems?, Phys. Rev. ST Phys. Educ. Res. 9, 020104 (2013).
[14] A. Schüler, Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for textpicture integration, Learn. Instr. 49, 218 (2017).
[15] A. Schüler, Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for textpicture integration, Learn. Instr. 49, 218 (2017).
[16] N. Ott, R. Brünken, M. Vogel, and S. Malone, Multiple symbolic representations: The combination of formula and text supports problem solving in the mathematical field of propositional logic, Learn. Instr. 58, 88 (2018).
[17] B. A. Lindsey and M. L. Nagel, Do students know what they know? Exploring the accuracy of students’

self-assessments, Phys. Rev. ST Phys. Educ. Res. 11, 020103 (2015). [18] P. Klein, A. Müller, and J. Kuhn, Assessment of representational competence in kinematics, Phys. Rev. Phys. Educ. Res. 13, 010132 (2017). [19] J. S. Aslanides and C. M. Savage, Relativity concept inventory: Development, analysis, and results, Phys. Rev. ST Phys. Educ. Res. 9, 010118 (2013). [20] P. Bell and D. Volckmann, Knowledge surveys in General Chemistry: Confidence, overconfidence, and performance, J. Chem. Educ. 88, 1469 (2011). [21] Since the tasks in this study involve manipulating spatial representations in the form of graphs, students’ spatial abilities (that is, the ability to perform spatial transformations of mental images or their parts) were considered as an important covariate in our study design. The test is described at K. Heller and C. Perleth, Kognitiver Fhigkeitstest für 4. bis 12. Klassen, Revision (KFT 4-12+R) [Cognitive Abilities Test for Grades 4 to 12, revision] (Hogrefe, Göttingen, Germany 2000). [22] See Supplemental Material at http://link.aps.org/ supplemental/10.1103/PhysRevPhysEducRes.15.010126 for a full evaluation of the study quality. [23] More specifications can be found on the product website https://www.tobiipro.com. [24] C. J. Huberty and S. Olejnik, Applied MANOVA and Discriminant Analysis (John Wiley & Sons, Hoboken, New Jersey, 2006). [25] D. D. Salvucci and J. H. Goldberg, Identifying fixations and saccades in eye-tracking protocols, Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (ACM, New York, 2000), pp. 71–78. [26] L. C. McDermott, M. Rosenquist, and E. van Zee, Student difficulties in connecting graphs and physics: Examples from kinematics, Am. J. Phys. 55, 503 (1987). [27] B. W. Bergey, G. C. Jennifer, and N. S. Newcombe, Teaching high school biology students to coordinate text and diagrams: Relations with transfer, effort, and spatial skill, Int. J. Sci. Educ. 37, 2476 (2015). [28] C. Leopold and D. Leutner, Science text comprehension: Drawing, main idea selection, and summarizing as learning strategies, Learn. Instr. 22, 16 (2012). [29] W. Zahner, T. Dai, J. G. Cromley, T. W. Wills, J. L. Booth, T. F. Shipley, and W. Stepnowski, Coordinating multiple representations of polynomials: What do patterns in students’ solution strategies reveal?, Learn. Instr. 49, 131 (2017). [30] P. A. O’Keefe, S. M. Letourneau, B. D. Homer, R. N. Schwartz, and J. L. Plass, Learning from multiple representations: An examination of fixation patterns in a science simulation, Comp. Human Behav. 35, 234 (2014). [31] E. Ozcelik, I. Arslan-Ari, and K. Cagiltay, Why does signaling enhance multimedia learning? Evidence from eye movements, Comp. Human Behav. 26, 110 (2010).

010126-9