Today, learning outcomes in schools and universities are mainly measured with written exams, where calculations and answers to a number of problems are evaluated and graded. You can at best get a glance of how the students have identified the problem, how the problems have been represented and the methods of solutions. However, too often it is impossible from the written information to obtain any detailed knowledge about the processes the students went through leading up to their answers.
Several researchers have used eye tracking to trace the problem solving process in mathematics and related disciplines on a level corresponding to high school education. However, it seems to be a lack of studies in higher education, in particular within the mathematical domain. In this study, we investigate how students divide their attention across text, equations, and graphical illustrations of problems in vector calculus. Since vector calculus is a subject where there is a major need for figures explaining the physical interpretation of mathematical formulas, we are in particular interested in the dynamics between obtaining information from text, mathematical formulas, and figures.
We collected eye movements and speech from 36 second year students from the engineering physics program two weeks into a vector calculus course. The students solved eight problem related to vector calculus presented with text and equations (N = 16) or text, equations, and a graphical illustration of the problem (N = 20). They were asked to 'think aloud' and verbalize their thoughts while solving the problems. The experiment was self-paced, with the restriction that each problem had a maximum allowed presentation time of two minutes. After each question, the students were asked to answer a true or false statement about the problem, and to state how confident they were about their answer.
Overall, we found no evidence that illustrations increased the number of correct answers. However, there were large inter-problem differences; the illustrations significantly increased the number of correct answers in two of the problems and decreased the number of correct answers in one problem. The illustrations also did not seem to change the relative amount of total dwell time on the text and equations compared to the non-illustrated problems. Instead, the time when the illustrations were inspected was taken equally much from other parts of the problem.
These results suggest that care should be taken when designing an illustration to a problem in vector calculus, which, instead of helping the students, may have a detrimental effect on learning.