There are several reasons why eye movement researchers are interested in quantifying the effect of task difficulty on scanpaths. First, this allows us to gauge the suitability of educational software; if the implementation is too hard, what kinds of eye movement sequences are associated? Second, the task can be difficult for different reasons, some higher-level (cognitive) some lower (visual); understanding the relationship between these allows us disambiguate top-down from bottom-up influences on the oculomotor system (e.g. Foulsham, Barton, Kingstone, Dewhurst , and Underwood, 2011). Until recently however, studying the effect of task difficulty on eye movements has been limited either to unitary eye movement events (fixations and saccades in isolation (Wertheim, Hooge, Krikke, and Johnson, 2006)), or coarse averaging across many eye movement types (with attention maps (Grindinger, Duchowski, and Sawyer, 2010), and transition matrices (Holmqvist, Holsanova, Barthelson, and Lundqvist, 2003)). Here we present new data showing the effect of task difficulty on scanpaths as measured by our multidimensional scanpath similarity approach (Jarodzka, Nyström, and Holmqvist, 2010). This has the advantage of capturing sequence information, whilst retaining fundamental eye movement parameters like fixation position, duration, and saccadic amplitude. It also controls for scanpath shape, which can differ depending on the task at hand (e.g. Johansson, Holsanova, and Holmqvist, 2011).
Three experiments evaluated scanpath similarity with respect to task difficulty in different ways. In each experiment participants were presented with the numbers 1-5 and their task was to saccade to each number in order. In Experiment 1 the numbers were of different size per trial, according to five levels of task difficulty (small to large). In Experiment 2 the numbers were presented along with a varying number of distractors, giving five set sizes (1-6: 1-10), easy to difficult. Experiment 3 introduced noise by degrading the background in five steps relative to the numbers themselves, making them harder to identify in peripheral vision. Task difficulty was manipulated under these conditions according to the hypothesis that when the numbers are less conspicuous, participants will produce more divergent scanpaths between individuals. This was assessed both with our multidimensional scanpath similarity metric, and with the most advanced alternative: ScanMatch (Cristino, Mathot, Theeuwes, and Gilchrist, 2010).
Results revealed that scanpaths do indeed become less similar as the task becomes harder, but critically this depends on the task. Smaller numbers are harder to locate, but the larger margin of spatial error for fixating bigger numbers leads to scanpath variability. Conversely, larger set sizes are more difficult, with decreasing similarity in terms of position and shape as task difficulty increases. However, the oculomotor system compensates with a greater number of shorter fixations, actually improving performance accuracy. Only when the task was made harder with increasing background noise, were both the ocolomotor and scanpath comparison measures consistent, both for our multidimensional method and for ScanMatch.
These results indicate the importance of task and stimulus features when considering scanpath similarity, as well as the subtle interplay between basic eye movement events and the kinds of similarity they produce.