7.1.3　General patterns of the eye movement data

As mentioned in Section 7.1.1，the pictures and verbal texts have to be demarcated as different Areas of Interest（AOIs）before the eye movement data on them can be computed and analyzed.The results will be divided into the quantitative global eye movement data and the reading path which is based on qualitative observation.Three indicators of visual attention are employed in this research：gaze time，fixation counts and average fixation duration.Their definitions are given in the following.

Gaze time[Unit ms]—The total time of all fixations in the respective AOIs.

Fixation count[Unit 1]—Number of fixations in the respective AOIs.

Average fixation duration[Unit ms]—The average length of all fixations during all recordings on the respective AOIs.

7.1.3.1　Global eye movement data

On the picture portions of the RA pages，the looking behavior was characterized by fewer fixation counts and long saccades.Actually，in many cases，the participants made a quick scan across the whole picture with few fixation on it（see the gaze plot in Figure 7.4）.On the verbal portions of the pages，however，the looking behavior was characterized by dense fixation points，small saccade sizes，and longer gaze time.Figure 7.4 is an example of the different eye movement patterns of pictorial and verbal portions of a page.As it shows，the verbal elements are located on two locations of the page—the top and the bottom，whereas the picture occupies large area in the centre.The two verbal locations have much denser fixations and shorter saccades whereas the picture has only two fixations and long saccades.Actually the participant scanned quickly across the picture twice without fixating on it.

Figure 7.4　Viewing patterns in picture vs.verbal text

This intuitive observation was verified by the global eye movement data on different parts of the materials.Table 7.1 presents descriptive statistics of the ad elements and three indices of visual attention.It was found that participants'gaze time（909 ms）and average fixation durations（206 ms）on the picture were a little shorter than on the verbal portion.In addition，their fixation count（4.1）was also fewer than that of the verbal parts（6.4）.Their saccade sizes were longer in the picture than in the verbal parts.

It means that the total gaze time participants spend on looking at verbal slogans is，on average，the longest，followed by picture，and brand receives relatively least looking time.This pattern of gaze time accords with the findings in Pieters and Wedel（2004）that“it is typical when consumers explore regular magazines at their own pace but lower than when advertisements are tested in laboratory conditions”.Therefore it suggests that the current experiment is successful in eliciting natural eye movement data from the participants.

Table 7.1　Global eye movement data on different elements of Reader Attraction

However，it is not wise to infer that the verbal slogan text has superiority in attracting attention compared to picture because the picture and verbal text has very different characteristics in information representation and consequently sharp divergence in the viewing behavior.

First of all，with regard to the ways the two semiotic modes represent information，it is apparent and also acknowledged that verbal text is usually much more heavily packed in the visual field than picture（see for example，Rayner et al，2001）.Thus，more attention per unit surface is required for verbal text than for scene/picture perception（see for example，Pieters and Wedel，2004）.This difference can be evidenced by the facts that saccades are considerably longer on average for the picture than for verbal text.

Secondly，“pictures are often perceptually more distinct than words，which draws bottom-up attention”，summarize Pieters and Wedel（2004）on the basis of a number of empirical studies.The perception of pictures“relies more on peripheral and preattentive processes that are automatic，parallel，fast，and less effortful.”In contrast，the perception of verbal text“relies more on focal attentive processes，which are voluntary，serial，slow，and effortful.Thus，although the gist of a scene can often be comprehended in a few glances，text requires more eye fixations to be comprehended.”

Thirdly，the average looking time per unit needed for a picture and a verbal text differs greatly.It has been acknowledged in many researches（see for example，Rayner et al，2009）and summarized in the Tobii Eye Tracking Whitepaper that，most people need between 50—60 ms of seeing a word in order to perceive it.In reading，the perceptual span is roughly two to three words per fixation（Jay，2004：248）.Although these results are obtained from eye-tracking experiments which were conducted with alphabetical languages like English，studies on logographic languages such as Japanese and Chinese have found patterns that support my argument even more.For example，according to Jay（2004），“fixation durations tend to be longer for readers of Japanese and Chinese than for readers of English”and“the average fixation duration for Chinese readers is about 300 ms”（Jay，2004：249）.Therefore，the mean value of average fixation duration for verbal slogans（212 ms）in the current research shows that the verbal slogans are read very fast.On the other hand，when looking at a picture，people need to see it for about 150 ms to interpret it.Actually，the time for the scene/picture perception might be shorter：the gist can be understood from a 40-to 50-ms scene exposure（see for example，Rayner et al，2009）.Many studies have shown that objects can be encoded from very brief presentations（～50 ms），even when the object appears in a scene（see for example，Rayner et al，2009）.The gaze time（909 ms）and average fixation duration（206 ms）for the picture in the current study show that the participants looked at the pictures slowly and carefully.

In a word，we can summarize the above data as：（i）pictures in all received longer relative looking time in the current experiment than verbal slogans and than in normal conditions.（ii）The average fixation duration in the picture area is longer than both that in normal conditions and that of verbal slogans and headlines，which indicates that pictures were viewed very carefully.

This finding is consistent with that in other eye-tracking experiments on advertisements.For example，Rayner et al（2001）have found similar patterns，that is，participants are more attracted by pictures although the overall looking time on the verbal texts is longer than that on the picture.Their explanations are also similar：viewers can encode much more information per fixation from the pictorial information than from the textual information.Pieters and Wedel（2004）verify that，compared to brand and verbal text，the pictorial has superiority and an intrinsic tendency to capture a substantial amount of attention，independent of its surface size and of person factors including product involvement，motivation and brand familiarity.The finding in the current study further supports this argument and verifies that the pictures are better at attracting attention than verbal information in the English learning advertisements.

7.1.3.2　Reading path

Reading path，or scan path in the terminology of cognitive psychology，is“the pattern of locations that saccades visit”（Richardson et al，2007）.As widely acknowledged by eye-tracking researchers（see for example，Rayner et al，2001;Burmester and Mast，2010），the reading path or scan path is difficult to quantify.Moreover，the results of reading path from the existing studies in multimodal discourses such as newspapers are often different，or even contradictory（see for example，Holsanova et al，2006）.

As for the reading path on advertisements，eye-tracking researches also reveal highly variable results which will be elaborated later.The current study shows that there are some very striking characteristics of how viewers scan the pages.The initial fixation on the page was almost always located in the top left part because that was where the participants were intended to start by a red dot at the top left part of the screen.Looking behavior varied from individual to individual，but there did exist some general patterns and fairly consistent features that were shared by the majority of participants.Two important findings are elaborated below：

Finding 1：Twice-reading

It's almost a universal phenomenon for viewers to view the page twice.For the first time they tend to look at the page in the directionality from top to bottom and from left to right.When they finish this first looking，they will have a quick review backwards.That is，their eye gaze will move quickly back across the sections they saw earlier，either without or with a small number of fixations.The gaze plot in Figure 7.5 is a typical example for the twice-reading.We can see that the participant started from the picture（Fixation 452）and went to the slogan（Fixation 453 to Fixation 456）and then to the logo at the bottom（Fixation 457 to Fixation 459）.After the first reading，she went back from the bottom of page upwards，first to the verbal slogan（Fixation 460 and 461）and then to the picture（Fixation 462）.

Figure 7.5　An example of twice reading

Finding 2：Effects of spatial arrangement

The findings on the verbal-vs.-pictorial reading path of the current study will be reported with reference to Rayner et al（2001）which has a similar purpose.They found that viewers“typically initially made an eye movement to the large print，regardless of its spatial position within the ad...After looking at the large print，participants either made a very cursory scan of the picture，or more typically，they moved from the large print to the small print then to the picture.”

The current study found that this finding is true to some extent of the materials under study，but some finer and important deviations were found and need to be reported.Specifically，apart from the superiority of large print reported in Rayner et al（2001），the spatial position of the semiotic units interferes with the looking behavior to a large extent.

When the large print is on the upper or central part of the page，viewers tend to look at it in the first few fixations，although some participants may give a very cursory scan of the picture before or during the reading of the large print（see Figure 7.6 for example）.In this case，the findings are in accordance with that of Rayner et al（2001）.

However，if the large print（verbal）is on the lower part of the page and the picture（and logo）is on the upper part，viewers tend to look at the picture（and logo）first before looking at the large print.The following Figure 7.7 is a typical example.It shows a series of three gaze plot of a participant at three time phases.Figure 7.7：1 shows the first time phase，during which the participant looked at the logo first（Fixation 400 to Fixation 405）and then the picture（Fixation 406 to Fixation 410）.During the second time phase（Figure 7.7：2），she went to look at the large print below the picture（Fixation 411 to Fixation 422）.Until this time，the participants did not fixate on the large print at all.The last gaze plot（Figure 7.7：3）shows the review phase in which she moved backwards up to the picture（Fixation 423 to Fixation 429）.

Figure 7.6　Example 1 of reading path

（Phase 1：time segment：only include fixations inside interval[150424，160244]ms.Phase 2：time segment：only include fixations inside interval[150424，163598]ms.Phase 3：time segment：only include fixations inside interval[150424，166233]ms.）

The differences of reading path between the current experiment and Rayner et al（2001）may be at least partially attributed to the technical instructions imposed on participants.In Rayner et al（2001），the participants were technically set to start from the centre of the ad，therefore，they said，

Because the first fixation was alwaysin the centreof the ad（where there typically was not any text），participants were obviously moving to the text very quickly...Most of the time，the first fixation on text was on large print...（Rayner et al，2001：223）（Underlining added by the current author）

Figure 7.7　Three phases of a reading path

Since they were comparing the eye movement in verbal against the pictorial portions of advertisements and thus divided ad page into these two types，we had adequate reason to infer that the centre of the ad“where there typically was not any text”should be the picture.Then in their experiment the first fixation was always in the picture part of the ad，and then moved to the large print verbal text.The technical details of how this starting point instruction had been realized was not provided in Rayner et al（2001）.But from the wording like“always”in the quoted text，it might be speculated that it was done in a compulsive way by putting a trigger zone/point before each experiment advertisement was shown on the screen.Trigger zone/point is a zone/point by which participants can start a program through fixating on it.

In the current experiment，the participants were visually led to start from the top left of page because it is acknowledged as the starting location of typical reading path in natural reading behavior.Technically it was done by showing a blank screen with a red dot at the top left part for 1 second before each of the experiment material pages.Participants were told before the experiment that they would see a red dot flash before each material page appeared.Therefore，the technical setting of the starting fixations were solicited rather than prescribed or pre-determined.Actually the location of first fixation for different pages by different participants varied greatly although the technical solicitation was generally effective.

The essential factor underlying these details of technical setting is the principle of human visual attention.As the Tobii Eye Tracking Whitepaper states，whenever we look at things，there are two kinds of visual attention involved.One is overt attention which is commonly done by moving our eyes from one place of the visual field to another;the other is covert attention which is done in the peripheral areas of our visual field without eye movements.Change of the overt attention is conscious whereas that in the covert attention is unconscious.Although we can use these two mechanisms separately，they very frequently occur together.Usually when we see something we first use our covert attention in our peripheral vision to roughly identify what it is and detect interesting points，then we direct our gaze to that location allowing our brain to access more detailed information.As Richardson，Dale and Spivey（2007）summarize，visual attention is not always coincident with eye position although it is highly likely that spatial attention and saccade planning are closely coupled during natural unconstrained eye movement.They find on the basis of many eye-tracking researches that“planning a saccade toward a location improves processing at that location，regardless of whether or not the saccade is launched”（Richardson et al，2007：325）.

According to this principle，if the participants start from the picture，they can see it and do a rough processing of it even when they do not fixate on it.Therefore，in Rayner et al's experiment（2001），the participants had already seen the picture before moving their eye gaze to the large print.In this sense，their claim that the participants first looked at the large print and then the small print and finally the picture needs to be read with caution.In other words，we might reinterpret their finding as the following：the participants first looked at the picture briefly then moved on to the large print and then small print，at the end of looking at all important information areas on the ad page，they moved back to the picture.If this reinterpretation is right，then the reading path concluded in their experiment will be consistent with Finding 1（twice reading）in the current experiment.

It is for this reason that the novice researchers on eye-tracking are advised to set the starting point at the top left instead of the central area.In this situation，the viewer starts processing the page in the visual directionality of top-to-bottom and left-to-right which complies with the natural reading habits of most people.Another advantage of this practice is that viewers will not see or process any part of the page before the overt reading path begins.Thus it can avoid the problem of starting from the central area.

To summarize the global eye movement data and reading path analysis，pictures have superiority in capturing attention.They receive longer relative looking time and longer average fixation duration than verbal slogans.The analysis of reading path shows that pictures are more likely to receive initial eye gaze and the universality of participants'final return to the picture before the viewing process terminates also confirms the attractiveness of pictures to visual attention.