6.1 A working definition of surface cohesion in multimodal discourses
The notion of cohesion is defined in the Routledge Companion to Semiotics and Linguistics as“the formal elements and principles which make a collection of sentences into a text”(Cobley,2001:173).When applied to the multimodal discourses such as the English learning advertisements in this book,this definition needs to be modified into“the typographical features in the layout which make a collection of semiotic elements,including verbal and visual,into a coherent text/discourse”.Therefore,this chapter will focus on the typographical devices in the layout of the multimodal English learning advertisements that help signal the meaning relations onto the page as a twodimensional space.
The reason for the proposal of the definition lies in the substantial effects of typographical features on the organization of discourse which will be discussed in the following.
The close relationship between the layout and coherence construction in multimodal discourses has been well documented.Specifically,the typographical features in the layout affect the establishment of coherence in the following ways:
Firstly,typographical features in the layout can structure visual space,create balance,shape textual order,and thus provide coherence,cohesion,and meaning at the document level in a similar way to the structure provided by content and punctuation at the word level.By way of the structuring function,they guide readers'attention by providing a page-map to navigate(Gregory,2003;Stöckl,2005;Natasha,1999).
Secondly,typographical features in the layout may support,reinforce,reinterpret or contradict verbally construed messages;at any rate,these formal and semantic interrelations are intended and aim to create a holistic entity(Gregory,2003;Stöckl,2005).
Thirdly,these devices in layout also work on the reader's cognitive mechanisms during comprehension.They can highlight particular elements of a leaflet,focus readers'attention,break the text into visually coherent parts,and label the text,so that readers can quickly locate the information they need(Gregory,2003).
In summary,the typographical features in the layout can,to a large extent,contribute to the creation and reinforcement of coherence in multimodal discourses.This function is similar to that of cohesive devices for coherence in purely verbal texts.Therefore the definition of surface cohesion in multimodal discourses is reasonable.It reveals that the analysis of typographical features in the layout as cohesive devices will be very important for us to understand how coherence is constructed at the surface level in multimodal discourses.
Before embarking on the analysis,we need to make clear the relation of this new form of cohesion with coherence through clarifying the place of layout in the descriptive system of multimodal discourses.Compared with content,layout is the realization level of the multimodal discourses and the surface level of the semiosis in document design.Its relationship with the content and macro-structure are double-folded:
On the one hand,it has the function of realizing the meanings intended and manipulated in previous phases of design.By assembling different verbal and visual signs on the same page,layout brings“the process of semiosis...to a temporary standstill in textual form”(Kress,2000:152).It is the textual form by which the process of semiosis materializes itself.In this sense,the way it turns out is subject to the influences from the underlying levels of advertising design,including the communicative purpose,the intended interpretation the signs wish to invoke in the receiver,the specific semantic content it chooses in the signification and the relations inside these content elements.By a“spatial display”of visuals and verbal chunks,layout may“lend itself with greater facility to the representation of elements and their relation to each other”(Kress,2000:146).
On the other hand,as an independent level of semiosis,layout design has its own principles of operation,is subject to certain constraints due to its specific materiality,and works in its special ways to give force to the meanings contained in the more underlying levels.It may also counter-act on,or even to some extent,manipulate,the meanings conveyed in the visual and verbal elements and their relations,and through which,adjust the way the argument is achieved by the advertisements.The first aspect is acknowledged by common sense.It is thought of as the act of performing one's idea onto a kind of affordance,such as paper,video and hypertext.The second aspect,though not widely conscious to people,has begun to draw attention from professionals and researchers(see for example,Peled-Elhanan,2009).Along this line,we hold in this book that these resources in the layout of the multimodal discourses can serve to help mold the way the meanings are presented in an organized and coherent whole.
As Mancini and Scott(2006)hold,there is a fundamental semiotic difference between visual configurations and textual expressions:since it is a symbolic code,verbal text can express relational concepts with degrees of precision and subtlety that are not easily available in the visual medium.In other words,unlike textual representations,visual representations tend to be regulated by conventions that are less strict and more dependent on the context of use.Thus the way the relations among semiotic elements are signaled and cued will be expected to be different in multimodal discourses.However,as a meaning-making system used in communication,the visual has some conventions and principles that are shared with language.Therefore it is expected that the operation of the cohesive devices linking the visual and verbal elements in the layout of multimodal discourses will be,on the one hand,different from those in purely verbal texts,and on the other hand,share some common basic principles with them.What the specific criteria and components of the cohesive devices in multimodal discourses are will be discussed in the next section.