1
多模态语篇的连贯构建研究 以中国英语学习广告为例 英文版
1.4.2.2 1.2.2 Discourse coherence in multimodal sense
1.2.2 Discourse coherence in multimodal sense

The concern of the existing literature on coherence lies almost exclusively in purely linguistic texts.They study,with various perspectives and approaches,how the sentences in linguistic texts are connected into a unified,coherent and cohesive whole.However,since multimodal discourses like the English learning advertisements involve not only linguistic signs,such as clauses and sentences,but also signs of other semiotic modes—pictorial,graphical and typographical,the notion of discourse coherence should be extended from the purely linguistic sense to a broader multimodal sense.Specifically,the discourse coherence in the current book refers not to the connectivity between linguistic units like sentences in the English learning advertisements,but to the connectivity between elements of different semiotic systems,especially the verbal and the visual.

In these advertisements where multiple semiotic modes work together,“the logics of linear progression and causal or temporal contiguity such as associated with language and linguistic expression are seceding in part to more disparate,non-linear,nonhierarchical,more freely recombinative,circular and serialized kinds of representation”.(Eco,1990:83,cited in Iedema,2003)Therefore,the analysis of coherence construction in them requires a more compatible perspective that is different from the traditional language-oriented one.Specifically,the analytical framework of these advertisements should be devised in such a way that both verbal and visual mode of representation can be included properly,and most importantly,the means and strategies that connect them into a coherent multimodal discourse can be explored and described.

This attempt is fascinating because on the one hand,the verbal and visual are totally different ways of representation,and on the other hand,the interaction between them is,although not explicit,traceable and decipherable.The first reason makes the attempt challenging whereas the second makes it possible to turn out a rewarding endeavor.More specifically,as far as the first reason is concerned,each semiotic has its own specific(systemic)constraints and(material)affordances.The visual semiotic is generally continuous,sensorial and spatial,while language is abstract,discontinuous,linear(temporal)and governed by strict syntactic conventions(see for example,Saint-Martin,1995,cited in Iedema,2003;Kress and Van Leeuwen,1996:37).To find a compatible way to analyze both modes and their interaction is,by no means,an easy task.It requires a comprehensive knowledge of the signification and meaning-making in both of the two semiotic modes,a multi-semiotic perspective and the ability to traverse and exploit the difference as well as commonalities between visual and verbal.Due to the difficulties,it is widely admitted that the research on how units of different semiotic systems in multimodal texts are cohered is immature yet(see for example Lillis and McKinney,2003:98;Mancini&Scott,2006;Jewitt,2006),and by far the linguistically-oriented semiotic accounts of meaning construction in visual media have achieved“rather marked lack of success”(Bateman and Wildfeuer,2014).As far as the second reason is concerned,I'll follow Kress and Van Leeuwen(1996)who hold that communication is based on an incessant translation or transcoding between semiotic modes and assume that verbal and visual messages“intermesh and interact at all times”(Kress and van Leeuwen,1996:40).The coherence of these multimodal advertisements is encoded in every level of meaning-making across visual and verbal,including the global level,local level and surface level.