1
多模态语篇的连贯构建研究 以中国英语学习广告为例 英文版
1.9.2.2 6.2.2 Evaluation of the existing frameworks
6.2.2 Evaluation of the existing frameworks

The above mentioned three frameworks of layout analysis will be evaluated one by one in this subsection.

6.2.2.1 Kress and van Leeuwen(1996)

Kress and van Leeuwen's(1996)framework bear clear traces from the socialsemiotic perspective.They state in the introduction“our work on visual representation is set within the theoretical framework of‘social semiotics’”(Kress and van Leeuwen,2006:6)and their visual grammar“deliberately a social one”.Visual grammar is defined as“a quite general grammar of contemporary visual design in‘Western’cultures,an account of the explicit and implicit knowledge and practices around a resource,consisting of the elements and rules underlying a culture-specific form of visual communication”(ibid:3).Their point of departure is the group and its practices,and from where they attempt to describe the grammar,rather than adopting an approach which says,“Here is our grammar;do the practices and knowledges of this group conform to it or not?”(ibid:3)

Due to this perspective,one particular feature of Kress and van Leeuwen's framework on the composition of multimodal texts is that it includes considerations of the influence of layout on the information or content-related elements on the page,that is,the influence of the placement of elements on their information values.Specifically,they believe that the placement of elements endows them with the specific informational values attached to the various zones of the image or page:left and right,top and bottom,centre and margin.The placement of elements is not paid much attention to in other two frameworks.

The ubiquitous consideration of information or meaning is rooted in the thinking modes of functional linguistics in which meaning always occupies the central position.As Halliday(1994:F43)claims in his An Introduction to Functional Grammar,“a discourse grammar needs to be functional and semantic in its orientation,with the grammatical categories explained as the realization of semantic patterns”.The influence of Halliday's systemic functional grammar is given explicit attribution in the introduction of Kress and van Leeuwen(1996).The book is even structured around the three metafunctions of their visual grammar adapted from Halliday's three meta-functions of language.Therefore,we can say that the idea of the spatial placement as a means of realizing information value reflects their root in the systemic functional linguistics.

However,this practice causes a serious problem in their framework for two reasons:first of all,the placement is used in two places in their framework in a contradictory way.In the first place,it is regarded as the sole factor for realizing“information value”which is a category in their framework.However,in their elaboration on another category in their framework—salience,they argue that it“is not objectively measurable,but results from complex interaction,a complex trading-off relationship between a number of factors”(Kress and van Leeuwen,2006:202).The factors they list are:size,sharpness of focus,tonal contrast,colour contrast,placement in the visual field,and perspective(2006:202).It means that the placement is used here as a factor constituting visual salience.This contradiction reflects their confusion in categorization.Information value,as a measurement in terms of meaning or content,should not be listed as a category parallel to salience and framing which are terms in the layout level.Secondly,according to their statement,visual salience/weight“creates a hierarchy of importance among the elements of spatially integrated texts”(Kress and van Leeuwen,2006:202).The“importance”they state here should mean the importance in terms of meaning,that is,information value.In other words,salience should be the means the designer uses in the layout to realize information values of various semiotic elements.Placement is,as they have identified,one of the multiple resources for the salience.This idea can find support in the analysis of the English learning advertisements.

Therefore,the remedy is to cancel the information value as an independent category and put placement as a factor under salience which is itself the realization of information value in the layout.

6.2.2.2 Bateman(2008)

In Bateman's framework layout segmentation which means the identification of minimal units in the layout is of little use for this book whose focus is on the coherence and cohesion between various units.The other two aspects—realization information and layout structure information—are useful for the book because they involve the relations between semiotic units in the layout.

6.2.2.3 Kostelnick and Roberts(1998)

Compared to the previous two frameworks which are theory-driven,Kostelnick and Roberts'framework from the perspective of design is more detailed and operational.They provide a systematic set of criteria for designing and analyzing layout.Their framework is generalized from their knowledge and experiences in the design practices and combines the cognitive mechanisms in reading such as the Gestalt principle in visual perception.Because reader's perception and comprehension of the multimodal English learning advertisements are also important for the construction of coherence as discussed in Chapter 2,many of their views are very useful in the formation of my own framework.Another reason for its usefulness lies in its orientation in documents in real contexts,many of which are multi-paged,such as leaflets and brochures.Therefore,many devices they list in the supra level in the framework,for instance,those signal the continuity between the pages,are especially useful in the analysis of the English learning leaflets.

6.2.2.4 Summary

Although the three frameworks for layout analysis start from different theoretical assumptions and follow different tracks of analysis,they share a lot in common.

First of all,because the visual features in layout are numerous and of different natures,they should be classified into categories and used separately in application.The complication of layout analysis provides opportunity to explore the full potentiality of layout as a layer of semiosis.Accordingly,we can expect that the cohesive devices for connecting the semiotic elements in the layout of discourse should also be placed in a complicated network.

Secondly,they all regard layout as a semiotic layer which is both interrelated with and independent of other layers like content and context.Kress and van Leeuwen regard the composition as“the way in which the representational and interactive elements are made to related to each other”and“the way they are integrated into a meaningful whole”(1996/2006:176).Bateman sees layout as a layer of material expression between the layer of visually perceptible elements(the GeM base layer)and the layer of meaningful distinctions(rhetorical base layer)(Bateman,2008:115).Kostelnick and Roberts(1998)see layout as a visual language which is independent and has a fully-developed system evidenced by their elaborated framework.What is more important,it provides knowledge and a set of principles and guidelines for design professionals.Accordingly,the cohesive devices in the layout should also be regarded as the formal cues for the coherence relations in content.

Thirdly,the classifications they made,regardless of how different the terminology they use,share some fundamental principles.Specifically,they all acknowledge that there are two respects of layout features:(i)about the structure and organization of the page as a visual space,and(ii)about the typographical resources for adjusting visual salience.The first respect is elaborated by the“framing”in Kress and van Leeuwen(1996/2006),the layout structure by Bateman and resources for arrangement,especially the spatial sub-system in Kostelnick and Roberts(1998).The specific resources for representing structure include framing,grouping and headings.The second respect of layout analysis is represented by salience of Kress and van Leeuwen,intrinsic and extrinsic features for realization information in Bateman(2008)and resources for emphasis in Kostelnick and Roberts(1998).

The two aspects are believed to have central importance in signaling the coherence relations,both global and local,in the layout of multimodal discourses.The reason is as follows:the first aspect,the macro-structure of the document,as argued in Chapter 4,is the realization of the global discourse topic in an organized and coherent way.The second aspect,visual salience,which is a signaling device of the relative degrees of importance in meaning attributed to the various semiotic elements,is important in marking the relations between different semiotic elements inside a discourse unit or segment,and thus is a useful device of signaling local coherence relations.

In all,what these frameworks have in common is very revealing for the analysis of how semiotic elements are organized in a cohesive manner in the layout of multimodal discourses,so they'll lend useful insights to in the formation of my own working framework of the analysis of cohesive devices in the multimodal English learning advertisements.