2.2.1　Key tenets of multimodal discourse analysis

Multimodality is a relatively new research interest that understands communication and representation to be more than about language，and which attend to the full range of communicational forms people use—image，gesture，gaze，posture，and so on—and the relationships between them.It is largely inspired by Systemic Functional Linguistics and thus shares the basic tenets with the social semiotic and functional theories of language.

Systemic functional linguistics regards language as a kind of social semiotic which has a range of counterparts in the complexity of human communication.This awareness of multimodality is reflected in Halliday and Hasan（1985），

...there are many other modes of meaning，in any culture，which are outside the realm of language.These will include both art forms such as painting，sculpture，music，dance，and so forth，and other modes of cultural behavior that are not classified under the heading of forms of art，such as modes of exchange，modes of dress，structures of the family，and so forth.These are all bearers of meaning in the culture.（Halliday and Hasan，1985：4）

Halliday's three metafunctions are the most frequently employed analytical framework in multimodal discourse analysis.For this purpose，they have been extended from purely linguistic sense to multimodal sense.Experiential meaning，in the multimodal sense，refers to the internal relations between the depicted participants and the settings.The analytical concepts are also borrowed or adapted from systemic functional theories of language，such as non-transactive and transactive processes，actions/vectors（direction）formed by the eyeline，gaze，etc.Interpersonal meaning refers to the social relations between participants and the world.The common resources for interpersonal meaning in image include distance，gaze and perspective.For example，in the case of distance，visual images may stimulate closeness or distance between viewer and the participants in the text in varying degrees that run from very close shot，medium close shot，medium long shot，long shot，to very long shot.Textual meaning，or compositional meaning in multimodal term，usually involves with two kinds of structures：（i）horizontal structure when presenting visual information as Given or New;（ii）vertical structure when presenting visual information as Ideal（top）or Real（bottom）（Baldry and Thibault，2006：39）.

Other key theoretical assumptions underlying multimodality are summarized in Jewitt（2009：14—15）.They are interconnected and listed as follows：

First of all，language is part of a multimodal ensemble.Although language enjoys priority in both research and society，multimodality assumes that representation and communication always draw on a multiplicity of modes，all of which have the potential to contribute equally to meaning.

Secondly，each mode in a multimodal ensemble is understood as realizing different communicative work.Multimodality assumes that all modes，like language，have been shaped through their cultural，historical and social uses to realize social functions.

Thirdly，people orchestrate meaning through their selection and configuration of modes.As a result，the interaction between modes is significant for meaning making.The meanings in any mode are always interwoven with the meanings made with those of all other modes co-present and“co-operating”in the communicative event.The interaction between modes is itself a part of the production of meaning.

Finally，the meanings of signs fashioned from multimodal semiotic resources are，like speech，social.That is，they are shaped by the norms and rules operating at the moment of sign-making，influenced by the motivations and interests of a sign-maker in a specific social context.

The implication of these tenets on the research of inter-semiotic relations in multimodal discourses is double-folded.On the one hand，the different meaning-making potential of each mode has to be acknowledged.That is，in order to reveal the potential of each mode，the researcher must avoid bias towards any semiotic and to give due attention to all.On the other hand，multimodality is also concerned with establishing common semiotic principles that operate in and across different modes.The second concern is even more important than the first one because it is the coherence，rather than distinction，between different modes that defines a multimodal text.However，due to the lack of knowledge and expertise in analyzing images，the task to uncover the way in which resources from different semiotic modes are integrated into a coherent discourse is a challenging one.The current book is a research effort in this direction.