Film Discourse Interpretation: Towards a New Paradigm for Multimodal Film Analysis

Multimodal Studies Exploring Issues and Domains Kay L. O’Halloran and Bradley A. Smith Multimodality, Cognition, and Experimental Literature Alison Gibbons

Film Discourse Interpretation

Edited by Kay L. O’Halloran, National University of Singapore

Janina Wildfeuer

ROUTLEDGE STUDIES IN MULTIMODALITY

ROUTLEDGE STUDIES IN MULTIMODALITY

Multimodality in Practice Investigating Theory-in-practice-through-methodology Sigrid Norris Multimodal Film Analysis How Films Mean John Bateman and Karl-Heinrich Schmidt Multimodality and Social Semiosis Communication, Meaning-Making, and Learning in the Work of Gunther Kress Edited by Margit Böck and Norbert Pachler Spoken and Written Discourse in Online Interactions A Multimodal Approach Maria Grazia Sindoni Critical Multimodal Studies of Popular Discourse Edited by Emilia Djonov and Sumin Zhao Film Discourse Interpretation Towards a New Paradigm for Multimodal Film Analysis Janina Wildfeuer

www.routledge.com

Film Discourse Interpretation Towards a New Paradigm for Multimodal Film Analysis Janina Wildfeuer

i

Film Discourse Interpretation

This book contributes to the analysis of film from a multimodal and textual perspective by extending formal semantics into the realm of multimodal discourse analysis. It accounts for both the inferential as well as intersemiotic meaning-making processes in filmic discourse and therefore addresses one of the main questions that have been asked within film theory and multimodal analysis: How do we understand film and multimodal texts? The book offers an analytical answer to this question by providing a systematic tool for the description of this comprehension process. It aims to advance knowledge of the various resources in filmic texts, the ways the resources work together in constructing meaning and the ways people understand this meaning construction. This new approach to film interpretation is thus able to remodel and improve the classical paradigm of film text analysis. Janina Wildfeuer is a postdoctoral research fellow in the Faculty of Linguistics and Literary Science at Bremen University, Germany.

i

i

i

229x152 HB

i

Routledge Studies in Multimodality Edited by Kay L. O’Halloran, National University of Singapore

1. New Perspectives on Narrative and Multimodality Edited by Ruth Page 2. Multimodal Studies Exploring Issues and Domains Kay L. O’Halloran and Bradley A. Smith 3. Multimodality, Cognition, and Experimental Literature Alison Gibbons 4. Multimodality in Practice Investigating Theory-in-practice-through-methodology Sigrid Norris 5. Multimodal Film Analysis How Films Mean John Bateman and Karl-Heinrich Schmidt 6. Multimodality and Social Semiosis Communication, Meaning-Making, and Learning in the Work of Gunther Kress Edited by Margit B¨ock and Norbert Pachler 7. Spoken and Written Discourse in Online Interactions A Multimodal Approach Maria Grazia Sindoni 8. Critical Multimodal Studies of Popular Discourse Edited by Emilia Djonov and Sumin Zhao 9. Film Discourse Interpretation Towards a New Paradigm for Multimodal Film Analysis Janina Wildfeuer

i i

i

i

i

Film Discourse Interpretation Towards a New Paradigm for Multimodal Film Analysis

Janina Wildfeuer

i i

First published 2014 by Routledge 711 Third Avenue, New York, NY 10017 Simultaneously published in the UK by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business c 2014 Taylor & Francis  The right of Janina Wildfeuer to be identified as author of this work has been asserted by her in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Wildfeuer, Janina, 1984Film discourse interpretation: towards a new paradigm for multimodal film analysis / Janina Wildfeuer. pages cm. – (Routledge Studies in Multimodality ; #9) Includes bibliographical references and index. 1. Motion pictures and language. 2. Motion pictures—Philosophy. 3. Motion pictures—Semiotics. 4. Discourse analysis. 5. Modality (Linguistics) I. Title. PN1995.4.W55 2013 791.4301–dc23 2013017446 ISBN: 978-0-415-84115-3 (hbk) ISBN: 978-0-203-76662-0 (ebk) Typeset in Palatino by the author, Janina Wildfeuer

i

i

This publication was handed in as a dissertation to the PhD Committee for the Degree of Doctor of Philosophy at Bremen University, Germany. Academic supervisor: Prof. John A. Bateman Academic supervisor: Prof. Wolfgang Wildgen The colloquium took place on June 26, 2012.

i i

i

Contents

1

Introduction 1.1 Making Sense of Film . . . . . . . . . . . . . . . . . . 1.2 Investigating Filmic Textuality . . . . . . . . . . . . 1.3 Exploring Filmic Narrative . . . . . . . . . . . . . . . 1.4 Towards the Logic of Film Discourse Interpretation 1.5 Organisation of the Book . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

2 The Logic of Film Discourse Interpretation 2.1 Relational Meaning-Making in Film . . . . . . . . . . . . . 2.2 Model of Analysis . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 The Analytical Vocabulary for Multimodal Film Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 The Formal Framework for Film Discourse Analysis 2.2.3 The Logical Form of Discourse Segments . . . . . . 2.2.4 The Experimental Set of Film Discourse Relations . 2.2.5 The Syntax for Segmented Film Discourse Representation Structures . . . . . . . . . . . . . . . . . . . . 2.2.6 Glue Language and Discourse Update in Filmic Discourse . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Conclusion of Chapter 2 . . . . . . . . . . . . . . . . . . . . 3 Analysis 3.1 The Lives of Others (2006) . . . . 3.1.1 Summary of the Extract 3.1.2 Analysis . . . . . . . . . 3.1.3 Conclusion . . . . . . . 3.2 Vanilla Sky (2001) . . . . . . . . 3.2.1 Summary of the Extract 3.2.2 Analysis . . . . . . . . . 3.2.3 Conclusion . . . . . . . 3.3 Am´elie (2001) . . . . . . . . . . . 3.3.1 Summary of the Extract 3.3.2 Analysis . . . . . . . . . 3.3.3 Conclusion . . . . . . . 3.4 Requiem for a Dream (2000) . . . 3.4.1 Summary of the Extract 3.4.2 Analysis . . . . . . . . . 3.4.3 Conclusion . . . . . . . 3.5 Conclusion of Chapter 3 . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

1 2 6 11 14 19 21 22 31 32 36 53 59 75 79 84 87 88 89 90 107 110 111 112 122 123 123 124 135 138 138 139 163 164

vi i i

i

i

i

Contents vii

4 Multimodal Narrative Construction in Filmic Text 4.1 The Functional Framework for Multimodal Film Analysis 4.2 The Strata of Film Discourse Interpretation . . . . . . . . . 4.2.1 Intersemiotic Meaning Construction in Filmic Discourse . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Metafunctional Diversification . . . . . . . . . . . . 4.2.3 Knowledge Sources for Building Logical Forms . . 4.2.4 Summary: A Stratificational View of Filmic Logical Forms . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 The Narrative Logic of Film Interpretation . . . . . . . . . 4.4 Towards a Paradigmatic Organisation of Film Discourse Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Conclusion: Common Sense in Multimodal Film Analysis . 5

Conclusions and Outlook 5.1 A Method for Constructing Logical Forms of Filmic Discourse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Towards New Perspectives of Analysis . . . . . . . . . . . 5.3 Towards Bridging the Gaps in Film Theory . . . . . . . . 5.4 Moving on to a Comprehensive Film Theory . . . . . . .

A Transcriptions A.1 WORDS (Everynone, 2010) . . . . . . . . . . . . . . . . A.2 The Lives of Others (Henckel von Donnersmarck, 2006) A.3 Vanilla Sky (Crowe, 2001) . . . . . . . . . . . . . . . . . A.4 Am´elie (Jeunet, 2001) . . . . . . . . . . . . . . . . . . . . A.5 Requiem for a Dream (Aronofsky, 2000) . . . . . . . . .

. . . . .

167 170 178 179 183 186 190 192 196 200 202

. . . .

202 204 207 210

. . . . .

212 213 227 231 245 248

Filmography

265

Bibliography

266

Index

274

i i

i

Page Intentionally Left Blank

i i

i

i

i

1 Introduction “Not the smallest advance can be made in knowledge beyond the stage of vacant staring, without making an abduction at every step.” (Peirce, Hartshorne and Weiss 1979: 8.238)

This book provides a framework for the systematic examination of film interpretation from a linguistic perspective. It aims at a detailed description of how coherence and structure as textual qualities of film guide the recipients’ meaning-making process during reception and evoke interpretive inferences as the grounding elements of narrative comprehension. Meaning in film arises out of the multiple interaction of various modalities such as images, sounds, music, gestures, camera effects, etc., which are stringed together by film editing in a chronological, linear order. The interplay of the modalities results in a narrative text whose comprehension and interpretation requires the spectator’s active participation. As a dynamically unfolding discourse, the combinations of resources change in time and space and continually produce meaningful sequences which have to be analysed during their progression. This analysis is in the same way dynamic and progressive in that it tries to find connections between filmic devices and to construct the unfolding discourse structure of the text by means of inferences and abductive reasoning. Film interpretation is thus an active process of relational meaning-making and inferring its propositional content in terms of assumptions and hypotheses, which the recipient makes according to concrete cues within the text. Contemporary approaches to film analysis have not yet succeeded in systematically defining and describing how filmic devices are intersemiotically combined to narrative structures. This book investigates exactly how this interpretation process takes place with regard to the overall coherence of the text. On the basis of recent theories in functional and formal discourse semantics, it assumes a certain logic of film discourse interpretation which, on the one hand, takes inferential strategies and defeasible reasoning as a fundament for making meaning, but, on the other hand, offers distinct constraints to control the interpretation. Therefore, this book develops a method for constructing the logical form of filmic discourse which gives evidence for the film’s structure and coherence and elucidates the idea of its general textuality. The notion of film as text today can be seen as a new starting point for investigation that bridges the gap between general approaches to film interpretation on the one hand and modern linguistic analysis of how meaning in multimodal texts is created on the other. Whereas traditional 1 i i

i

2 Introduction

film semioticians in the last decades unavailingly tried to find analogies between film and language on the level of syntax and compositional semantics in order to describe decoding mechanisms, new approaches in contemporary linguistics and in particular in discourse semantics make it possible to readopt and revise the examination of filmic characteristics similar to those of language on the level of the text. The main point of interest is the concrete description of the inference processes that operate during the interpretation of film as a narrative text. According to Bordwell, narration is “the process by which the film prompts the viewer to construct the ongoing fabula on the basis of syuzhet organization and stylistic patterning. This is, we might say, the experiential logic of understanding a film’s narrative, the equivalent of the tourist’s guided path through a building.” (Bordwell 2008: 98) It is exactly these mechanisms of guiding and prompting the recipient, thus affecting and constraining his/her inferences within the interpretation of the narration, which, in modern linguistic accounts, are defined as central mechanisms of textuality (Bateman 2013: 11) and which can de facto be described in a logical framework (cf. Asher and Lascarides 2003). The book will apply this framework and its analytical principles to filmic text in order to formulate the logic of film discourse interpretation as a new approach to film analysis which combines a cognitive view of the recipient’s comprehension activity with a linguistic examination of the film’s textual contribution. The aim of this introduction is to establish the underlying basis of film analysis from a textlinguistic perspective. In section 1.1, the subject matter of this book, multimodal meaning construction in filmic text, will be elucidated in detail with the help of a brief example. In section 1.2, the notion of film as text and its textual qualities will be further expanded and related to the claim of an adequate linguistic analysis of film, which will furthermore be followed from the view of filmic narrative comprehension in section 1.3. Finally, section 1.4 will set out the basis of its logical foundation.

1.1

Making Sense of Film

As a spatially, temporally, and in particular dynamically unfolding artefact, film shows “constantly varying configurations of sound, image, gesture, text and language” (O’Halloran 2004: 110). It is thus a multimodal document whose semiotic resources interact and operate according to various principles and in order to create the film’s overall meaning potential. Building on basic assumptions in Multimodal Discourse Analysis (cf. Kress and

i

i

i

i

Film Discourse Interpretation 3

van Leeuwen 1996; Kress and van Leeuwen 2001; Jewitt 2009), these principles are manifold and cannot be generalised with regard to individual modes and their task within the interaction. Instead, filmic specificities such as montage, or continuity editing, additionally play a role and affect meaning-making constructions. The principles operate not only on the level of one single mode, but in particular across different modes. Their intersemiosis accounts for the diverse meanings which have to be identified by a recipient when watching a movie. A brief example will point out these mechanisms and the meanings involved. In figure 1.1, a short sequence from the movie The Lives of Others (Henckel von Donnersmarck, 2006), which will be analysed in detail later, is portrayed. It displays a common technique used within the film, a cross-cutting montage of two different settings. On the one hand, there is the first protagonist playing the piano; on the other hand, a second protagonist is listening to the playing by a headset at another location (depicted in the second line of figure 1.1). This monitoring situation is one of the film’s main topics and is therefore introduced from the very beginning. When the first protagonist begins playing the piano, the camera starts moving from the close-up of his hands to a close-up of his face and the character standing behind him. Despite the cut to the second setting, the camera movement continues as a 360-degree tracking shot, portraying the second protagonist first from behind and then in a medium close-up front view, focusing on his emotions. The piano playing runs through all shots with the same sound quality, although the second protagonist only hears it with the headset.

Figure 1.1: Montage sequence from The Lives of Others (Henckel von Donnersmarck, 2006) (see section 3.1.2 for further analysis)

i i

i

4 Introduction

The example extract shows two different settings with different events happening at the same time: ‘playing the piano’ (shown in the first two and the last images) and ‘listening to the piano playing with a headset’ (shown in the second line). The parts of the story are deeply connected and uphold a coherent film extract, which also influences the unfolding of the film’s story (cf., e.g., Falck 2006). Camera movement and music create a dynamic association of the two settings and their protagonists. The cross-cutting montage does not only juxtapose the two parts of the story by different colour schemes (the images of the piano playing are held in brown and beige, the images of the observation scene in grey colours), but also interrelates them by strong parallels within the images. The continuous sound track and the slight camera movement, continued across the cuts, strongly support this interrelationship. Meaning here is created cross-modally due to the intersemiosis of the image and music. As a consequence, a recipient is able to construct the diegetic world, its participants and the actions they are involved in. He/she can identify the two protagonists because of the names given to them in the story’s previous unfolding. The music in the background is recognised as a diegetic piano sound, which is played by the recipient and which can be heard at the second setting because of the monitoring situation introduced previously. The short sequence is one of the key scenes within the film that depicts a significant change of the main protagonist’s attitude towards the monitoring routine. This is partly manifested within the extract, when the camera focuses on the second protagonist listening to the piano playing and a teardrop on his face becomes visible. There is thus an event of ‘crying’, which can be interpreted from the multimodal composition and in particular the work of the camera, and which is somehow connected to the preceding events. It can furthermore be interpreted that the second protagonist presumably begins to cry because of the music he is listening to. His reaction can therefore be seen as a consequence or result of the events happening before. This interpretation can be traced back to the structural description of the story, since the protagonist and in particular his emotions are clearly focused by the medium close-up in the tracking shot (image 4 in figure 1.1), so that a recipient will likely interpret this emotion as playing an important role for the whole extract. Thus, the filmic composition does not only hold the two settings together, but also accentuates several more fine-grained aspects of the description to guide the recipient’s focus. Bateman and Schmidt point out that this kind of composition cannot only be considered as the film’s audiovisual design. Instead, “it contributes already to filmic comprehension, guiding the viewer to particular conclusions concerning how the story is being told” (Bateman and Schmidt 2011: 7). This process of guiding the viewer is the textual logic of the film, which is based on inferences the recipient draws from the multimodal content and

i i

i

i

i

Film Discourse Interpretation 5

context due to his/her world and film knowledge and further information sources, just as has already been described for the interpretation of the short extract above. Understanding and interpreting a film is thus not a matter of simply decoding the semiotic resources, but a process of abductive reasoning and logically concluding the content (see also section 1.4). As Bordwell puts it: “Understanding is mediated by transformative acts, both ‘bottom-up’—mandatory, automatic psychological processes—and ‘top-down’—conceptual, strategic ones. The sensory data of the film at hand furnish the materials out of which inferential processes of perception and cognition build meanings. Meanings are not found but made.” (Bordwell 1989: 3) For the example depicted above, it follows that the shot of the crying protagonist is likely to fulfil a specific function within the context, namely emphasising the emotional reaction, which is not only evident in the image but also has considerable influence on the development of the story. This function, however, has to be inferred from the overall filmic context and the knowledge the recipient already has because of the preceding unfolding. It is an effect of the logical interconnection between the filmic devices, but cannot be defined simply because of its particular semiotic existence. In contrast to the traditional structuralistic approach to semiotic codes, which, according to Bateman and Schmidt (2011), has to be abandoned in the realm of film interpretation, meaning-making is a dynamic process of inferential reasoning. This refers to the recipient’s cognitive capacity on the one hand and the contextual circumstances of the text on the other, and also relies on finding connections between filmic devices and their functional embedding. These connections, constituting the structural composition of the extract, can be described formally as given in figure 1.2. This depiction is the result of relating the different events that have been described above to each other by discourse relations which outline the causal and text-structuring circumstances within the story. It then becomes visible that the event of crying, which is depicted here by the formal label π3 , is in fact related to the other events by a relation called Result. This relation indicates the causal logic given within the text, which refers, on the one hand, to the multimodal composition of the film and, on the other hand, to the knowledge that music sometimes evokes emotions. The protagonist’s emotion can therefore be considered as a plausible reaction to the music he is listening to. The two Parallel-relations refer to the filmic strategy of cross-cutting the two events happening at the same time in different settings. They indicate the textual parallelism within the description, which gives evidence for

i i

i

6 Introduction

π1 = ‘playing’

Parallel π2 = ‘listening’

Result π3 = ‘crying’

Parallel π4 = ‘playing’

Figure 1.2: Graphical representation of the discourse structure of the extract from The Lives of Others the overall coherence of the extract. Although the event of crying is parallelised to the event of playing, its specific function is indicated here by another, causal relation that goes beyond the simple structural positioning of the other relations. The viewer’s activity and attention is thus directly guided to the specific device and the inference process is constrained by the textual focus and the structural development. Exactly this structural composition and the resulting coherence make visible the textual logic operating within the film, its textuality. This is the point of contact between the cognitive approaches to film analysis and the linguistic examination of its meaning-making strategies. As will become clear throughout the following chapters, the general understanding of text and discourse today makes it possible to combine the notion of inference with text organising principles and to point out exactly, and in particular systematically, how meaning in film can be constructed.

1.2

Investigating Filmic Textuality

The notion of film as text is a deep-seated convention in film theory which has been pursued since its early beginnings. The search for meaningful connections between filmic devices and the attempts to systematically outline these connections have always been central interests in film analysis. Simultaneously, literary approaches often served as a basis for examinations of film’s narrative, for example. Thereby, the primary source for the

i i

i

i

i

Film Discourse Interpretation 7

comparison between film and text has been, and still is, the general structural composition of the film, as already indicated above. Andr´e Bazin for example states that “the final significance of the film was found to reside in the ordering of the[ ] elements much more than in their objective content. The substance of the narrative, whatever the realism of the individual image, is born essentially from these relationships [. . . ]—that is to say an abstract result, none of the concrete elements of which are to be found in the premises [. . . ].” (Bazin 1967: 25f.) For this reason, questions concerning the notion of montage and filmic composition in general have always been placed in the foreground of description in order to analyse film’s ability to create coherence in time and space (cf. Bordwell and Thompson 2001). The general assumption that film and language share relevant features of their organisation in this respect is one of the innovative ideas of film theory established and followed over the years. On the basis of this analogy, early film theorists such as Pudovkin (1926) and Eisenstein (1949) tried to illustrate logical relations between shots and to apply basic linguistic features, such as syntactic structure, to film analysis. A short summary of the work of the Russian formalists in this regard is given for example in Tseng (2009, 2013). In the 1960s, mainly, but not exclusively, the Metzian semiotic tradition gave a new and strongly linguistically influenced basis to film analysis that first and foremost included the question of how meaning is created in larger units, i.e., across different cuts, throughout the filmic text. Building on the fundamental distinction of ‘langue’, ‘parole’ and ‘langage’ introduced by Saussure (cf. Bouquet and Sanders 2006), Christian Metz proposed to define film as ‘langage sans langue’, i.e., as a system of these large units, but without any grammatical or syntactical rules (cf. Metz 1974: 88). The ordering of these larger units made it possible, in his opinion, to maintain space and time, and thus carry narrative structure. As Bateman and Schmidt point out: “The manifest intelligibility of film, not only at the level of the individual images but also at the more seemingly conventionalised levels of narration and storytelling, led Metz to consider the basic assumption that film is made up exclusively of infinitely varying natural signs untenable. He concluded that there then had to be some other level(s) of characterisation that would provide the formal distinctions necessary to build a code. This would then cover the meaningful options for creating filmic meaning that appear to be regularly employed by both film viewers and film-makers.” (Bateman and Schmidt 2011: 102)

i i

i

8 Introduction

Metz accordingly developed the grande syntagmatique, the famous model of those larger units of film which can be manifested as meaningful combinations of shots. A depiction of the classification proposed is given in figure 1.3.

Figure 1.3: The Grande Syntagmatique proposed by Christian Metz (adapted from Metz 1974: 146) This is a hierarchical organisation of eight types of syntagma organised in terms of dichotomies made successively because of the different relationships holding between filmic elements. The autonomous segment at the top of the classification is the largest narrative unit considered, which can then be subdivided into smaller units. A detailed description of the individual syntagmas on the one hand and a comprehensive discussion of the problems of this approach from a more general point of view, as well as a reformulation of the framework, is given in the works of Bateman (2007) and Bateman and Schmidt (2011). They underline in particular the complexity of the Metzian account and the regrettably rare practical application, which has led to fundamental misunderstandings and misinterpretations of its status and value in the past. Various other accounts (cf., e.g., Moeller-Nass 1986; Colin 1995) have similarly attested a ubiquitous lack of results within the approach, compromising its effectiveness for detailed film analysis. One of the main points of criticism is “that, despite its name, the grande syntagmatique in fact has a very weak notion of syntagmatic organisation—i.e., there is no structure. Without a stronger specification of syntagmatic organisation, there is no way of constructing both smaller units [. . . ] and larger units, which can help move us towards more adequate interpretations of filmic interpretation as a whole.” (Bateman and Schmidt 2011: 114) Furthermore, Bateman (2007) argues that Metz did not only fail to elaborate exhaustively on the structural unfolding of the syntagmatic entities,

i i

i

i

i

Film Discourse Interpretation 9

but also misleadingly ignored further semiotic dimensions of analysis and description, such as the paradigmatic organisation of the units. This fundamental distinction again goes back to Saussure (cf. Bouquet and Sanders 2006), who defined the paradigmatic axis as providing sets of different items which may be substituted one for another, whereas the syntagmatic axis describes the structural composition of items—as for example on the level of the sentence, which is constituted by different words sequenced according to syntactic rules. Bateman and Schmidt propose that “all non-material semiotic contributions must be characterised in terms of these axes of organisation. Even though this is also commonly found in discussions of film, [. . . ] the filmic application of these constructs leaves much to be desired and has led to considerable confusion in the development of usable accounts.” (Bateman and Schmidt 2011: 83; see section 3.2 in their book for more information) Thus, despite its pioneering work in examining the validity of the filmas-text phenomenon, the Metzian account mainly lacks in linguistic appropriateness and integrity, which, according to Bateman (2007), itself arises out of the problem that linguistics at that time was not able to provide innovative developments concerning texts due to a very constrained view of language. Today, effectual advance in this area makes it possible to readopt the semiotic basis and the general idea of textuality for film. Progress particularly relevant here has recently been made in SystemicFunctional Linguistics, Multimodal Discourse Analysis and Social Semiotics on the one hand, and in formal accounts of discourse on the other. New achievements in these directions now provide tools for analysing not only how meaning in film is created across different modes but also how this meaning construction can be guided and constrained by textual patterns. These are the developments in the domain of discourse semantics that allow re-raising the question of filmic textuality in terms of a reasonable and up-to-date linguistic-semiotics of film. From this perspective, the notion of film as text corresponds well with a modern point of view building on the basic assumption that texts today are no longer seen as static and exclusively verbal messages. Instead, they are dynamically unfolding discourses which have to be analysed according to their contextual embedding and their social functions (cf. Martin and Rose 2003). Nevertheless, they also feature basic qualities of traditional verbal texts. Gunther Kress, for example, highlights the following attributes for multimodal texts in general: “A text is a multimodal semiotic entity, seen as ‘having completeness’, by those who engage with it. Its sense of completeness derives from a (shared) understanding of the social occasions in which it was produced, in which it functions or to

i i

i

10 Introduction

which it alludes. The text has features of internal and external cohesion and, as an integrated meaning-entity, of coherence.” (Kress 2010: 148) On this account, we propose a concrete definition of film as a multimodal text which is meaningfully structured by a variety of semiotic modes. It is a dynamic, but formally confined artefact in chronological, linear order. It may have intertextual references to further text types and may produce various communicative intentions according to the context (cf. Wildfeuer 2013).1 Both definitions point out forcefully the qualities of coherence and structure, which have equally been described as an important basis for the interpretation of the film extract above. This goes back to the generally accepted postulate that coherence in discourse is a sine qua non. It is commonly assumed that its absence will certainly lead to a misunderstanding of the meaning of any text (cf., e.g., Bamberg 1983: 417). This general assumption includes the very complex view of linguistic meaning that Kamp calls ‘the dimension of interpretation’: “the meanings of linguistic expressions are—or so we would want to claim—inextricably linked with the interpretive canons that must be brought into play by anyone who wants to grasp their sense.” (Kamp and Reyle 1993: 1) This accounts as well for filmic meaning, which is constantly linked to its context and the recipient’s knowledge about the world that has to be activated during the interpretation process. In general, a film viewer is today quite familiar with the way films are created and how they express meaning. It can therefore be assumed that there is a general ability for film understanding (cf. Bateman 2013: 2), which, as has been suggested in the analysis above, functions in great part similarly to the understanding of verbal texts. Text interpretation, both for verbal and filmic discourses, is thus always an interactive process in which the recipient tries to reflect connections and links between entities in order to activate various meanings. Following the tracks back historically, even Aristotle holds the idea that texts are coherent by virtue of rhetorical relationships between clauses (Sandys and Cope 1970). For him, the meaning of relations is covered in properties that are characterised by causing connections. In the context of discourse studies today, this point of origin is restated in that it is the overall relatedness of discourse that generates a coherent text. 1 “Der Film ist ein multimodales, das heißt mit sprachlichen und nicht-sprachlichen Zeichenmodalit¨aten sinnvoll strukturiertes und dynamisches, aber formal begrenztes Artefakt in zeitlich-linearer Abfolge. Er kann auf verschiedensten Ebenen intertextuelle ¨ zu anderen Textvorkommen herstellen und je nach Kontext bestimmte KommuBezuge nikationsabsichten produzieren.” (Wildfeuer 2013: 47)

i i

i

i

i

Film Discourse Interpretation 11

The large number of approaches dealing with topics of cohesion and coherence in discourse gives good evidence for the general interest in describing the comprehension and creation of discursive connectedness by means of relations within the text. Coherence is therefore one of the terms that has been discussed intensely and at considerable length in linguistics as well as in philosophy, Artificial Intelligence research and cognitive science, psychology and logic. Seen as a general principle in the interpretation of all human activities (cf. Lundquist 1989: 123), its construction and recognition is attributed to thinking processes that are established by means of inferences. Hence, coherence is primarily a mental representation assigned to any object by a recipient. For film, David Bordwell has pointed out this process of assigning coherence to the text in the following words: “In watching a film, the perceiver identifies certain cues which prompt her to execute many inferential activities—ranging from the mandatory and very fast activity of perceiving apparent motion, through the more “cognitively penetrable” process of constructing, say, links between scenes, to the still more open process of ascribing abstract meanings to the film. In most cases, the spectator applies knowledge structures to cues which she identifies within the film.” (Bordwell 1989: 3) In discourse semantics, the connections found by the recipient are called rhetorical or discourse relations; they are defined as the “coherent continuation moves” (Hobbs 1979: 68) that the recipient makes during interpretation. Examples for these discourse relations have already been shown within the extract described in the preceding section. They exemplify the process of building narrative discourse structures of the film, which again elucidates its coherence. The framework that will be provided in this book will deliver a set of these relations as film discourse relations. It will then be possible to show with this framework and its analysis of relations how specific filmic strategies, such as the cross-cutting montage in The Lives of Others example, guide a viewer’s interpretation by means of the narrative structure the relations give rise to.

1.3

Exploring Filmic Narrative

As has already been explained in the preceding sections, film comprehension and interpretation is a matter of inferential reasoning concerning the film’s content, the events and participants involved in the story and their functional and social embedding in the film’s context. In reasoning about the discursive relationships which maintain these principles, the recipient tries to find spatial and temporal as well as causal circumstances within the film and its story. It is thus a comprehension of basic narrative

i i

i

12 Introduction

principles, i.e., the “perceptual activity that organizes data into a special pattern which represents and explains experience” (Branigan 1992: 3). The close contact between inferential meaning-making and the construction of filmic narrative has been elucidated in detail by David Bordwell and colleagues (cf. Bordwell 1985; Bordwell, Staiger and Thompson 1985; Thompson 1988; Bordwell 2006; Bordwell 2008). Exploring the viewer’s dynamic prediction of meanings in films based on fundamental narrative principles is still very much up to date within this context. Building on fundamental distinctions made by the Russian formalists at the beginning of the 20th century, Bordwell points out in his theoretical approach significant patterns that account for the construction of a narrative and, in particular, for the creation of the fabula, the actual representation of the film’s story inferred by the recipient as a cognitive construct. In order to relate the fabula to the syuzhet, the articulation of story events by montage and cuts in a film, the recipient constructs certain patterns among those events. These are the patterns of time and space as well as the narrative logic (cf. Bordwell 1985: 34). Mainly the pattern of time plays a central role in film, since the filmic content unfolds in temporal succession and, at the same time, the film as a medium is played linearly in narrative time (cf. Genette 1980). Spatial information is often additionally provided, for example visually described in the depiction of the setting or as inserts giving concrete locations. This information represents the fabula in a “spatial frame of reference, however vague or abstract” (Bordwell 1985: 51). Finally, the narrative logic explores relations among the events which are, according to Bordwell, primarily causal or feature more abstract, comparative principles. These are for example relations such as the Result-relation found in the extract from The Lives of Others above, which draws a causal consequence-relation between two parts of the story. In this regard, Bateman and Schmidt point out that these patterns and, in particular, the ‘double time structuring’ are intrinsic qualities of discourse semantics and therefore qualities of all discourses: “Any semiotic mode provides resources for managing the textual development or ‘unfolding’ of its ‘texts’: this is precisely the function of a discourse semantics” (Bateman and Schmidt 2011: 118). Narrative is thus a general discursive phenomenon which is able to organise segments of a discourse and, on the basis of the assumptions made above, also of a film in terms of its meaningful unfolding. Consequently, this book will take the level of discourse semantics as a starting point for the analysis of filmic meaning. It will formulate a framework of film discourse analysis which accounts in general for the attribute of textuality and in particular for the qualities of coherence and structure in order to outline how the viewer constructs the film’s narrative as an organising principle for his/her experiences. The textual analysis of filmic extracts as explored in this book will therefore

i i

i

i

i

Film Discourse Interpretation 13

outline the viewer’s abstract configurations of narrative, which can be derived from the textual qualities of the film, its structure and coherence, elucidated in terms of film discourse relations (see above). According to Bateman and Schmidt, “shifting attention from narrative as a particular form of textuality to the general notion of textuality itself as a broad principle of discourse semantics reoccurring across semiotic modes then gives us precisely the robust semiotic foundation necessary for approaching the specifics of film.” (Bateman and Schmidt 2011: 291) With this approach, the present book joins the ranks of several works (cf., e.g., van Leeuwen 1991; O’Halloran 2004; Bateman 2007; Tseng 2009; Bateman and Schmidt 2011; Tseng and Bateman 2012; Tseng 2013; and others) that try to combine the notion of discourse and discourse semantics with, for example, the Bordwellian approach of constructivism and Neoformalism in order to account for filmic meanings and film interpretation. Among these, most of the accounts focus on functional approaches to discourse semantics, which highlight “the cultural and social dimensions which enter into the formation and constitution of language and of texts” (Tseng 2009: 9). Tseng, for instance, analyses the viewer’s narrative comprehension by outlining identification patterns and thematic configurations in filmic text. Whereas these approaches by no means give up the notion of inference and abductive reasoning about film’s interpretation, they concentrate more on the functional description of meaning and semiotic resources. This book, in contrast, will mainly focus on the description of the logical background of the inference process, which was pointed out at the outset above as the experiential logic of understanding a film’s narrative (see the quotation taken from Bordwell (2008) on page 2) and which will be further emphasised in the following. The book will thereby combine functional approaches to discourse semantics with more formal accounts that characterise the construction of logical forms of the discourse on the basis of inferential reasoning. Thus, the book aims at including both the bottomup and top-down processes of understanding described by Bordwell as transformative acts of meaning construction (see the quote from Bordwell (1989) in section 1.1). This enables not only the analysis of how meaning is constructed in filmic text, but also the examination of how this meaning construction and the interpretation process is constrained by mechanisms such as those depicted in the short analysis of the extract in section 1.1. Consequently, a film’s narrative is equally understood as the perceptual activity of organising the filmic data, but this organisation will at the same time be characterised in more formal detail and on the basis of fundamental assumptions concerning the inference processes involved.

i i

i

14 Introduction

1.4

Towards the Logic of Film Discourse Interpretation

Both in approaches to discourse semantics as well as in more general accounts of film analysis and interpretation, the notion of inference plays an important role. Meaning-making, as has already been pointed out above, is a matter of reasoning about the best interpretation; the idea of simply decoding the semiotic message of the film no longer persists. Instead, “code is supplemented with and complemented by inference” (Tseng 2009: 3) in order to be able to relate technical devices to their interpretation in context. This is the judgement of the discourse stratum within discourse semantics, which makes it possible to pursue exactly the meaning-making strategies outlined in the example above and which will be portrayed in further detail in the chapter following. Meaning construction based on inferences is essentially defeasible and goes back to the notion of abduction introduced by Peirce as a basic logical form (cf. Peirce et al. 1979; Wirth 2000; Wirth 2005). Within his framework of pragmatism, Peirce distinguishes three classes of arguments which serve for ascertaining truth: deduction, induction and abduction, whereby the latter plays the most central role within his overall pragmasemiotic theory (cf. Gallie 1966: 94): “Abduction is the process of forming an explanatory hypothesis. It is the only logical operation which introduces any new idea. [. . . ] Abduction merely suggests that something may be.” (Peirce et al. 1979: 5.171) As a logical conclusion, abduction seeks a possible cause of an assumption which still remains questionable. It is thus, on the one hand, a very weak conclusion, which, on the other hand, brings about new ideas and interpretations. By this means, certain basic, logical principles follow which do not only aim at formulating new hypotheses, but at the same time also prove and verify those hypotheses. Abduction has been formally defined as follows: “Long before I first classed abduction as an inference it was recognized by logicians that the operation of adopting an explanatory hypothesis—which is just what abduction is—was subject to certain conditions. Namely, the hypothesis cannot be admitted, even as a hypothesis, unless it be supposed that it would account for the facts or some of them. The form of inference, therefore, is this: The surprising fact, C, is observed; But if A were true, C would be a matter of course, Hence, there is reason to suspect that A is true.

i i

i

i

i

Film Discourse Interpretation 15

Thus, A cannot be abductively inferred, or if you prefer the expression, cannot be abductively conjectured until its entire content is already present in the premise, ‘If A were true, C would be a matter of course’.” (Peirce et al. 1979: 5.189) According to Wirth, the logical principles formulated in this quotation require that good and plausible hypotheses can be tested evidently, simply and effectively (cf. Wirth 1995: 408). Thus, abduction involves a certain tension in that, on the one hand, it must as a part of the logical process be accessible to rational control while, on the other hand, it must also elude that control just where its own specific strength of producing fruitful, if also potentially fallible, results comes to bear.2 This is exactly the complexity which has been pointed out for filmic discourse, namely the still-missing description of how technical devices of the text, which can be determined very clearly, and their interpretation by a recipient, which cannot be completely controlled, interact. Abductive reasoning about this interpretation bridges this gap by asking for the discourse’s plausibility, its property of making sense. The logical foundation described above has therefore to be restored to the notion of inference accentuated within, for example, the Bordwellian framework. Whereas Bordwell and his colleagues indeed identify a certain “logic of mapping and modeling that underlies interpretive problemsolving” (Bordwell 1989: 202), they do not, however, formulate a concrete framework capable of analysing the processes and principles operating within the interpretation process on this logical basis. In formal discourse semantics, in contrast, the logical foundation described by Peirce has been applied to the analysis of discourse precisely in order to be able to conceptualise in more detail the problems of interpretation (cf. Hobbs, Stickel, Appelt and Martin 1993; Kamp and Reyle 1993; Asher and Lascarides 2003; Hobbs 2003; Muller and Reymonet 2005). Kamp, for example, points out the inextricable connection between logic as the “science of inference“ and semantics in general by highlighting truth conditions of natural language sentences that also account for the structure of the text (cf. Kamp and Reyle 1993: 13). On this basis, Asher and Lascarides formulate a logic of discourse interpretation which takes account of the principles of abduction and inferential reasoning, while also including important information from the context and the recipient’s world knowledge in order to relate textual qualities to the recipient’s interpretive activity. These researchers assume that the plausibility of discourse is given by its coherence and structure, which, at the same time, enable the description of explicit con2 “Jede Abduktion steht in dem Spannungsverh¨ altnis, dass sie als Teil des logischen Prozesses zwar der rationalen Kontrolle zug¨anglich sein muss, sich andererseits jedoch gerade da der Kontrolle entzieht, wo sie ihre spezifische Leistung vollbringt: fruchtbare, wenn auch potentiell fallible Resultate hervorzubringen.” (Wirth 1995: 408)

i i

i

16 Introduction

straints holding on the interpretation process (see section 2.2.2 for further detail). With the aim of formulating a further approach to film analysis from a discursive perspective, this book will offer a logical approach to film discourse interpretation. Building on a general semiotic framework on the one hand and the recipient’s reasoning activity on the other, logic is understood as the necessary link between the two perspectives; logic is the theory which makes available conditions that determine the reasoning process to be secure (cf. Peirce et al. 1979: 2.1). This book will therefore assume that the framework provided by Asher and Lascarides and in particular their logic of discourse interpretation can be applied to filmic text in order to outline its discursive structure and the discourse relations that construct this structure. In their book Logics of Conversation (2003), Asher and Lascarides provide a set of discourse relations which are supposed to hold between discourse segments in a verbal text because of the truth conditional effects they have on the discourse. These relations will be similarly assumed here and further refined for filmic discourse. This analysis will create Segmented Film Discourse Representation Structures that give evidence for a film’s coherence. As already outlined in the description of abduction, coherence is seen as “an explanation for why the text would be true” (Hobbs et al. 1993: 60). It is the textual quality which recipients fundamentally aim at constructing in their interpretation. This leads Asher and Lascarides to formulate the following principle of Maximise Discourse Coherence(MDC): “The logical form for a discourse is always a logical form that’s maximal in the partial order of the possible interpretations [. . . ].” (Asher and Lascarides 2003: 21) This principle will be assumed to hold for filmic discourse similarly. The partial order of the possible interpretations will be shown by rhetorical connections between the discursive items, which in turn will explain the intuitively preferred interpretations (cf. Asher and Lascarides 2003: 21). These interpretations are accompanied by further principles, such as, for example, relevance, which has been described in detail in the approach of Relevance Theory (Sperber and Wilson 1986). Whereas Asher and Lascarides point out in this context that the notion of relevance can be related to coherence in that discourse segments are relevant only if they are connected somehow to other discourse segments (cf. Asher and Lascarides 2003: 20), Tseng and Bateman highlight the reliance on inferential interpretation based on coded mechanisms in this account (cf. Tseng and Bateman 2010: 3). They cite the following basic statement from Sperber and Wilson: “Inferential communication involves the application, not of special-purpose decoding rules, but of general-purpose in-

i i

i

i

i

Film Discourse Interpretation 17

ference rules, which apply to any conceptually represented information.” (Sperber and Wilson 1986: 176) In contrast to Sperber and Wilson’s approach, which does not provide any specific logic in order to describe these inference rules, this book emphasises logic as the theory which provides the rules and conditions required with clear and distinct information for the inference process. For film, it is assumed that the logic of film discourse interpretation, which will be provided in this book as the analytical framework, will deliver a set of film discourse relations and corresponding logical principles that together give evidence for coherence and structure of a filmic text as the result of the recipient’s activity of interpreting that text. This activity aims at constructing the film’s narrative, as has been pointed out in section 1.3. The interconnection of narrative and logic dates back to the classical era of Greek culture, when representations of narrative came into conflict within the genre of rhetoric (cf. Doxiadis 2010: 79f.). Today, narrative logic is mainly seen as an explanation of the “different blends of states, actions, and events, different proportions of stereotypic and nonstereotypic knowledge, different strategies for distributing participant roles among individuals and entities in the storyworld, and so on” (Herman 2002: 22f.). Furthermore, Herman points out that, in his opinion, stories constitute their own logic which must be seen as “an unreplaceable resource for structuring and comprehending experience, a distinctive way of coming to terms with time, process, change” (Herman 2002: 23). Consequently, narrative—or story logic—is the way recipients understand the world around them and make sense of their experiences. It is thus also concerned with the recipient’s actual perception of the story. Bateman and Schmidt point out that perception in general plays a role within film discourse analysis in that the diverse techniques of creating the diegetic world “rely on perception and, in particular, on natural processes of guiding or directing attention, but film actively intervenes and manipulates this possibility for its own purposes of discourse development. Seen semiotically, we have here abstract textual functions of picking out entities relevant for the exposition that is realised in material form by shaping the audiovisual material so as to bring about attention shifts.” (Bateman and Schmidt 2011: 147) An example of such an attention shift has already been given in the short example extract in section 1.1, and it has become evident that it is exactly the textual structuring which plays an important role for the interpretation process. It is thus necessary to have a combined view of both the perceptual input of the multimodal material and its textual

i i

i

18 Introduction

composition in order to portray the recipient’s understanding and sensemaking. According to Bateman and Schmidt, this is “precisely the role of discourse” (Bateman and Schmidt 2011: 150). The text analytical framework provided within this book will therefore offer the needed discourse analytical description of meaning construction in terms of logical forms which can be built within the interpretation process. The construction of these logical forms for the filmic discourse, both as those forms that describe different filmic events (see section 2.2.3) as well as those displaying film’s discursive structure (see figure 1.2), will outline abstract narrative configurations which are intended to demonstrate how the recipient logically makes sense of filmic devices’ meaningful arrangement within the film text. The discussion in the following chapters then explains how film can be understood on the basis of the viewers’ world, film and context knowledge and with the help of basic textual principles. It is thus a textual analysis constructed on the basis of fundamental approaches in text linguistics and discourse semantics in combination with modern accounts of semiotics and multimodal analysis. This will, as a consequence, prepare the ground for more empirical examination in these fields. This position notwithstanding, according to Christian Metz, “[t]extual analysis, even when enunciative, remains textual analysis. If you want information about audiences and filmmakers, you have to go and get them on the spot.” (Metz 1995: 152) The book will therefore focus on the theoretical description of the framework, the logic of film discourse interpretation, and a first practical application to some film extracts as a basis for further research, but will not itself describe patterns that go beyond the filmic text and the logical patterns available for analysis. Although it may reasonably be assumed that the inferences and patterns proposed here will provide significant sources of hypotheses for empirical exploitation, that exploitation itself lies beyond the scope of the current work. As a textual analysis that builds not only on the film’s inherent technical devices, but also refers to knowledge sources that accompany the interpretation process, the intended analyses will reveal semantic and pragmatic patterns of and within the filmic discourse. Nevertheless, these results will be substantially defeasible, since the inference process is based on situated interpretations depending on the actual context, which can only be assumed according to very general principles, but which cannot be entirely verified without empirical testing. The conclusions which will be drawn concerning film interpretation will therefore remain tentative and may need to concede at any time to refutation on the basis of further information.

i i

i

i

i

Film Discourse Interpretation 19

The analysis will however offer valuable clues to the comprehension of film as one of the most powerful, contemporary text types, and will therefore contribute to those accounts that try to apply recent advances in linguistics in order to establish more detailed and comprehensive examinations of this kind of text.

1.5

Organisation of the Book

The goal of this book is to provide a method of film discourse analysis which makes it possible to outline how meaning in filmic text is constructed by means of inferential reasoning about its narrative composition. Therefore, the book aims at describing logical principles within the discourse that constrain the recipient’s interpretive inferences with regard to the textual qualities of structure and coherence. On the basis of recent accounts in discourse semantics, which will be elucidated further in the following chapter, chapter 2 will develop the analytical framework of the logic of film discourse interpretation. A first short analysis of relational meaning-making in filmic text in section 2.1 introduces this framework with example descriptions of its formal notations. The model of analysis in section 2.2 then gives a comprehensive overview of the framework. This first explains and portrays the analytic vocabulary needed for the analysis (section 2.2.1) and furthermore demonstrates the theoretical background in formal discourse semantics in order to provide the main analytical tools (section 2.2.2). On this basis, the chapter then describes the construction of the logical forms of the discourse segments (section 2.2.3) and the set of film discourse relations (section 2.2.4). Furthermore, it gives a detailed description of the syntax of the formal language used in this book (section 2.2.5) and the logical principles’ operation in the interpretation process, specifically discursive glue language and discourse update (section 2.2.6). In chapter 3, the theoretically formulated framework is applied to four different film extracts from different genres. Whereas detailed transcriptions of each film extract are given in the appendix, each section in this chapter includes a summary of the respective film extract, the detailed analysis of structure and coherence of the excerpts and a short conclusion of what is achieved by the analysis. The different films have been chosen for analysis in order to encompass the variety of film techniques available in contemporary cinema and to demonstrate the adequacy of the analytical tool for describing these techniques and their textual composition. On the basis of this analysis of concrete examples, chapter 4 sets out how the analysis in the preceding chapter gives evidence for multimodal narrative comprehension in filmic text. It therefore combines the formal examination of the filmic extracts with a functional consideration of the abstract process of meaning construction in order to further emphasise

i i

i

20 Introduction

the viewer’s comprehension mechanisms. Furthermore, chapter 4 gives a concluding overview of the interpretation of filmic narrative in linguistic terms and with regard to the specifically filmic, textual logic. Finally, in chapter 5, the main contributions of the book to the linguistic examination of meaning construction in film are summarised and future prospects for research that the book opens up explored.

i i

i

i

i

2 The Logic of Film Discourse Interpretation The aim of this book and its analysis is an explicit formulation and presentation of a film discourse framework that enables the comprehensive examination of multimodal meaning construction in filmic text. Here, the interaction of the modalities in the discourse is of central interest, since it is the interrelationships between the different semiotic resources that are decisive for the overall structure and coherence of a film and which therefore account for its meaning and interpretation as text. This meaning construction and attribution within the unfolding discourse is a process of abductive and defeasible reasoning to be described mainly in terms of inferences. The inferential strategies operating during the interpretation are then the main analytical focus which the following framework will develop. Within this account, the synthesis of research in discourse semantics will elucidate what is specifically required in modern discussions concerning film analysis: i.e., the combined examination of semiotic codes and inferences and their interdependencies. Tseng points out that “codes should not be restricted to non-inferential mechanisms and inference should not be totally isolated from narrativeknowledge systematizable paradigmatically. Codes need to account for two things: inferential strategies and their relation to functions/purposes.” (Tseng 2009: 37) It is exactly the last point which this research primarily addresses. In order to bridge the gap between a basic description of filmic devices on the one hand and the interpretational process of inferring the meaning potential of the filmic text on the other, the analytical framework presented here will examine these two different discourse strata (cf. Tseng 2009: 37) from two different levels: it will combine the formal description of the inference process with a functional analysis of the communicative purposes identifiable in the discourse. It is assumed that it is equally possible to explore the reciprocity between filmic coherence and its interpretation in the logical manner supposed for verbal discourse, as for example described in Segmented Discourse Representation Theory (SDRT) (cf. Asher and Lascarides 2003; see section 2.2.2 for further information). This book therefore intends to describe the logic of film discourse interpretation. This logic is differentiated into two individual logics which influence the process of discourse interpretation: (1) the logic of information content, which reasons about the discourse’s meaning potential, and (2) the logic of constructing the logical form of this

21 i i

i

22 The Logic of Film Discourse Interpretation

discourse. These two logics will be described by means of the analytical tools they provide for film analysis. The first logic gives information about the formal language which is supposed to describe the semantic representation of the discourse’s content. It therefore outlines how the discourse’s content is transferred into logical forms by means of the semantics of the formal language. In addition, the first logic provides a (paradigmatically available) choice of film discourse relations which will be defined in terms of meaning postulates holding for each relation due to the truth conditional effects they have on filmic discourse. The second logic, which is, following SDRT (cf. Asher and Lascarides 2003), the logic of information packaging and discourse update, provides the default axioms needed for the definition of the discourse relations within the discourse structure. It produces the syntagmatic articulation of the relations in the semantic representation of a concrete filmic text excerpt and explains how meaning is constructed and attributed to the discourse and its segments. Furthermore, it presents how the discourse relations are inferred in and due to the particular context and specifies the knowledge sources activated and consulted for the inference process. For a better understanding of the interpretational process when transferred into a logical representation form, a first example analysis will give introductory remarks on how the more detailed examination following in chapter 3 will take place. Subsequently, the qualities of the two logics will be elucidated within the concrete model of analysis developed.

2.1

Relational Meaning-Making in Film

As an introductory example for the intended style of analysis, the short film WORDS (2010) will be examined in the following.1 The film was produced by Will Hoffman, Daniel Mercadante and Julius Metoyer, who together represent the production company Everynone, and was published online in July 2010 on the website vimeo.com. The short film is an appropriate example for a first illustration of the analysis of filmic meaning-making processes, since it thematises precisely those cross-modal, i.e., intersemiotic, devices between its discursive units and their meaning potential that are typical of film texts with greater length, such as feature films. The analysis will outline on the one hand the different processes of interpretation, and on the other hand the necessity for analytical tools to describe these processes adequately. A transcription of the first 28 seconds of the short film is given in table 2.1. For the transcription of the whole text, see section A.1 in the appendix. 1 The whole video is available at http://vimeo.com/13768695. A comprehensive analysis of the film can also be found in Wildfeuer (2012).

i i

i

i

i

Film Discourse Interpretation 23

WORDS depicts a sequence of at first sight unconnected shots with a continuous soundtrack and a synchronously inserted audio track. It starts with the insert of the title on a black background. The following shot is a close-up of a tape recorder whose play button is pressed. Music then starts and the image immediately cuts to an image of a theatre stage where two persons are playing in the spotlight. A shot of two children running around on a playground outside follows. The next shot depicts a meeting situation where some members of a football team are sitting in front of a board. A medium close-up to this board shows the drawings done by one of the persons planning a new play. A cut to another scene shows a baseball coach with a helmet and a face mask shouting “Play ball!”. The next shot represents a man playing the trumpet; a following close-up of his mouth blowing into the trumpet immediately cuts to another scene where a man blows out candles on a cake. An image of a ventilator blowing slips of paper off a table and a shot showing a bank note blown across a street follow. The continuing music is calm piano and guitar playing, at times supported by a synthesiser. As the title implies, each image or scene depicted in the film is associated with a particular word that has to be interpreted by the recipient. The overall predicate for the first eight shots, for example, is play. Whereas this is written on the play button in shot 2, thus constructed by written language in the shot, the next four shots (3–6) do not contain any verbal information associated with this word. Shot 7 then depicts a person normally associated with a baseball player or coach. Here, it is the audio track, in particular the spoken language (“Play ball!”) which actualises the predicate. The words predicted by the title are thus in the majority of cases not explicitly mentioned, neither through spoken nor written language. They can be interpreted solely by analysing the meaning-making potential of the images and the additional information in the voice track. Consequently, this is a multimodal meaning construction, and it is clearly necessary to consult the interplay of the different modalities for the interpretation. Furthermore, it is necessary to take the whole context into consideration to infer how the particular shots represent the respective predicates. In shot 9, for example, there is a close-up of the trumpeter seen in shot 8 before. The trumpet playing is thus depicted in more detail and can be associated with the predicate of the preceding shots: play. However, the next shots, shot 10 and following, introduce a new predicate which has not been displayed before: blow. There is thus also a direct link between the close-up of the trumpeter who is blowing into his trumpet and the person blowing out candles in the following shot, so that the former image can retrospectively be associated with the predicate blow as well.

i i

i sound of children at play

two girls in a playground outside

4.

2.

sound of batting staves

sound of pressing the button

close-up of recorder buttons, “play” is pressed

a theatre stage, two persons playing in the front

/

title “WORDS” on black background

1.

3.

audio track

shot

shot description

#

/

/

/

/

spoken language

calm piano and guitar music

calm piano and guitar music

calm music starts immediately after the play-button is pressed

/

music

i

24

i

i

i /

/

/

trumpet play

sound of blowing heavily

a football team planning a new play

close-up of a board in the meeting situation

a man with helmet and face mask

a man sounding the trumpet

close-up of the man’s mouth blowing into the trumpet

5.

6.

7.

8.

9.

/

/

“Play ball!”

“Here, the end is pitching. . . ”

“Here, the back is coming off. . . ”

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

25

i

i

i noises from outside (cars)

a bank note on the street

/

/

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

Table 2.1: Transcription of the first 28 seconds of the short film WORDS (Everynone, 2010)

sound of ventilator and the flying paper

a ventilator blowing away some papers

11.

12.

sound of blowing out

a man blowing out candles on a cake

10.

i

26

i

i

i

i

Film Discourse Interpretation 27

The same principle applies for further shots in the film, such as a scene in which a man and a woman are sitting on a bed, not talking to each other (see table 2.2). The images before and after can all be associated with the predicate break (or its phonologically similar counterpart brake as depicted, for example in shot 24; see the transcription). #

26.

27.

28.

shot

shot description man shouting into his cellular phone man & woman sitting on bed, heads down man hitting a TV with a bat

spoken language “You’re breaking up, man, I can’t hear you!”

music calm piano and guitar music

/

calm piano and guitar music

/

calm piano and guitar music

Table 2.2: Extract of the transcription of the short film WORDS (Everynone, 2010) Inferring that the two persons in shot 27 are representing a couple, it is possible to associate the situation of breaking off their relationship and thus to connect the image with the predicate break. What is needed here for finding the appropriate predicate is general knowledge and the ability to interpret the situation shown in the shot. The interpretation can mainly be based on the gestures and facial expressions of the persons in the image, but there is, for example, no influencing voice track. Since it is only the continuing sound track which accompanies the whole sequence of shots, the silence between the two persons is also a crucial factor for the interpretation. Additionally, adequate language skills are necessary to associate the particular vocabulary used. This is true for a considerable number of the scenes, since not all shots can directly be associated with the particular predicate but require a further analysis. The skydiver in shot 48, for example (see the transcription in the appendix), is attributed to the propositional context of fall. This is to a great extent reasonable, since the process of coming out of the sky

i i

i

28 The Logic of Film Discourse Interpretation

can be seen as falling down. Besides, the image is followed by different shots associated with the predicate fly, so that its meaning potential can also be linked to this content. However, it is hardly understandable that one of the following shots, shot 50, showing a boy diving off a high board, is integrated as well into the context of fall. High diving is normally not seen as falling from the platform, but rather as diving intentionally. It is thus mainly the detailed analysis of the image itself which authorises this association, since the action processes in the images closely resemble one another. They all show a movement of coming down from somewhere above and therefore show strong parallels in their structure. A similar parallelism is given between the first and the last shot of the short film which both represent the process of activating a button: play at the beginning, the ‘space’ bar at the end of the film. Since it is possible to associate every shot with words that summarise and conclude the different situations which are at first sight not directly related to each other, the film as a whole emerges as a coherent text whose meaning potential is identifiable by the recipient. The continuous sound track and the cross-modal realisation of meaning in some cases support the high density of relationships. Although the film does not have a clear narration or a definite figuration, it is a coherent text that ‘makes sense’. This gives evidence for relational meaning-making processes, i.e., the textual and discursive qualities which have been mentioned in chapter 1. For a more detailed and linguistically conclusive analysis of these qualities and the processes operating throughout the interpretation, a descriptive tool is needed for the examination of how recipients can make sense of the interplay of the various resources used in the film. The aim is to demonstrate that every predicate introduced in this filmic discourse can be connected to other text segments in a meaningful way. Accounts in formal discourse semantics explicitly provide the analytical tools needed for this kind of examination. Their description is a detailed outline of the inference process depending on different knowledge sources interacting during the interpretation process. As a result, it is then possible to describe the meaning construction in the example as a complex Segmented Discourse Representation Structure (SDRS) that is built by rhetorical relations which can be inferred between the different shots. In figure 2.1, for example, a first graphical representation of such a discourse structure is given for shots 2–10.2 This structure is built by rhetorical relations that can be inferred between the different shots because of specific default axioms that must be fulfilled by the discursive context. Every predicate described above is labelled by some πn which in turn can be related to another label πm by a particular relation. This depiction conspicuously resembles the representation forms chosen within the framework 2 The first shot in the transcription represents the title of the short film and is not part of the structure. Nevertheless, the title influences the meaning-making process considerably.

i i

i

i

i

Film Discourse Interpretation 29

of Multimodal Discourse Analysis (MDA) (cf., e.g., van Leeuwen 1991), but contains a more formal description that, in a further step, also enables a complex view of the inference processes involved (see section 2.2). π0

π1 Parallel π2 Parallel π3 Parallel π4 Parallel π5 Parallel π0

π6

Elaboration

π7 Contrast π8

Figure 2.1: First graphical representation of the SDRS of the first shots (2–10) of WORDS This structure gives an overview of the different relations found between the labels and describes at the same time the subordination of some parts of the filmic discourse which are dependent on the preceding context, as has been described above as the change from play to blow. It becomes visible that the overall content of the text, labelled π0 , consists mainly of parallel relations which can be seen as so-called text-structuring relations entailing that the content of the particular constituents is similar. This is

i i

i

30 The Logic of Film Discourse Interpretation

the case for all shots presenting analogical scenes, as for example described for the first two shots in the following representation in table 2.3.

eπ1 = play [v] button “play” (m) [v] unknown actor (n) [a] rhythmic music starts m, n |∼ play (eπ1 )

eπ2 = play [v] unknown actor (o) [v] unknown actor (p) [a] calm, rhythmic music o, p |∼ play (eπ2 )

Table 2.3: The modalities’ contribution in shots 2 and 3 of WORDS: eπ1 and eπ2 Whereas figure 2.1 represents the overall discourse structure of shots 2–10, table 2.3 gives a depiction of how the particular labels can be constructed as eventualities through the different semiotic resources given in the shots. The model of analysis in section 2.2 will return to these formal definitions and the role played by these boxes in detail. In table 2.3, eπ1 and eπ2 display those eventualities which can normally be inferred by combining the meaning potential of the modalities within the shots labelled π1 and π2 . In both forms, this eventuality is play, since the two events of playing are specified on the one hand by written language in the second shot and associated on the other hand with the action process observable in shot 3. Whereas the nontransactional process in shot 2 is dominated by the close-up focusing on the play button, the bidirectional process in shot 3 connects the two unknown actors and dominates the inference process that leads to the predicate play.3 The particular discourse referents, for example the unknown actor (n) in shot 2 (eπ2 ), are tagged with variables (shown in parentheses) to outline dependencies and conditions existing between the referents and to map the processes into a more formal depiction. As can be seen in figure 2.1, the two eventualities in table 2.3 are related to each other by a Parallel-relation. As will be explained in detail in section 2.2.4, this relation holds because of the common theme between the two logical forms; they both depict the activity of playing. The common theme is a condition that must hold for the inference of the discourse relation and that can be defined generally for every Parallel-relation in filmic discourse. In this case, the association given to the theatre play in 3 Kress and van Leeuwen (1996) provide these descriptions of processes for images in their work Reading Images. For a summary of these process types, see section 2.2.1.

i i

i

i

i

Film Discourse Interpretation 31

shot 3 describes an event which can be formulated relatively similarly to the eventuality play depicted in shot 2. The relation between them thus fulfils the default axiom for Parallel given for verbal discourse by Lascarides and Asher (2007) (see section 2.2.2 for further information): (?(α, β, λ) ∧ semantic similarity(α, β)) > P arallel(α, β, λ)

(2.1)

This relation is the preferred connection between the two segments, because no other default axiom can be fulfilled by the direct contextual circumstances. Since the predicate play is made explicit in the first shot of the short film, the association of the second event of the playing actors with this predicate is evident. As becomes visible throughout the graphical and formal representations, the analytical tool applied to the example of the short film WORDS offers an appropriate framework for outlining the process of making meaning in filmic discourse. The notation style, which is adapted from SDRT (cf. Asher and Lascarides 2003; see section 2.2.2), and its combination with rhetorical relations enables a closer analysis of a dynamically unfolding discourse such as film. The examination of discursive qualities, mainly for verbal discourse, is considerably more detailed and particularly well developed. Its application to multimodal discourse therefore also shows great promise for a precise film discourse analysis. Consequently, the framework of discourse relations and the specific theoretical background of SDRT will be elucidated in more detail in the following section in order to provide the main analytical tools for the framework.

2.2

Model of Analysis

This section presents the concrete model of analysis that is taken for the closer examination of film text excerpts in the following chapters. The overall aim is to formulate a logic of film discourse interpretation which uses Segmented Film Discourse Representation Structures (SFDRS) in order to display in detail the interpretation processes which operate during film reception. As described above, the model of analysis will therefore focus on the two logics of film discourse interpretation: the logic of information content on the one hand and the logic of information packaging and discourse update on the other. Together, they will produce a formal framework for constructing SFDRS. Analogous to the formal language for verbal discourse, the formal language provided in this framework uses different forms of vocabulary in the analysis. Firstly, it is necessary to describe and transfer the different modalities of the film extract into logical forms as used in the example analysis above. Consequently, the following sections will outline which analytical tools from MDA and formal discourse semantics will be used

i i

i

32 The Logic of Film Discourse Interpretation

to examine the particular modalities which contribute to the inference processes. The example analysis in section 2.1 sketched a selection of alternative relations which will now be described in detail as a paradigmatic system of film discourse relations. Therefore, section 2.2.4 provides the formal language to define these relations in terms of their meaning postulates and default axioms as illustrated for the Parallel-relation in equation 2.1 in section 2.1. In a further step, section 2.2.5 will give an overview of the syntax of the formal language used for specifying the SFDRS. How the concrete inference processes then operate within the logic of information packaging and discourse update will finally be outlined in section 2.2.6. 2.2.1

The Analytical Vocabulary for Multimodal Film Analysis

In order to demonstrate the concrete processes which operate during the analysis of Filmic Discourse Representation Structures (FDRS), it is necessary to provide a definite analytical vocabulary which explicitly describes the different modalities and their use in the discourse. The aim is to transfer the modalities of filmic discourse into logical forms of this discourse and to describe its overall structure. This transfer of the filmic content is already the product of an extensive inference process. In opposition to the content of verbal clauses, the particular semiotic resources in filmic text do not produce clear predicates which depict the content, but instead have the potential to fulfil different functions depending on the context (cf. Kress and van Leeuwen 2001). The respective level of representation they take up in any particular extract has to be interpreted in terms of the intersemiosis of the modalities. The various semiotic resources operate cross-modally, and it is exactly this interplay which produces the logical forms of the discourse segments. The logic of filmic information packaging and discourse update provides the language needed for describing this interplay exhaustively by building on comprehensive research in MDA. Extensive analytic practice in different fields of study has delivered a multitude of analytical frameworks which are applicable to multimodal texts such as film. For a detailed overview of the contributions the individual modalities make in the inference process, it is necessary to describe them on several levels to reveal their form and meaning. Therefore, the analysis will examine the significance of each separate mode that is to some extent involved in the inference process. The underlying basis for the analysis will be a transcription of each film extract or scene to be examined. According to Baldry and Thibault (2006), transcription and analysis are closely related to each other: “Transcription is a way of revealing both the codeployment of semiotic resources and their dynamic unfolding in time along

i i

i

i

i

Film Discourse Interpretation 33

textually constrained and enabled pathways or trajectories. Analysis synthesises the results of transcription in order to ground statements about textual meaning in a principled and replicable way.” (Baldry and Thibault 2006: xvi) In this regard, the method of transcribing is already an essential part of the analysis, since it elaborates which individual semiotic resources contribute to the meaning-making process and how they unfold in time and space in the filmic text. The ‘minimal unit’ of the transcription is the filmic shot which is, according to Bateman, “understood phenomenologically/perceptually as the apparent result of continuously running the camera” (Bateman 2007: 35). In Eisenstein’s early works on film, the shot had already been an important object for analysis. For him, “each shot acted like a circus attraction to deliver a particular psychological stimulus which could then combine with other neighbouring shots to build the film” (Andrew 1976: 47). Later works then concentrated more on the interaction of elements above the shot, i.e., on montage as “the creative power of film” (cf. Andrew 1976). Accordingly, Eisenstein emphasises the context-relatedness of the shot as an important factor in the creation of meaning in film. From the filmas-discourse perspective, it is exactly this context-relatedness which is a crucial factor in the examination of filmic discursive qualities. Although, as Bateman points out, the restriction to the shot as a minimal unit is clearly a simplification (cf. Bateman 2007: 35), it does offer a useful and sufficient compromise for current purposes. Focusing for example on a frame-based analysis would highly complicate the incorporation of modes other than the visual, since these semiotic resources (e.g., sound and movement) in most cases transcend the unit of a single frame and do not deploy their meaning potential in full within one frame. The analysis of intersemiotic devices between the different modalitites would thus not be possible. As shown in the example analysis in section 2.1, the transcription method used in this book will depict in all cases the main modalities of video and audio, i.e., the respective shots on the one hand and the corresponding parts of the auditory level such as voice and audio track as well as music on the other. This differentiation goes back to general typologies of filmic elements which have been made by several researchers. Kracauer, for example, in his famous early studies of film, distinguishes three types of related modes: the commentary, which includes verbal statements and occasional captions; the visuals, which feature the camera reality; and the sound, which is sound effects and music (cf. Weber 1984; Kracauer 2004). Fledelius, on the other hand, enumerates “a range of parallel-running ‘channels’ of expression” (Fledelius 1979: 50): a picture channel and a text channel on the visual level, a music and a commentary channel as well

i i

i

34 The Logic of Film Discourse Interpretation

as a synchronic-speech channel; and an acoustic-effect channel on the auditory level. All of these channels need to be elaborated more precisely. ¨ In this regard, Stockl (2004) offers a network of modes, sub-modes and features of filmic (and TV) elements which provides for a more detailed differentiation of elements. On the macrolevel, he distinguishes between the two sensory channels which have been described above as the visual and auditory level. The core mode of the visual level is the image, which has two medial variants: the static or still image and the dynamic, moving image. The auditory level has sound and music as core modes. The overlapping mode on both levels is that of language, which is available in written form on the visual level and as speech on the auditory levels. The core modes can again be differentiated into so-called peripheral modes such as, for example, paraverbal means in speech or typography in writing. They differ from so-called sub-modes in that the latter are made up by distinct features. Visual effects of the camera, such as panning and tilting, are sub-modes of the dynamic image which are made up by features such as speed and quality (cf. Stockl ¨ 2004: 13).4 This book will mainly take into account the three upper levels of Stockl’s ¨ network, in particular the core modes and their medial variants. In a second step, each sub-mode or feature which is relevant for the inference process in an actual context will be examined. For the analysis on the visual level, the work of Kress and van Leeuwen (1996) is employed. Their innovative research on the grammatical construal of images provides a framework for describing visual process types. Based on the categorisation of transitivity processes by Halliday, Kress and van Leeuwen assume that images “can ‘say’ (some of) the same things as language—in very different ways” (Kress and van Leeuwen 1996: 50). They further point out that these different ways can be defined as visual structures which have either a narrative or a more conceptual function. Accordingly, their main distinction lies in the difference between narrative and conceptual processes which can relate so-called participants (people, places, objects) to each other. Narrative processes are constructed visually by means of vectors between the participants. Depending on the source of the vector and its 4 Although Stockl ¨ admits to some general inaccurate details in his network (cf. Stockl ¨ 2004: 14f.), he does not provide any indication of the problematic classification of some sub-modes. Speech as a medial variant of language is by no means visual, but is supposed to be clearly mapped to the visual sensory channel in the network. Likewise, typography and layout of score music are attached to the auditory channel, although they are clearly visual qualities. This is mainly a problem of the illustration, since the network cannot effectively combine the different qualities of the core modes. However, the point of hierarchisation leads to further difficulties in the general evaluation of the different modes and sub-modes. Mainly the so-called features are rather indefinite qualities of a multitude of ¨ sub-modes and modes and are in no way distinctive. Nevertheless, Stockl’s network does give a comprehensive overview of most used semiotic resources and is therefore a useful summary for this book.

i i

i

i

i

Film Discourse Interpretation 35

direction, the participants are classified as Actors or Goals. If both participants are realised in one process, this process is called transactional (cf. Kress and van Leeuwen 1996). In addition, Kress and van Leeuwen describe reaction processes, which have a Senser and a Phenomenon as participants, and distinguish between mental and verbal narrative processes. All narrative processes describe actions or states of affairs. Conceptual processes, in contrast, assign attributes to objects. They are classified into analytical, classificational and symbolic processes whose participants are Carriers and Attributes. These processes are not connected by a vector, but present their participants “in terms of their more generalized and more or less stable and timeless essence, in terms of class, or structure or meaning” (Kress and van Leeuwen 1996: 79). Details on the auditory level will be elucidated similarly. Here, the main theoretical basis is given by the work of van Leeuwen (1999). In Speech, Music, Sound, van Leeuwen brings the integration of these three modalities into the focus of analysis and provides a vocabulary for description. Whereas further researchers most commonly concentrate on the supportive function of sound and music in film, van Leeuwen points out their significance as general semiotic resources. He distinguishes “six major domains, sound perspective, sound time and rhythm, the interaction of ‘voices’ (for instance by taking turns or speaking, singing, playing or sounding together in different ways), melody, voice quality and timbre, and modality” (van Leeuwen 1999: 9). Whenever these details play an important role in the interplay with other modalities, they will be mentioned in the transcription or examination. Further categories by van Leeuwen, i.e., the distinction between presentational and representational functions of sound or its capacity to establish social relations, will also be incorporated in the analysis.5 Of further particular interest for this analysis is the theorisation of salience. Tseng (2013) points out that the notion of visual salience and its different factors—which have for example been described by Arnheim (1960) and Arnheim (1969)—have strongly influenced research on visual meaning. Accordingly, van Leeuwen examines three stages of salience on the auditory level: figure, ground and field (van Leeuwen 1999: 16). These three stages are hierarchised in terms of their perspective: “If a sound or group of sounds is positioned as Figure, it is thereby treated as the most important sound, the sound which the listener must identify with, and/or react to and/or react upon. [. . . ] If a sound or group of sounds is positioned as Ground, it is thereby treated as still part of the listener’s social world, but only in a minor and less involved way. [. . . ] If a 5 A comprehensive overview of the work of van Leeuwen is given in Noad and Unsworth (2007).

i i

i

36 The Logic of Film Discourse Interpretation

sound or group of sounds is positioned as Field, it is thereby treated as existing, not in the listener’s social, but in his or her physical world.” (van Leeuwen 1999: 23) In the work of Baldry and Thibault (2006), several further entries within the transcription are mentioned. Columns additionally contain patterns ¨ of kinesic action, which is also part of the level of sub-modes in Stockl’s network (see above), and a metafunctional interpretation which already implies a meta-level of analysis. However, since a transcription always presents the result of a completed and considerably more abstract analysis, it is absolutely necessary first to make the underlying interpretational processes more concrete. As Forceville (2007) for example points out, it is essential to identify significant clusters of variables which take part in the cross-modal meaning-making process and which might help to motivate the choice of entities to be described within the transcription. This book therefore tries to sharpen attention to details of the interpretation which cannot be included in the analysis done by Baldry and Thibault. The description of the eventualities, as given in the analysis in section 2.1, for example, gives reason for significant patterns in multimodal texts that, according to Forceville, should then make comparative studies of multimodality in film feasible (cf. Forceville 2007: 1237). These descriptions will be augmented with a set of film discourse relations which establish relationships between the particular discourse segments. 2.2.2

The Formal Framework for Film Discourse Analysis

This section elucidates in detail the theoretical foundations in text linguistics and discourse semantics that provide the required analytical tools and principles for the framework to be developed in the preceding sections. It has already been pointed out in the introduction to this book that recent progress in linguistics concerning the examination of multimodal texts has primarily been made in the realm of discourse semantics. As one essential part of a larger theory of semiotics, semantics in general is the component that deals with the construction and transmission of meaning conveyed through language. Although it engages a considerable part of linguistics, the theory of semantics not only deals with linguistic data, but also cares about nonverbal meanings that are linked together in a text and that have to be interpreted by the recipient. Van Dijk points out that the concept of interpretation is probably the key concept of semantic theory, since it assigns the semantic object of meaning to the expressions of a text (cf. van Dijk 1985). In discourse, which is, according to Kamp, the principle unit of semantic analysis (see Kamp 2007: 8), this process of meaning attribution goes beyond the limits of the more traditional accounts of semantics, responding to the facts that (1) discourse unfolds in time and therefore is a dynamic construction between the producer and

i i

i

i

i

Film Discourse Interpretation 37

the recipient, and that (2) much information which is taken as part of the discourse’s meaning often is not explicitly mentioned. As a consequence, this meaning has to be inferred by the recipient according to his/her world knowledge, further information sources, and with the help of abduction and defeasible reasoning (see section 1.1). A further basic idea is the notion of context and contextual interpretation. Meaning in discourse generally has a Context Change Potential (CCP) (see Asher and Lascarides 2003: 42), i.e., sentential meaning always has to be analysed intersententially, in context and with regard to information added to the discourse that may change preceding interpretations. It is thus a relational meaning-making process of interpretation that takes into account the dynamicity of the unfolding discourse. In modern discourse semantics, this interpretation is basically disengaged from grammatical or syntactical information in the text, and instead concerns connections between text segments that can be logically inferred by the recipient. These so-called discourse relations build up the structural foundation of the discourse and define its meaning potential, as van Dijk points out: “We merely assume (1) that discourse expressions can be analyzed as sequences of sentences and (2) that meaning units assigned to sentences are propositions, which consist of a predicate and a number of arguments that may have various (case) roles. Hence a first aspect of semantic discourse analysis is to investigate how sequences of sentences of a discourse are related to sequences of underlying propositions and how the meaning of such sequences is a function of the meaning of the constituent sentences or propositions.” (van Dijk 1985: 105) This is most likely to be true for many kinds of discourse, since propositions of meaning can often be established in a similar way for other semiotic resources. It is therefore assumed here that filmic discourse can be organised by relating representations of its events to their interpretation in context while dynamically unfolding in space and time. The short example in section 2.1 has already shown that this kind of organisation can be fruitful with regard to the analysis of how the connections between filmic representations produce an unfolding discourse structure. The concrete description of the various relations holding between the filmic events made it possible to outline how their interpretation in context is guided by this structure and the overall coherence. Generally speaking, the structure of the text can be seen as the starting point of research interest in text linguistics and discourse semantics. In the 1960s and 1970s, the focus in linguistics broadened from the subject of sentences to the reflection of the level beyond the sentence, the level of texts or discursive entities (see for example Harris 1952). Until then, system-theoretical thinking regarded the sentence as the major linguistic

i i

i

38 The Logic of Film Discourse Interpretation

unit in the hierarchy of the language system. However, detailed textual analysis has uncovered that some phenomena can only be described adequately on the higher level of text. Mainly intersentential and contextual devices such as reference or conjunction led to a focus of attention on this level and the relationships between sentences and propositions. One of the most notable works on cohesion and coherence in text linguistics is that of Halliday and Hasan (1976). Their book Cohesion in English focuses attention on the effective textual qualities of a text. Whereas cohesion and the more general term of connexity refer to the mostly verbal and inherent construction of a text, coherence refers to noninherent relationships between textual entities and their context. This basic distinction has given a new impetus for theoretical linguistic analysis. Halliday and Hasan’s work is mainly based on conjunctive and cohesive relations, thus relations that are based on text-inherent linguistic devices, but their effort in describing and analysing the quality of texture and coherence in general (see Halliday and Hasan 1976: 2) has given direction to a trend that is still relevant to this day. At the same time, considerations of discourse studies in general (e.g., Grimes 1975), its grammar and connectedness (e.g., Longacre 1996) have arisen. Other approaches to structuring discourse have defined segmentary units to flexibly build structural patterns by sequential chain-linking (e.g., Rumelhart 1975; Reichman 1985) or have tried to detect macro- and superstructures (van Dijk 1980). To this day, most research in text linguistics and discourse semantics has been and is still done in defining and describing the semantic relationships that account for the structure of some text and at the same time bear some kind of meaning. The approaches concerning the question of coherence and structure will be outlined in the following with the specific objective of formulating a framework of film discourse analysis which mainly accounts for these attributes of textuality. In particular, the discipline of formal semantics will be elucidated in greater detail, since it is mainly concerned with the application of logic to semantic analysis—an aspect that has already been stated to be fruitful and appropriate for the analysis of film interpretation. With the help of the economic and formal tools of logic, the aim is to state the semantics of the text in the first place. The main difference to other accounts then lies in the perspective taken for the description of semantics in general. Whereas representational approaches search for a level of representation of expressions which is comparable to the syntactic or phonological level, model-theoretical and formal accounts try to relate elements of models to representations (cf. Portner and Partee 2002). These models consist of different constituents (entities, properties and situations). Meaning in models is always seen in relation to these situations and properties; truth is determined by certain conditions holding over them. Investigations in this field are mainly conducted by philosophers, logicians and mathematicians; foundational

i i

i

i

i

Film Discourse Interpretation 39

works are those of Gottlob Frege (1980) and Richard Montague (1974). On a model-theoretic level, it was mainly the latter account which led to further developments and gave a basis for recent accounts of formal semantics. The ‘Montague grammar’ undertook the application of formal logics to natural language in order to represent logical structures of language. It was assumed that meaning is characterised by its truth conditions, which can be represented exactly by logical formulae. The logical structures then are translated into an artificial language of intensional logic which can be interpreted due to both its syntactical and semantical compositionality. Further approaches followed in particular the idea “that with a more powerful semantics, a less powerful syntax might be adequate” (Portner and Partee 2002: 4). Today, Montague’s work presents an important step towards the combination of logic and linguistics.6 With a view to the analysis of texts and discourses, that is, on a level above the clause, more dynamic approaches to meaning were developed due to the contextual influence which is given in text and discourse. The efforts of a speaker to adapt his/her utterances to the recipient’s knowledge and interpretive task came to the fore (cf. Saeed 2003: 327), and so dynamically unfolding processes in discourse were also described logically (cf., e.g., Heim 1983; Groenendijk and Stokhof 1991). For the description of multimodal texts and discourses, this is a matter of particular interest. As pointed out above, the notion of syntax and grammar is not a crucial factor for the description of such texts. Instead, structure and coherence established through interrelationships between units have to be elucidated in detail with a view to their dynamically unfolding meaning potential. The most established model developed in the context of dynamic semantics is that of Discourse Representation Theory (DRT) by Hans Kamp and Uwe Reyle (Kamp 1981; Kamp and Reyle 1993). This mainly deals with modelling contextual dependencies between sentences. As an operational framework, it affords the analysis of connections beyond the clause by representing discourse with a Discourse Representation Structure (DRS) that captures intersentential dependencies such as, for example, anaphora or subordination. It presents moreover the syntactical contribution to logical forms and describes how context affects meaning: “a sentence S is interpreted as a relation between an input context and an output one. Assuming that the model M is fixed, these contexts consist of variable assignment functions.” (Lascarides and Asher 2007: 90) Various kinds of anaphora, mainly pronouns and presuppositions, can be described in such logical forms of discourse. However, the set of phenomena DRT deals with is very restricted. Although it focuses on the 6 A short overview of the various accounts dealing with the notion of formal semantics is given in Portner and Partee (2002).

i i

i

40 The Logic of Film Discourse Interpretation

effects of logical structure on anaphora, discourse structure is defined by linguistic expressions and the logical form of sentences, but does not present interpretations of these sentences. Asher and Lascarides claim that DRT and other dynamic semantic theories use procedures that “cannot adequately account for the phenomena they aim to model. [They] provide[ ] unsatisfactory constraints on the resolution of pronouns, temporal structure, presuppositions, and the like [. . . ].” (Asher and Lascarides 2003: 59) To avoid these unsatisfactory constraints, Asher and Lascarides propose to enrich the logical forms for discourse with rhetorical relations (see Asher and Lascarides 2003: 60ff.). They point out that rhetorical relations have “truth conditional effects that contribute to meaning but lie outside the purview of compositional semantics” (Asher and Lascarides 2003: 7). Their use in text analysis can be equated with the application of world knowledge to “reflect the capacity of a discourse to describe things at different levels of detail” (Lascarides and Asher 2007: 94). For verbal text, their strength to provide information about linguistically implicit circumstances in discourse is highlighted. Rhetorical relations not only predict anaphoric bindings in text, but even apply to such elements that refer back to previous elements in discourse (cf. Asher and Lascarides 2003: 6). They are at the same time associated with speech act types and conversational implicatures (cf. Asher and Lascarides 2003: 31) and give information about other text-linguistic phenomena such as temporal structure, presuppositions, etc. This correlates with the description given by Hobbs that coherence relations are “text-building strategies” (Hobbs 1990: 101): “They are also resources to which the speaker may appeal to get him from one sentence to the next when global constraints are insufficient or insufficiently attended to. They are a means of finding a next thing to say.” (Hobbs 1990: 110f.) Based on these assumptions, Hobbs formally defines the different coherence relations by describing the assertions of two sentences or discourse segments and their connection. For instance, the explanation relation can be stated as follows: “Infer that the state or event asserted by S1 causes or could cause the state or event asserted by S0 ” (Hobbs 1990: 91). This explanation can be expressed in the logical formula in equation 2.2 where e1 stands for the issue addressed by S0 and e2 for that of S1 in an environment e (which is the last reference in the formula): cause(e2 , e1 ) ⊃ CoRel(e1 , e2 , e)

(2.2)

The operator ⊃ symbolises a material conditional. This means that cause(e2 , e1 ) as the antecedent in this formula implies CoRel(e1 , e2 , e)

i i

i

i

i

Film Discourse Interpretation 41

as the consequent as in a typical if . . . then-consequence. As Hobbs et al. point out, the relation is named with one item that describes the “information conveyed by the particular segments”; they give the following description of the relation according to this formula: “If a segment which asserts the eventuality e1 is followed by a segment which asserts the eventuality e2 , where e2 causes e1 , then there is a coherence relation between the two segments. The predicate CoRel expresses the vague relation conveyed by the adjacency of the segments, and the predicate cause is one of its possible pragmatic strengthenings.” (Hobbs and Redeker 1997: 2) Thus, the second sentence, S1 , states a reason for the issue of S0 . An example for such an Explanation-relation is given in the sequence cited by Asher and Lascarides (2003: 91): Max fell. John pushed him. As Hobbs and Redeker point out, “there is no mention of ‘explanation’ in the Explanation pattern” (Hobbs and Redeker 1997: 17) given in the formula above. They rewrite and explicitly customise the formula as follows in equation 2.3: cause(e2 , e1 ) ⊃ Explanation(e1 , e2 )

(2.3)

where ‘explanation’ is now explicitly mentioned (Hobbs and Redeker 1997: 17). But this rule is inalienably accompanied by the second rule: Explanation(e1 , e2 , e) ⊃ CoRel(e1 , e2 , e)

(2.4)

which then definitely explains that the segments e1 and e2 are connected with a coherence relation in the environment e. Coherence relations are commonly regarded as “the cornerstone of comprehension” (Graesser, McNamara and Louwerse 2003: 82). This is a convenient depiction for the general process of text interpretation operating when recognising the connections between propositions in a text. This is, as Asher and Lascarides (2003) point out, a complex reasoning process for computing an appropriate discourse structure and therefore the meaning of the text. In the example mentioned above, the process of defining the Explanation-relation operates by the principle of abduction, as highlighted by Hobbs and Redeker (1997) and also by Asher and Lascarides (2003). This principle is commonly assumed by most of the accounts that deal with the topic of coherence relations—and it has been assumed in equal manner for film interpretation (see sections 1.1 and 1.4). More generally, the accounts in discourse semantics, as well as those pursuing film analysis from this perspective, talk about inferring relations between discourse segments and defining them by means of abductive reasoning. For verbal discourse, this process of interpreting is based upon the general possibility to simply assume the coherent correlation between propositions in a text that exist

i i

i

42 The Logic of Film Discourse Interpretation

for example as a result of the Gricean cooperation principle in terms of rational speech acts (see Hartung 2006: 22).7 The recipient can thus infer the implicit context wherein the proposition is embedded while referring to his/her knowledge and coordinating the particular knowledge facts with others that together demonstrate coherence. The same principles are assumed here for filmic discourse. The framework provided in this book is based primarily on the SDRT of Asher and Lascarides (mainly in their work from 2003). The simultaneous integration of discourse semantics and artificial intelligence approaches into one account allows a concrete distinction between the construction of the logical form of a discourse on the one hand and the logic of interpreting discourse on the other. The former is the logic of information content that provides rhetorical relations and the syntax and semantics of a formal language to represent the logical form of discourse. Whereas this logic does not model the above mentioned interaction of semantics and pragmatics, the latter, the logic of information packaging and discourse update, addresses the inference process of the rhetorical relations by using pragmatic information that enriches the compositional semantics of those relations. “We’re interested in the relationship between an underspecified logical form and its pragmatically preferred specifications, rather than simply all of its possible specifications. Discourse update (or SDRT update) incorporates the pragmatically preferred specifications and preferred interpretations. It thus provides an alternative model of discourse interpretation [. . . ].” (Asher and Lascarides 2003: 179) This is of particular importance for the inference process since the logic of information packaging provides the nonmonotonic “glue logic” that makes available the default axioms necessary for inferring relations. The reasoning process is seen as gluing together the logical forms of clauses in order to build the logical form for discourse (see Asher and Lascarides 2003: 184). The glue language for defining default axioms builds on a language of underspecified logical forms (ULF), extending its vocabulary by incorporating the logical operator > that is used to present defaults and that can be read as “if. . . then normally. . . ” These default axioms then follow the schema in equation 2.5, given by Asher and Lascarides (2003: 199). (?(α, β, λ) ∧ some stuff) > R(α, β, λ)

(2.5)

7 This principle takes communication as a social act that is goal-oriented and that operates on behalf of the intended purpose of the situation: “Make your contribution such as required, at the stage at which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged” (Grice 1975: 45).

i i

i

i

i

Film Discourse Interpretation 43

?(α, β, λ) indicates an underspecified rhetorical relation between the segments labelled by α and β. The discourse structure context is labelled by λ. In combination with some stuff, this relation then becomes specified as the relation R holding between the two segments α and β within the context λ. An important development in the account of Asher and Lascarides is that the rhetorical relations relate labels and not propositions. These labels are part of the discourse’s logical form itself (see Asher and Lascarides 2003: 136) and their use enables similar tagging of different semantic content even though they are expressed in the same linguistic form. Labels can thus tag content of different discourses, so that they are mutually satisfiable. Asher and Lascarides also emphasise that “[b]y using labels we can erect a ‘porous fence’ between discourse content and the reasoning process for constructing representations of that content” (Asher and Lascarides 2003: 137). In the schema above, some stuff represents conditions that have to hold in the antecedent in order to state that there is evidence for the particular relation, specified as R. “Proving coherence relations relies on axioms that typically have some domain relationships in their antecedents and a particular coherence relation in the consequent. And the problem of choosing which axiom to abduce on is guided by the weights on the antecedents of the axioms.”(Asher and Lascarides 2003: 96) To consider an example, the axiom for inferring the Explanation-relation is then expressed as given in equation 2.6. (?(α, β, λ) ∧ causeD (β, α)) > Explanation(α, β, λ)

(2.6)

This axiom fires when a causal relationship holds between the events that are described in the segments labelled by β and α. Evidence for the relationship is stated by the antecedent condition causeD (β, α). If this is the case, then the rhetorical connection is normally an Explanation-relation. Asher and Lascarides highlight that relations in discourse are not always linguistically marked via cue phrases or other concrete cues in the text. Thus, the process of discourse update constructing logical forms has to operate together with knowledge of lexical semantics, domain knowledge and cognitive states that, when combined, reveal specific clues about the discourse relations (see Asher and Lascarides 2003: 110). The axiom schemata are seen as default rules that determine the relevant information from the various knowledge sources and transfer them into the glue language to express conditions such as causeD (β, α). The underlying logic for this transfer is a nonmonotonic logic that supports the general principle of defeasible reasoning. This principle

i i

i

44 The Logic of Film Discourse Interpretation

consists of consequence relations which are partly not conclusive and do not guarantee truth (Pollock 1974; Colburn 1991). Instead, these relations provide a prima facie reason which is “a logical reason that is defeasible. In other words, a prima facie reason is a reason that by itself would be a good reason for believing something, and would ensure justification, but may cease to be a good reason when taken together with some additional beliefs” (Pollock 1974: 39f.). Whereas the logical validity of conclusions in general is a monotonic consequence relation which is expressed by the operator `, the operator |∼ is here used for a defeasible consequence relation which validates several inference patterns that are useful for inferring discourse relations. Asher and Lascarides (2003: 190f.) list the following. Defeasible Modus Ponens (DMP) In a theory T all antecedents are verified by the premises and the default laws all have consequents that are consistent with T and each other. The consequents of the defaults are then all valid inferences. In contrast to the traditional form of modus ponens in deductive logic expressed as if A then B; A, therefore B ( = A, A → B ` B), DMP involves the defeasible argumentation that is symbolised by the operator > in equation 2.7. A, A > B |∼ B

(2.7)

A and B here are variables which stand for statements that are, for example, expressed by propositions. The variables may be filled by specific sentences (in verbal discourse) to produce a concrete argument. In the argument form in equation 2.7, the variables together assign default rules. This then means that given the situation specified on the left of the |∼ operator, we can infer the situation on the right. It thus means that if we know that A holds and that A normally entails B, then we know that B holds normally: if A then normally B; A, therefore normally B. The same is true for the negated version in equation 2.8. A, A > B, ¬B |6∼ B

(2.8)

The Specificity Principle Eickmann explains this very general pattern in the following: “If the information available licenses different but mutually inconsistent inferences of rhetorical relations, then a specificity principle is applied, that states that more specific information overrules inferences licensed by less specific information.” (Eickmann 2003: 24) This pattern is also known as the Penguin Principle, where the inference that a penguin as a bird can fly is blocked by virtue of the more specific

i i

i

i

i

Film Discourse Interpretation 45

information that penguins are not normal birds. Asher and Lascarides describe this inference process as follows: “We’ll show that because of the axiom Specificity in G [the monotonic logic], the nonmonotonoc logic resolves the conflict in favour of the more specific conflicting default, if there is one (that is, the default rule whose antecedent entails the antecedents of the other default rules).” (Asher and Lascarides, 2003: 190) This is expressed logically in equation 2.9. If ` A → C then A > ¬B, C > B, A |∼ ¬B

(2.9)

This formula can be read as follows: If we know that A entails C, then follows that A entails normally not B, but that C normally entails B and A, then it follows that A normally entails not B. Thus, the monotonic inference A → C leads to the conclusion that penguins are not normal birds, and this again produces the nonmonotonic consequence A > ¬B that a penguin does not fly, although birds normally fly (C > B). The Nixon Diamond This principle goes back to an illustrative example as well (see Eickmann 2003: 24). It demonstrates that inferences are blocked when conflicting axioms are not distinct in their specificity so that further information that would be necessary for a resolution is missing to find a conclusion. A > B, C > ¬B, C, A |6∼ B (2.10) This formula can be read as follows: If we know that A normally entails B and that C normally entails not B, but that both C and A hold, then it does not follow that B holds. In the second formula in equation 2.11, B as the consequent is negated. A > B, C > ¬B, C, A |6∼ ¬B

(2.11)

It can be read: If we know that A normally entails B and that C normally entails not B, but that both C and A hold, then it does not follow that not B holds. In the formulae, then, both inferences, A > B and C > ¬B, do not provide information which is specific enough to overwrite the respectively conflicting axiom. Consequently, in both cases, there is no successful conclusion, which is marked by |6∼ . Eickmann describes the traditional example for this principle as follows: “If we know that Nixon is Republican, we could normally infer that Nixon is no pacifist (as Republicans don’t tend towards

i i

i

46 The Logic of Film Discourse Interpretation

pacifism), but if we know that Nixon is Quaker, we normally could infer that he is pacifist. When we know that Nixon is both Republican and Quaker, we have to withhold, in absence of further inormation, any conclusion if he is pacifist or not.” (Eickmann 2003: 24) Further patterns are for example Robustness and the Weak Deduction Principle (see Asher and Lascarides 2003: 191 and Lascarides and Asher 1991 for further discussion). With the help of these patterns, default rules for inferring discourse relations can be established. This has been done for various relations, for example for Narration, Elaboration, Explanation, Result, Background, Contrast and Parallel. Asher and Lascarides describe these relations as typically occurring in narrative or expository text. They can be defined by virtue of their truth conditional effects (see Asher and Lascarides 2003: 145). In addition, they all follow the Satisfaction Schema for Veridical Rhetorical Relations that is the initial point for defining the semantics of the relations. Veridicality is defined in equation 2.12. R(α, β) ⇒ (Kα ∧ Kβ )

(2.12)

This means that if the relation R holds between the discourse segments α and β, then the interpretation of the Segmented Discourse Representation Structure (SDRS) Kα must be available to the interpretation of Kβ . Thus, the dynamic conjunction of the two structures, which is expressed by the operator ∧, produces a value of true; the relation entails the two propositions (cf. Lascarides and Asher 2007: 20). Hence, the Satisfaction Schema “does carry forward the values of any previously introduced discourse referent to the output contexts [. . . ]” (Asher and Lascarides 2003: 158). This input is then combined with the more specific axiom schemata that restrict the interpretation of the relations. For each relation, an axiom schema is then defined formally on the basis of the general form already mentioned above. (?(α, β, λ) ∧ some stuff) > R(α, β, λ)

(2.13)

The logic of information content provides meaning postulates that capture the particular semantic effects of the discourse relations and constrain their interpretation. They have the form of equation 2.14 (cf. Asher and Lascarides 2003: 159). φR(α,β) ⇒ conditions(α, β)

(2.14)

The conditions are conditions on the SDRS Kα and Kβ or on discourse referents introduced by them. These meaning postulates are translated

i i

i

i

i

Film Discourse Interpretation 47

into the glue language, “so as to achieve a shallow form of the relations’ semantics [. . . ]” (Asher and Lascarides 2003: 199). Let us now look at some examples of these axiom schemata in use to show how they work. These examples include the translated meaning postulates and the information transferred from the language of information content. All logical formulae are taken from Asher and Lascarides (2003: § 4.8 and § 5.6). Narration The Narration-relation holds if the first mentioned event eα “occasions” the second mentioned event eβ , i.e.: (?(α, β, λ) ∧ occasion(α, β)) > N arration(α, β, λ)

(2.15)

The concept of occasion here describes “a ‘natural-event-sequence’ such that events of the sort described by α lead to events of the sort described by β” (Asher and Lascarides 2003: 200). With this, the following meaning postulate for Narration, a spatiotemporal consequence, is assumed: φN arration(α,β) ⇒ overlap(prestate(eβ ), Advβ (poststate(eα )))

(2.16)

Considering the following example allows a concrete description of the inference drawn by virtue of this axiom schema: Max fell. John helped him up. Max’s falling and John’s helping are related in that the first event occasions the second. Falling and helping thus correspond to an axiom schema in the glue logic that is formulated by domain knowledge (see Asher and Lascarides 2003: 201 for further discussion). Since the consequent is consistent with the premises, the inference pattern DMP (A, A > B |∼ B) shown above in equation 2.7 operates as follows: (π1 , π2 ), (π1 , π2 ) > occasion(π1 , π2 ) |∼ N arration(π1 , π2 , π0 )

(2.17)

The DMP here offers occasion(π1 , π2 ) which verifies the antecedent needed to infer the relation (see equation 2.15). N arration(π1 , π2 , π0 ) is then formally established. Explanation quence:

The meaning postulate for Explanation is a temporal conseφExplanation(α, β) ⇒ (¬eα ≺ eβ )

(2.18)

φExplanation(α, β) ⇒ (event(eβ ) ⇒ eβ ≺ eα )

(2.19)

The event eβ thus has to precede (described by the operator ≺) the event eα . Only then can it give a reason for the first event. The semantics of Explanation thus also include the role of cause. The discourse has to give evidence that a causal relationship holds. “This evidence may be ‘justified’ on the grounds of domain knowledge, lexical

i i

i

48 The Logic of Film Discourse Interpretation

semantics or some other knowledge source” (Asher and Lascarides 2003: 204). (?(α, β, λ) ∧ causeD (β, α)) > Explanation(α, β) (2.20) The following propositions satisfy an Explanation-relation: Max fell. John pushed him. It can be inferred that John’s pushing precedes Max’s falling. Thus, the temporal consequence given in the second formula above holds. Asher and Lascarides give many more examples for inferring discourse relations by exploring the logical interactions between the axioms and their constraints on the interpretation (see Asher and Lascarides 2003: 209ff.). The process of constructing the logical form of the discourse by using inferences is then formulated within the updateSDRT procedure: “This defines how the SDRS for the discourse context plus the (S)DRS for the new information are integrated into a new SDRS. Decisions about how to glue the SDRSs together depend on inferences performed in the glue logic.” (Asher and Lascarides 2003: 212) By means of the inventory of discourse relations, each discourse structure Kβ , i.e., the SDRS associated with the new information expressed in β, is related to an element of the previous context σ to generate a new context σ 0 . This is done by adding supplemental constraints that result from the specific correlation between new and old information to an ULF that represents the present context. An important update mechanism is the principle of Maximise Discourse Coherence (MDC), which is responsible for specifying the pragmatically preferred discourse structures. An SRDS is pragmatically preferred if its coherence is stronger than that of another SDRS—which means that more rhetorical connections are given. The MDC-principle then constrains the update procedure in order to maintain a partial order on discourse structures. This is achieved by comparing the strength of the discourse relations and introducing a partial order on them that is relative to the content of the context. The process of discourse update is guided by the MDC-principle to build a logical form of the discourse that “maximises the number and quality of rhetorical connections, minimises the number of underspecifications, minimises the complexity of the structure, and minimises the number of semantic inconsistencies and pragmatic clashes” (Asher and Lascarides 2003: 247). The most distinctive feature in the account of Asher and Lascarides overall is definitely the use of diverse, separate logics that are combined in a general architecture: “Each logic is designed to do a specific task: e.g., there’s a logic in which you construct a representation of what is said, another

i i

i

i

i

Film Discourse Interpretation 49

logic in which you evaluate (the consequences of) that representation, a logic in which you reason about lexical polysemy, a logic in which you reason about another person’s cognitive state on the evidence of his utterances and the assumption that he’s rational and cooperative, and so on.” (Asher and Lascarides 2003: 430)

Figure 2.2: The logics of discourse interpretation adapted from Asher and Lascarides (2003: 431) Adapted from Asher and Lascarides, figure 2.2 sets out the logics that cooperate in the discourse interpretation process described above. The depicted patterns for inferring discourse relations offer accurate operationalisations for the exploitation of textual coherence. They therefore provide inventories of discourse relations and adequate operating methods, such as the principles of discourse update and MDC in SDRT or the more general principle of abduction as described by Hobbs, to depict the involved inference processes. In the following, one of the larger examples for verbal discourse given by Asher and Lascarides (2003: 223) will be summarised in order to give a better overview of the possibilities to describe discourse interpretation made available by this logical framework. Consider the following sequence of utterances:

i i

i

50 The Logic of Film Discourse Interpretation

π1 Max experienced a lovely evening last night. π2 He had a fantastic meal. π3 He ate salmon. π4 He devoured lots of cheese. π5 He won a dancing competition. This small discourse talks about Max’s evening in that it elaborates on the meal and the various courses Max had. π2 , having the fantastic meal, is thus included in having a lovely evening and therefore can be interpreted as giving more information about the first proposition, i.e., as an Elaboration of the preceding content. The propositions following (π3 and π4 ) give again more information about this meal and are therefore also related to π2 by an Elaboration-relation. The two discourse segments, however, are themselves related by a Narration-relation, since eating salmon and devouring cheese are not causally related to each other. The two events depict a simple natural event-sequence. However, the discourse then switches the topic to the dancing competition. This new topic is introduced by the fact that the new information in π5 cannot be attached to the prior clause in π4 , but rather to π2 , for example. This phenomenon is generally called discourse popping because of the leap that the recipient has to make from the last utterance back to the second, for example. Asher and Lascarides emphasise the difficulty to relate the last event (π5 ) to the discourse: “Attaching π5 was an example of a discourse ‘pop’; it didn’t attach to the previous clause but rather to an earlier one. This discourse pop results from choosing an update SDRS that maximises coherence. Before leaving this example, we should note that because of Constraints on Update [. . . ], once π5 is attached to π2 it cannot then also attach to π4 (even though before this attachment π4 was available). This predicts that π5 couldn’t have an anaphor that depended on elements in π4 .” (Asher and Lascarides 2003: 229f.) It is thus a dynamic interpretation process which is necessary to attach the last information given in the discourse to the preceding utterances. This interpretation process is based on logical reasoning about the interdependencies between the utterances and the conditions that must work for the interpretation of some relations. The meaning postulates and default axioms given for each discourse relation, as in parts described above, give clear constraints on how the relations between the sentences have to be inferred. According to these conditions, it has not been possible to find a relation between π4 and π5 . Due to the MDC-principle, however, “attempting to attach π5 to just π1 and π2 yields something more coherent according to our assumptions, in that the update won’t include these (rhetorical) underspecifications” (Asher and Lascarides 2003: 228).

i i

i

i

i

Film Discourse Interpretation 51

The formal framework of SDRT and the notation style introduced by Asher and Lascarides (2003) can now make visible this dynamic reasoning process about the best and maximally coherent interpretation, in that it formalises the process of inferring the logical relations between the sentences and their propositions. The result is an SDRS which can be set out as given in table 2.4. π0 π1 , π 00 π1 : Kπ1 , Elaboration(π1 , π 00 ) π2 , π5 , π 0 π2 : Kπ2 , π5 : Kπ5 N arration(π2 , π5 ) Elaboration(π2 , π 0 )

π0 : π 00 :

π3 , π4 π : Kπ3 , π4 : Kπ4 π0 : 3 N arration(π3 , π4 )

Table 2.4: Segmented Discourse Representation Structure of the example discourse adapted from Asher and Lascarides (2003: 223) This depiction makes visible the semantic representation of the overall SDRS of the discourse as the formalised structure π0 , which is displayed as a large box consisting of different subordinated structures, each depicted as a further single box. The first line of each box describes the discourse segments labelled with πn which configure the structure, as for example π3 and π4 in the small box representing π 0 . These particular discourse segments can be interpreted as logical forms of the discourse, DRS, which are described formally as Kπn in the second line of the box. The lines following then list the discourse relations holding between the segments. Within the large box representing π0 as the overall structure of the discourse, there is a subordinated structure labelled π 00 , which again includes the subordinated structure π 0 . Each (subordinated) structure consists of various discourse relations holding between the sentences. The two subordinations are established by the labels π 00 and π 0 due to the subordinating function an Elaboration-relation generally has in verbal discourse and also holds in the example discourse above. Asher and Lascarides describe that

i i

i

52 The Logic of Film Discourse Interpretation

this relationship normally holds if there is a change in the granularity of description (Asher and Lascarides 2003: 204f.). The list of things Max had is thus a subordinated enumeration which refers to the proposition of having a fantastic meal. The same applies to π2 and π5 . The events of having a fantastic meal and winning a dancing competition both give further information about the fact that Max experienced a lovely evening. They are therefore subordinated to π1 . The two discourse segments themselves are related by a Narration-relation, since they uphold a natural event-sequence. The description of the logical form in this SDRS makes it possible to outline dependencies between the sentences and therefore to demonstrate the overall coherence of this short discourse. The SDRS in table 2.4 thus makes visible that π5 is actually related to π2 in the subordinated structure π 00 and that it cannot be related to π4 . Consequently, the use of this framework enables the detailed and, in particular, formal description of the inference processes operating in reasoning about the best interpretation of a discourse. It is thus an appropriate basis for the application of its analysis to multimodal texts such as film. Until now, explicitly formal accounts dealing with the notion of rhetorical relations or such a formal description of the inference processes have been quite rarely used for this examination. There are some examinations using the notion of coherence relations in order to describe a ‘cinematic language’ for the construction of narrative in different media and with particular regard to hypertext (cf. Mancini 2000, 2005). Research in these fields analyses cinematic rhetorical patterns, which are assumed to reoccur in hypertexts, in comparison to patterns in verbal discourse with the help of a set of ‘Cognitive Coherence Relations (CCR)’. However, the examination of these patterns does not use logical formalisms either, and focuses attention on hypertextual criteria that hardly ever shed light on how filmic discourse is interpreted within dynamic processing. Furthermore, some small applications of the DRT-formalisms to film can be found in the works of Jasinskaja (2010), Jasinskaja and Rossdeutscher (2009) and von Stutterheim, Halm and Carroll (2012). Within their research on human discourse production of narrative, they also try to build DRTbased narrative generation models for filmic discourse, in particular for the silent animation film Quest. Jasinskaja (2010), for instance, uses a socalled ‘DRT encoded knowledge base’ as the starting point of examination in order to represent the content of the film and to analyse typical retellings. These accounts, however, do not include the analysis of rhetorical relations in their examination. The very application of the descriptions given in the account of DRT is not sufficient to elucidate the concrete properties which manifest textuality and therefore the construction of meaning. Analysing in particular the cross-modal patterns which represent filmic specificity as kinds of coherence relations is intended to produce the struc-

i i

i

i

i

Film Discourse Interpretation 53

ture of the filmic discourse and to demonstrate its meaning potential. Mainly the interrelationships between the relations themselves and the depicted structure of the filmic discourse are therefore the starting point for the application of formal accounts developed in this book. As Asher and Lascarides (2003) point out, discourse relations play a distinctive role for the interpretation of discourse, since their use expands the dimension of traditional dynamic semantics with mechanisms for modelling several textual phenomena that cannot be described sufficiently with other accounts. In particular, the immediate association with the properties of structurality and constituency is profitable for a detailed analysis of the interrelationships in multimodal discourse. The inclusion of different interpretive strategies that assign truth conditions to discourse and build on the notion of defeasible reasoning puts an emphasis on the interaction between different knowledge sources that are all responsible for an effective interpretation process, while at the same time restricting this process. On the basis of the truth conditional semantic interpretation, a choice of discourse relations is given which differentiates the particular relations. Meaning postulates constrain on the one hand the definitions of the relations, so that a concrete set of them is available. On the other hand, they also constrain the models in which discourse structures are satisfied. Since the framework goes beyond the limits of compositional and lexical semantics, it is highly sensitive to the discourse context in general and allows the examination of various phenomena in this context. It is comprehensive enough to prove that “every bit of information is rhetorically connected to some other bit of information” (Asher and Lascarides 2003: 441) and the discourse is coherent. SDRT is therefore taken as a basis for the development of a framework for the analysis of filmic discourse. This framework will now be described in further detail in the following sections. 2.2.3

The Logical Form of Discourse Segments

Although the transcriptions which have been used for the example analysis in section 2.1 enable the outlining of the different modalities in a particular film extract, they do not explain their concrete interaction in terms of their intersemiosis. Since it is this interaction which forms the determining meaning potential of the respective context, it is necessary to have a further concrete framework of description which then finally produces the logical form of the discourse segments that has been introduced. We saw them in the so-called eventuality in section 2.1. Therefore, and according to the examination of logical forms of verbal discourse shown above, this book modifies the DRS notation style provided by Asher and Lascarides (2003) for presenting FDRS. These FDRS outline the individual

i i

i

54 The Logic of Film Discourse Interpretation

events within the text that can be segmented by the recipient, as shown for the first two shots in section 2.1. Whereas the still images in the transcription are rather abstract and definitely lack the dynamicity of the depicted event (cf. Kress and van Leeuwen 1996: 261), the description of the logical forms of the discourse segments will take this dynamicity of the moving image and its perception into closer consideration. According to Magliano, Miller and Zwaan (2001) and Zacks, Speer and Reynolds (2009), film viewers segment the ongoing activity of the filmic text into so-called meaningful events. As key concepts of perception, these events are defined as “segment[s] of time at a given location that [are] conceived by an observer to have a beginning and an end” (Zacks, Speer, Swallow, Braver and Reynolds 2007: 273). This definition covers what has been described as the eventualities of the shots in the example analysis of the short film WORDS above. The propositional content of the individual processes in the shots was inferred and formulated on the basis of associations. Zacks et al. point out that these associations can be found due to “a number of physical visual and auditory cues that are not available from narrative language, including quantitative information about the movement of actors, objects, changes in object contact relations, facial expressions, and environmental sounds” (Zacks et al. 2009: 318). In a similar manner, Kress and van Leeuwen underline that it is mainly the aspect of movement in filmic images which leads to subtle differences in the construal of the narrative processes (see section 2.2.1 for a detailed description of these processes). They emphasise that dynamicity is often constructed through one of the submodalities of the video track, for example camera movement as well as cuts between shots, and that these factors “need[ ] to be added to the list of means of expression that can cue modality. Like visual detail, background, depth, light and shade, colour, etc., movement can be represented with different degrees of realism or abstraction and hence play a role in modality judgments.” (Kress and van Leeuwen 1996: 264) In an analysis of the documentary Industrial Britain (1931), van Leeuwen exemplifies that a sequence of three shots can depict only one “process which more typically would be realised by a downwards camera movement, but which is here realised by a ‘montage’ of three shots. The technique is based on Eisenstein’s filmmaking methods, and foregrounds the process, ‘shocking’ us into awareness of it by means of the abrupt jerkiness of the cuts.” (van Leeuwen 1991: 96)

i i

i

i

i

Film Discourse Interpretation 55

However, according to Zacks and Magliano, cuts are often not significant for the identification of event boundaries: “Notably, cuts in and of themselves were not associated with event boundaries—if a cut merely changed the camera viewpoint within a scene, it had no discernable effect on coarsegrained segmentation [of events] and only a small effect on fine-grained segmentation.” (Zacks and Magliano 2011: 6) The cut in the following sequence of shots (see table 2.5), for example, would normally preserve a certain continuity of action, since both shots display the same activity of playing the trumpet. A conceptual change cannot be found. However, the second shot in this example is a zoomed shot audio description track

music

8.

a man sounding the trumpet

trumpet play

calm piano and guitar music

9.

close-up of his mouth blowing into the trumpet

sound of blowing heavily

calm piano and guitar music

#

shot

Table 2.5: Extract of the transcription of the short film WORDS (Everynone, 2010) close-up of the man’s mouth and turns out to have another function to that of the first one, namely depicting another eventuality that can be inferred during the discourse’s unfolding (see section 2.1). The semantic content of the two shots thus has to be differentiated into two eventualities that describe two different events, which are separated from each other because of the contextual development. The change of the camera perspective represents a point “when features in a narrated situation are changing” (Zacks et al. 2009: 318) and thus gives reason for this segmentation. In this case, this feature is directly motivated by the following context which requires this change to enable associations with another predicate (‘blow’). Normally, according to Magliano et al. (2001), changes in time and movement as well as in region are decisive for the segmentation of events in a narrative film. These events can often be found on the scale of the shot and above (cf. Magliano et al. 2001), but in a few cases also within

i i

i

56 The Logic of Film Discourse Interpretation

shots (cf. Zacks et al. 2009). In the example of WORDS, every shot has been defined as an individual event whose semantic content has been described as an eventuality that then, in a second step, has been related to another eventuality by a film discourse relation. The analysis has shown that each shot depicts a different action or at least a different perspective on some event, and so has been separated out as a single eventuality as the abstract formulation and summarisation of the event. Nevertheless, the shot as the minimal unit of the transcription can by no means generally be seen as an event boundary. This gives evidence for the recipient’s separation of processes in the filmic text. A cut, which separates two shots, simply depicts the physical break made by the camera, but is not able to shed light on how the recipient separates the ongoing activity into individual events. The minimal semantic unit of the framework provided in this book therefore is the eventuality itself. This will be taken as the starting point of examination, which then aims at describing the multimodal composition of the individual activities and processes in filmic discourse. Accordingly, similarly to the SDRS of Asher and Lascarides we described above, a box will be used to illustrate each segmented event and its formulation as an eventuality as an FDRS. The first line of this box represents the eventuality and its position in the discourse structure, which is marked with πn . The inferred verbalisation of this eventuality summarises the depicted event. In the example in table 2.6, which refers to shot 7 in the short film WORDS, this eventuality is play. eπ10 = play [v] unknown actor (m) [a] “Play ball!” (n) [a] calm piano/guitar music (o) m, n |∼ play (eπ10 ) Table 2.6: Example notation of the eventuality eπ10 The different lines within the second part of the box are the discourse referents which can be inferred from the different contributions of the modalities that are generally distinguished as visual, [v], and auditory, [a]. According to requirements and due to the respective modalities and submodalities used in the context, these levels may be described in more detail during analysis as needed. In addition, every discourse referent that can be interpreted from the interplay of the modalities is tagged with a variable such as (m) or (n), which makes it possible to outline dependencies between them. The description of the eventuality itself in bold symbolises how this eventuality is inferred. In the example, the discourse referents (m) and

i i

i

i

i

Film Discourse Interpretation 57

(n) enable the association with the predicate ‘play’ because it is explicitly given in the voice track that is uttered by the unknown actor. Since this actor did not appear in one of the shots before, he depicts an important change in the description and therefore gives evidence for the segmentation of an event. Calm music in the background accompanies this event, but also the whole extract. It is therefore described in each logical form, which can be formulated for the discourse segments of this film, and creates a certain continuity without contributing itself to the determination of ‘play’. The event itself, however, is distinguished from the others because of the narrative process which can be separated with the help of the referents m and n. The last line of the box therefore depicts the eventuality as a defeasible consequence which is described by the logical operator |∼ . This formulation indicates that it is these two discourse referents that allow the inference of play in this context. This style of description makes it possible to illustrate different functions of the modalities and the emerging discourse referents. Whereas (m) and (n) here directly influence the formulation of the eventuality, the third referent, the sound track on the auditory level, supports the continuity of the whole extract and cannot similarly be separated from the other events. In this way, the dynamicity of the filmic discourse is clearly maintained. Depending on the respective event, which will be described within the logical form, it is then also possible to describe in more detail temporal and spatial circumstances which may be important for the analysis of the unfolding discourse structure. More examples of this will be given in chapter 3. The summarisation of the events as eventualities is an abstract description of the semantic content, which corresponds well to what Zacks and Magliano (2011) call a propositional representation of the text content that the viewers exhibit during perception. Research in discourse processing generally assumes that the construction of this representation is carried out on several levels (cf. van Dijk and Kintsch 1978; van Dijk and Kintsch 1983) from which two basic ones are focused on in the following: “Readers construct a representation of the surface structure of a text, which captures the particular words and phrasing, but this representation is fleeting. In film, this would be analogous to a representation that included visual information about the details of characters’ clothing and props. Readers also construct a representation of explicit text content, which is referred to as the textbase. This representation contains a network of propositions that is incrementally constructed during reading and represents the explicit ideas contained in a text. Film viewers likely construct a similar representation as well [. . . ].” (Zacks and Magliano 2011: 9)

i i

i

58 The Logic of Film Discourse Interpretation

It is this similar representation which is described by the logical forms of the discourse segments. They depict on the one hand visual and auditory information about details of the filmic content, which are described as discourse referents and constitute the surface structure. The textbase, on the other hand, is not constructed out of explicit text content, which, in filmic text, is mostly not at hand, but is instead inferred as explicitly formulated eventualities. These have a reduced form that captures the semantic meaning of the discourse segment but which does not feature details from the surface structure. The propositional formulation thus refers to an abstract level of interpretation which is always defeasible. As Bateman and Schmidt (2011) point out, this “depend[s] on the particular observers involved and their own respective bodies of knowledge [. . . ] and refer[s] to interpretations, allocations of structure, units and paradigmatic relations as being relative to specific sets of observers rather than as properties of the artefacts analysed” (Bateman and Schmidt 2011: 149). The eventuality is thus seen as the result of what is normally inferred from the narrative processes depicted in the film by “some set of observers” (cf. Bateman and Schmidt 2011: 149). In the example of WORDS, the first predicate ‘play’ is fairly evident in that it is made explicit in written form in the first shot of the film. It is later again verbalised in the voice track of the coach, shouting “Play ball!”. The second eventuality, ‘blow’, in contrast, has to be inferred only from the processes visible in the shots without any direct formulation within the filmic text. It is assumed that this is the normal case for film interpretation, since the narrative processes in film are only rarely accompanied by a voice track that explicitly describes what is happening in the image. The eventualities therefore have to be formulated as an abstract, but concise description of the semantic content, which can conceivably be made by another predicate. It is thus often a question of granularity which identifies the predicate that summarises the semantic content most adequately. However, similar to the comprehension of verbal texts, it is assumed that the construction of the semantic representation is guided by both bottom-up and top-down processes. Van Dijk and Kintsch (1983) highlight that in verbal discourse processing the context normally affects the identification of words, which then are taken for the formulation of the predicate because of pathways which are activated during perception. At the same time, these words are hypothesised by the recipient in order to benefit or interfere with the ongoing perception (van Dijk and Kintsch 1983: 23; cf. Stanovich and West 1981). Assuming that film comprehension processes equally construct a mental representation of the content (cf. Baggett 1979, Tibus 2008), the inference and formulation of the eventualities in this book is taken to be the abstract verbalisation which sums up what is mentally constructed by the recipient within the event model.

i i

i

i

i

Film Discourse Interpretation 59

Following the concept of the situation model in discourse comprehension according to Kintsch (1988) and van Dijk and Kintsch (1983), this event model “provides an index of how story events are related along a number of dimensions, such as agents and objects, temporality, spatiality, causality, and intentionality” (Zacks and Magliano 2011: 9). It represents exactly what is described as the unfolding discourse structure in this book and features not only the semantic representations of the filmic content, but includes as well the inferences which are needed for relating these representations to each other. “These inferences are conceptualized as being part of the event model because the[y] go beyond the propositions that are derived to reflect the explicitly seen events. [. . . ] viewers must infer and re-present the causal relationships between events within a scene and across scenes, the latter of which provide a basis for understanding the plot structure.” (Zacks and Magliano 2011: 9) The sequence of eventualities can be seen as a first, conceptual depiction of this plot or—as decribed in SDRT—its topic structure, which not only gives a view of how film is generally comprehended, but also refers to questions concerning the thematic structure of filmic texts or the examination of filmic genre, for example. These questions go beyond the linguistic style of analysis provided in this book, but indicate how the interpretation of films can be based on research which focuses on meaning-making processes (see chapter 5 for more information). The discourse analytical perspective on how these processes can be described by the analysis of the relations holding between the logical forms of the discourse segments will be pursued further in the following section, which will define these relationships as film discourse relations. 2.2.4

The Experimental Set of Film Discourse Relations

For the extended analyses to be carried out in chapter 3, the following experimental set of film discourse relations has been selected: Narration Elaboration Explanation Result Background Parallel Contrast This set represents a number of discourse relations that have been described for verbal discourse in the works of Asher and Lascarides (2003).

i i

i

60 The Logic of Film Discourse Interpretation

They can be defined to a certain extent by virtue of the truth conditional effects they have on the discourse’s meaning. Their default axioms as given for verbal discourse will be modified in the following to apply them to filmic discourse. Concrete analysis will then outline if and to what extent the axioms have to be corrected for film and if other relations have to be added to the paradigm. As Asher and Lascarides (2003) point out, the choice of relation is motivated by semantic interpretation: “The[ ] logical forms convey a content that goes beyond compositional and lexical semantics [. . . ], but it is a much more restricted notion of meaning than the kind pursued by Searle and much of the AI research.” (Asher and Lascarides 2003: 145) This restricted notion of meaning accounts as well for the limited number of relations—for example in comparison to the taxonomies provided by Mann and Thompson (1987) or Maier and Hovy (1993). The detailed differences between several relations with a very similar meaning (such as the Contrast-relation in Rhetorical Structure Theory; see Asher and Lascarides 2003: 145) provide insight into their different communicative purposes, but do not describe the truth conditions of the relations which in turn influence the meaning potential. It is therefore necessary in a first step to outline the truth conditions establishing the filmic meaning potential with the help of discourse relations. After having defined precise axioms to indicate these conditions and effects, concrete analysis may afford further differentiation of underlying particular intentions. Consequently, the set of discourse relations will later be classified on a metafunctional level which also captures these intentions from a functional point of view. Asher and Lascarides assembled one of the first—but by no means exhaustive— categorisations in their description of structural relations (cf. Asher and Lascarides 2003: 146). These relations affect the discourse’s meaning on the level of the textual metafunction. For a comprehensive analysis of the communicative purposes underlying the relations, however, it is necessary to outline their concrete syntagmatic articulation in a particular discourse structure. As remarked above, according to Asher and Lascarides (2003), the relations are normally part of a narrative or expository text. They are therefore particularly qualified to describe the meaning-making potential of filmic discourse since this is often analogously a similarly qualified text. Asher and Lascarides also provide discourse relations which account for the cognitive states of participants in a dialogue situation on the basis of the effects their cognition has on discourse interpretation (cf. Asher and Lascarides 2003: 293ff.). However, these relations are less significant for the

i i

i

i

i

Film Discourse Interpretation 61

examination of film discourse interpretation, since film is not typically a dialogic situation. Based on the notation used by Asher and Lascarides (2003) (cf. section 2.2.2), the form given in equation 2.21 is chosen to describe the selected relations and their appropriate meaning postulates, depicted as conditions: φR(α,β) ⇒ conditions(α, β) (2.21) In this formula, φR(α,β) indicates the order in which the axioms are applied in the dynamic logic so as to satisfy the discourse structures corresponding to α and β, which are written as Kα and Kβ respectively. On the basis of the Satisfaction Schema for Veridical Relations given in equation 2.12 in section 2.2.2, “the discourse referents introduced in Kα and Kβ must be bound by the (input) context in order for φR(α,β) to be interpretable” (Asher and Lascarides 2003: 159). As the default axiom schema for inferring some particular relation, the form given in equation 2.22 will be used. It resembles the schema provided by Asher and Lascarides, introduced in section 2.2.2 above. (?(α, β, λ) ∧ some principles) > R(α, β, λ)

(2.22)

This schema generalises the principles needed for the correct inference individualised for each discourse relation. The axiom features the operator >, which indicates the defeasibility of the implication, and the metavariables α and β, which abstract the eventualities πn and πm given for each discourse segment. As described in section 2.2.2, the eventualities are used as labels which tag the content of the discourse segments. Consequently, these labels not only refer to concrete predicates, but also enable a more abstract description of this content. The use of λ means that the two other variables can be attached to each other in the discourse context. It is said: “if β is attached to α with a discourse relation in some constituent λ (this implies that α must be an (available) attachment site to β in the discourse context), and moreover ‘some stuff’ [some principles, as formulated above] (about α and β and perhaps also λ) holds, then normally, the discourse relation is R.” (Asher and Lascarides 2003: 187) According to Asher and Lascarides, these axiom schemata are part of the glue logic for inferring appropriate discourse relations; this is the second logic of information packaging and discourse update (see section 2.2.6). However, the following depiction of the above mentioned discourse relations will already include these axiom schemata, which can in equal measure be defined individually for each relation to hold. Together with the meaning postulates given in terms of equation 2.21, they make explicit the conditions and qualifications needed for any particular relation. How

i i

i

62 The Logic of Film Discourse Interpretation

discourse interpretation takes place on this basis will be described in section 2.2.6. Now, for each film discourse relation in the above described set, the meaning postulates and default axioms will be introduced. Narration (α, β) As depicted in the Glossary of Discourse Relations in the works of Asher and Lascarides, a Narration-relation holds “if the constituents express eventualities that occur in the sequence in which they were described” (Asher and Lascarides 2003: 462). As described in section 2.2.2, they therefore provide the • Spatiotemporal Consequence of Narration in Verbal Discourse φN arration(α,β) ⇒ overlap(prestate(eβ ), Advβ (poststate(eα ))) (2.23) This means that the end of the eventuality eα is directly succeeded by the beginning of eventuality eβ and both eventualities are arranged in a temporal and spatial continuity. In verbal discourse, the function Advβ is needed for the expression of the spatiotemporal location. This is not the case in filmic discourse, since information about space and time can equally be expressed by nonverbal discourse segments. Hence, the meaning postulate has to be modified as follows: • Spatiotemporal Consequence of Narration in Filmic Discourse φN arration(α,β) ⇒ overlap(prestate(eβ ), poststate(eα ))

(2.24)

This has considerable impact on the inference process, since the constraint is less specific than the one for verbal discourse. Whereas in verbal discourse the function Advβ demands explicit adverbial information, in film, this information is supposed to be constructed through, for example, the semantic content of the image. Temporal relationships then might be depicted by clear details such as a clock, for example, or a sequence of shots during the night and in the morning. Information like this is more or less explicit in the context and may therefore influence the inference process significantly. Furthermore, Asher and Lascarides emphasise that the quality of the Narration-relation is enhanced the more content the discourse segments α and β share (cf. Asher and Lascarides 2003: 164). As a second condition, they introduce the Topic Constraint on Narration which holds that the two discourse structures Kα and Kβ share a common content, but that this shared topic is not essentially necessary for the logical operation.

i i

i

i

i

Film Discourse Interpretation 63

• Topic Constraint on Narration φN arration(α,β) ⇒ ¬(Kα u Kβ )

(2.25)

Here, u is introduced to define the greatest lower bound of the two structures, which, at least, has to hold for the inference of the Narration-relation, and  denotes modal necessity. The authors gloss over explicitly this somewhat difficult definition and simply underline that the common content may in some cases be too weak for the inference of the Narrationrelation. This would be the case, if the greatest lower bound of the two structures is not given, or, as Asher and Lascarides put it, the common content is vacuous (Asher and Lascarides, 2003: 164), i.e., too weak to infer a Narration-relation. Another discourse relation must then be found to interpret the discourse as coherent. On this basis, it is assumed for filmic discourse that the two discourse structures must share a common topic which is strong to the degree that it is not overwritten by another condition provided for the other discourse relations, such as a cause, for example, which would then lead to the inference of a Result- or Explanation-relation. Concrete analysis in chapter 3 will give examples that will explain these cases in more detail. In narrative texts such as film, it is, however, a very common situation that the constraints on narration cannot be fulfilled. Going back to the example of the short film WORDS, the analysis of its structure does not contain any Narration-relations. However, some of the text segments are related to each other in a spatiotemporal sequence; see for example the sequence of shots 5 and 6, which is depicted in table 2.7. #

5.

6.

shot

shot description a football team planning a new play close-up of a board in the meeting situation

spoken language

music

“Here, the back is coming off. . . ”

calm piano and guitar music

“Here, the end is pinching. . . ”

calm piano and guitar music

Table 2.7: Extract of the transcription of the short film WORDS (Everynone, 2010)

i i

i

64 The Logic of Film Discourse Interpretation

The two shots share the common content of the coaching room in which a football team is planning a new play. In contrast to shot 5, which is a long shot of the whole room, shot 6 gives a close-up of the board on which the person speaking is drawing. In particular, his voice track underlines the continuity of time and space in these two shots which both represent the same time and location. If the cut between these two shots displayed a separation of two eventualities, Spatiotemporal Consequence would be fulfilled. However, in contrast to the example depicted in table 2.5, the two shots here are not separated from each other nor described as individual eventualities. Instead, they are inferred as one event being illustrated by two different shots with different camera perspectives. The zoom in the second shot is important to the degree that it makes clear that the characters in the shot are planning their play which then enables the association with the predicate of the preceding events, but it does not cause the segmentation of a further event. Both shots are thus described as eπ5 . This example demonstrates that cuts between shots often maintain spatiotemporal consequence and therefore create a certain continuity which is also responsible for the interpretation of the event. It gives evidence for the power of filmic continuity editing which, according to Bordwell and Thompson, “typically present story events in a 1-2-3 order. [. . . ] So chronological sequence and one-for-one frequency are the standard methods of handling order and frequency within the continuity style of editing” (Bordwell and Thompson 2001: 250). The Spatiotemporal Consequence for Narration in Filmic Discourse thus has to be interpreted more specifically as holding between two separated events which might be illustrated by individual shots or a sequence of shots. Likewise, it is conceivable that the relation holds between events within one single shot which depicts different actions. A crucial factor for the inference of this relation is then the default axiom, which will be taken from Asher and Lascarides (2003). For Narration, it is defined as follows: (?(α, β, λ) ∧ occasion(α, β)) > N arration(α, β, λ)

(2.26)

The equation indicates: if the first discourse segment occasions the second one, a Narration-relation is normally inferred. Occasion here is a relatively weak description of the condition which should hold between the entities. Asher and Lascarides give the following definition: “Scriptal knowledge basically takes the following form: if two event types of a certain kind (φ and ψ) are to be related, then occasion can normally be inferred:

i i

i

i

i

Film Discourse Interpretation 65

• Occasion I: (?(α, β, λ) ∧ φ(α) ∧ ψ(β)) > occasion(α, β)

(2.27)

For example, we assume there’s scriptal information or ‘domain knowledge’ that if x’s falling and a y’s helping x up are connected somehow, then the former occasioned the latter.” (Asher and Lascarides, 2003: 201) Furthermore, Asher and Lascarides point out that Narration indicates “a plan or a ‘natural event-sequence’ such that events of the sort described by α lead to events of the sort described by β” (Asher and Lascarides 2003: 200). Since a second axiom provided by the authors entails an anaphoric cue phrase which is not applicable to film, the given default axiom is the only information available for the inference of the relation. This means as well that the information in the discourse may be overwritten by another default axiom, which, for example, gives the condition for another relation as a more explicit definition within the context and therefore makes the discourse more coherent (see section 2.2.6 for more information on discourse update). Elaboration (α, β) An Elaboration-relation entails “that the events described in π2 describe in more detail those described in π1 ” (Lascarides and Asher 2007: 8; π1 , π2 here stand for some particular discourse referents). Asher and Lascarides accordingly introduce the following meaning postulate: • Temporal Consequence of Elaboration in Verbal Discourse φElaboration(α,β) ⇒ Part-of (eβ , eα )

(2.28)

Part-of is intended to capture the truth conditional effect of temporal inclusion which is inevitably needed for inferring an Elaboration-relation. The event described by the first eventuality contains the event of the second eventuality. The meaning postulate can then be related to a general temporal logic such as that described for example by Allen (1984). This temporal logic explains coherencies between temporal intervals and therefore takes the dynamicity of events into consideration. Allen’s framework is one of the standard formalisations which has been developed for the description of temporal relationships and which gives clear definitions for all types of relations that can exist between temporal intervals. These are for example situations in which two events follow each other (BEFORE (A, B) and AFTER (A, B)) or, as is the case in the Spatiotemporal Consequence for Narration, overlap

i i

i

66 The Logic of Film Discourse Interpretation

(OVERLAPS (A, B) and OVERLAPPED BY (A, B)). In total, the framework describes 13 base relations (cf. Allen 1984). Although Asher and Lascarides do not use these relations in their concrete definition, Part-of is supposed to express exactly what is described by Allen as one of the relations which is defined as CONTAINS (A, B), where A and B describe the temporal components of two eventualities eα and eβ , see figure 2.3.

Figure 2.3: Temporal inclusion according to Allen’s temporal logic, adapted from a graphical illustration by Bateman and Schmidt (2011) Since the Part-of -description is comparatively imprecise in this context, the framework provided in this book will follow Bateman and Schmidt (2011), who adopt Allen’s temporal logic for film, and will take its definitions for describing filmic meaning postulates. The meaning postulate for Elaboration then becomes: • Temporal Consequence of Elaboration in Filmic Discourse φElaboration(α,β) ⇒ contains (eα , eβ )

(2.29)

where contains(eα , eβ ) indicates that the temporal extent of the first eventuality has to contain, thus include the temporal extent of the second eventuality. The formula is an abstract abbreviation and simplification of what is described by Allen’s definition, since it replaces A and B by the eventualities on the whole and thereby adapts to the default meaning postulate described in equation 2.21. In the example of an Elaboration-relation in table 2.8, this entailment is given in that the second shot depicts an action which has already been described in the shot before. Therefore, the two shots would normally be inferred as one single event. Nevertheless, due to the context and the unfolding discourse structure (see section 2.2.3), the zoomed image of blowing into the trumpet has a specific function here, namely depicting the transition to another word (‘blow’). By this means, it can be seen as temporally included into the more general event of playing the trumpet, as is described in the graphical illustration of the temporal relations given in figure 2.3 (cf. Bateman and Schmidt 2011: 204). In addition, the two shots demonstrate the subordinative function of Elaboration which is “to reflect its semantic function of changing granularity of description” (Lascarides and Asher 2007: 8). As became clear in

i i

i

i

i

Film Discourse Interpretation 67

shot audio description track

music

8.

a man sounding the trumpet

trumpet play

calm piano and guitar music

9.

close-up of the man’s mouth blowing into the trumpet

sound of blowing heavily

calm piano and guitar music

#

shot

Table 2.8: Extract of the transcription of the short film WORDS (Everynone, 2010) the representations of the SDRS in section 2.1 above, shots 8 and 9 both represent π 0 which entails the subordination. This again underlines the specification in terms of a close-up which is definitely subordinated to the more general information given in the shot before. Asher and Lascarides define this condition as a subtype (Asher and Lascarides 2003: 206f.). The default axiom for the film discourse relation Elaboration, in contrast, emphasises the need of clear specification which is described as a change in the granularity: (?(α, β, λ) ∧ specif icationD (β, α)) > Elaboration(α, β, λ)

(2.30)

specif icationD assures that it is the discourse itself which gives evidence for this kind of relation. In the example, the specification is realised by the zoom-effect arranged on the visual level. This effect may for example have the function of focusing on the second event, since, depending on the context, the motivation to change the character of the description varies. It is then to be assumed that mainly camera effects such as the zoom or a change in the perspective, which have a specific function in the unfolding of the discourse structure, fulfil the condition for Elaboration. Explanation (α, β) In SDRT, Explanation is identified as a relation with strong parallels to Elaboration. Its truth conditional effect also entails a temporal condition which differs from that of Elaboration in the dependency of the discourse segments (see section 2.2.2):

i i

i

68 The Logic of Film Discourse Interpretation

• Temporal Consequence of Explanation in Verbal Discourse φExplanation(α,β) ⇒ (¬eβ ≺ eα ) φExplanation(α,β) ⇒ (event(eβ ) ⇒ (eβ ≺ eα ))

(2.31)

In contrast to Elaboration which entails a temporal inclusion, Explanation requires a reversed sequence of the eventualities eα and eβ . For discourses such as film whose basic quality amongst others is linearity, this is of considerable importance, since this quality does not logically include a temporal sequence of the events in chronology. On the contrary, narration in film is manifold and often differs in its story schemata. It is therefore necessary to have precise formulations of the conditions needed for the right inference and this depends on temporal circumstances. Taking the temporal logic relation again as a more concrete and standardised definition, the logic of filmic information content provides the following meaning postulate, which is again an abstract formulation of the temporal components of the eventualities: • Temporal Consequence of Explanation in Filmic Discourse φExplanation(α,β) ⇒ bef ore(eα , eβ )

(2.32)

The axiom schema for Explanation is given in equation 2.33. (?(α, β, λ) ∧ causeD (β, α)) > Explanation(α, β, λ)

(2.33)

That is: the inference for Explanation can be drawn when the discourse gives evidence that the second segment causes the first segment. causeD assures again that it is the discourse which must provide this evidence. As Lascarides and Asher (2007) point out: “Evidence of a causal relation is distinct from a causal relation actually holding; the glue logic expresses evidence in the discourse of a causal relation with causeD (β, α) and the actual causal relation between events with cause(eβ , eα ); note that the former does not entail the latter.” (Lascarides and Asher 2007: 25) This evidence given by the discourse is directly connected with the meaning postulate for Explanation which is immediately influenced by the discourse’s structure and linearity. Explanation is thus inferred when the interpretation of the discourse enables a reasonable conjunction between the discourse segments.

i i

i

i

i

Film Discourse Interpretation 69

In table 2.9, two shots from the beginning of the film Requiem for a Dream (Aronofsky, 2000) are transcribed. This film starts with a replay of a TV infomercial. The replay is cross-cut with the opening credits of the film; it is stopped abruptly and cuts to the shot of an electrical connector. The transcription shows the last two shots (10 and 11) of this scene which are inferred as two individual events. shot description

audio track

10.

TV replay, candidate and gameshow host

applause, background noise of the TV show

11.

a connector pulled out of a socket

sound of pulling out the connector

shot

#

spoken language “Juice by Mary, juice by Mary! Ooaahh. . . ” “Ooah. . . ” (stopped abruptly)

Table 2.9: Transcription of the beginning of Requiem for a Dream (Aronofsky, 2000) The eventuality of shot 11, pulling out the connector, gives a plausible cause for the ending of the replay which is the eventuality of shot 10 and the preceding shots. The eventuality is depicted on the one hand through the cut from the TV show to a completely different setting and on the other hand through the sound which then fades out. It can be inferred as ‘go off’. Since the second eventuality of pulling the connector temporally precedes the first eventuality in the discourse and this gives evidence for a causal relation between the two eventualities, an Explanation-relation can be inferred. The two events are therefore related to each other by an Explanation-relation.8 Result (α, β) The reverse relation to Explanation is Result. Just as in Explanation, the underlying basis for this relation is a cause which is connected to its effect. The default axiom for Result therefore is the following (cf. Asher and Lascarides 2003: 463): • Axiom on Result φResult(α,β) ⇒ cause(eα , eβ ) 8A

(2.34)

detailed analysis of this film extract is given in section 3.4.2.

i i

i

70 The Logic of Film Discourse Interpretation

In the work of Asher and Lascarides (2003), there is no explicit temporal condition given for the relation, but due to the temporal consequence for Explanation, it is evident that the order of the eventualities here is also important. Hence, it is assumed that the following second axiom can be given according to the temporal relation defined by Allen (1984): • Temporal Condition for Result φResult(α,β) ⇒ af ter(eα , eβ )

(2.35)

This temporal succession of the eventualities is the inverse of the relation given in equation 2.31 for Elaboration and is expressed as well in the default axiom schema given for Result: (?(α, β, λ) ∧ causeD (α, β)) > Result(α, β, λ)

(2.36)

The reversed order of the eventualities eα and eβ in contrast to the axiom schema for Explanation marks the difference between the two relations. A typical case of such a relation in filmic discourse is for example the elliptic depiction of action processes which do not show the whole storyline, but rather outline its result in one closing and often highlighted shot. Violence in film is often presented by events which are related in such a manner, for example when a gun or weapon is visible in one event and the next shot shows the killed person. When the discourse gives evidence for a reason that the weapon causes the murder, the events can be related by a Result-relation, even though they are in a spatiotemporal consequence which would also be decisive for a Narration-relation. Again, the effects for Result then would be stronger than those for Narration, since the eventualities do not exactly overlap. Background (α, β) The discourse relation Background shows strong parallels to Narration. It imposes, for example, the same temporal consequence: • Temporal Consequence of Background φBackground(α,β) ⇒ overlap(eβ , eα )

(2.37)

But whereas the meaning postulate for Narration explicitly indicates that overlap concerns both spatial and temporal consequence, it is only the latter which is given for a Background-relation here. The relation holds if a discourse segment gives information about the environment and circumstances around the eventuality which is given in the second discourse segment. The second eventuality has to overlap the first one, but is not necessarily located in the same place.

i i

i

i

i

Film Discourse Interpretation 71

However, the relation does require a common topic in both discourse segments. Asher and Lascarides point out that these topics have to be added within the process of building the logical form of the discourse: “Even if a topic isn’t explicitly mentioned in the discourse, the SDRS must explicitly feature that topic and its rhetorical relationship to the other constituents in order to predict the right interpretations [. . . ]” (Asher and Lascarides 2003: 167). There are two default axioms for inferring the Background-relation, which are unfortunately specified only approximately in Asher and Lascarides (2003): (?(α, β, λ) ∧ state(β) ∧ ¬state(α)) > Background1 (α, β, λ) (?(α, β, λ) ∧ state(α) ∧ ¬state(β)) > Background2 (β, α, λ)

(2.38)

Both axioms demand that a state is combined with something which is not a state (¬state). Asher and Lascarides give the following description for this formal depiction: “This relation holds whenever one constituent provides information about the surrounding state of affairs in which the eventuality mentioned in the other constituent occured.” (Asher and Lascarides 2003: 460) The discourse segment defined as the nonstate thus indicates information about the environment of the first state or event, but does not itself represent a state. Asher and Lascarides here refer to the definition of clauses as stative and nonstative. Since this distinction cannot be directly transferred to filmic text and filmic discourse segments, the definition of the default axiom is customised as follows: (?(α, β, λ) ∧ circumstantial information) > Background(α, β, λ)

(2.39)

This definition is less explicit than the one for verbal discourse, but enables the description of an event as giving background and circumstantial information to a preceding event. From a more general point of view, it is assumed that the two segments do not resemble each other very strongly in their structure and semantic composition, but that they should share a common topic. These could, for example, be shots or depictions without any active characters or be establishing shots that introduce the setting. In contrast to Elaboration, Background does not specify the preceding event or give more detailed information, but rather highlights less important information about the surroundings. Their respective function in context therefore distinguishes the two relations. One example for Background is given in the extract from the film Requiem for a Dream from which shots 11 and 12 are depicted in table 2.10. Shot 11 has been identified above as showing how a connector is pulled out of a

i i

i

72 The Logic of Film Discourse Interpretation

socket. The following shot depicts two persons in a living room, talking about the TV which is identified as the device whose connector was pulled.

shot description

audio track

spoken language

11.

a connector pulled out of a socket

sound of pulling out the connector

“Oah. . . ” (stopped abruptly)

12.

an older female and a younger male person talking about the TV

steps on the ground, sound of closing the door

“Harold please, not again the TV.”— “Aw Ma. Come on, Ma.”

shot

#

Table 2.10: Shots 11 and 12 of Requiem for a Dream (Aronofsky, 2000) The second shot thus gives background information to the event depicted in shot 11 in that it identifies not only the device from which a small detail was shown in close-up before, but also the two protagonists, who are shown for the first time in this extract. The focus on the connector in the first shot is thus enriched by information about the setting, the characters and their names, which are given in the voice track. For the unfolding of the filmic content, this information is of particular interest, since it introduces the main protagonists and their relationship to each other. Parallel (α, β) Asher and Lascarides define Parallel and Contrast (see the paragraphs following for concrete definitions of the latter) as typical text structuring relations. Both relations require that the discourse structures of the respective segments are similar. In the case of Parallel, the structures have to be semantically similar as well, but neither Asher and Lascarides nor other researchers (e.g., Hobbs 1990 or Kehler 2002) give a concrete meaning postulate for Parallel. This book thus proposes the following constraint: • Constraint on Parallel φP arallel(α,β) ⇒ (Kα ∼ Kβ )

(2.40)

i i

i

i

i

Film Discourse Interpretation 73

#

3.

4.

shot

shot description a theatre stage, two persons playing in the front two girls in a playground outside

audio track

music

sound of batting staves

calm piano and guitar music

sound of children at play

calm piano and guitar music

Table 2.11: Shots 3 and 4 of the short film WORDS (Everynone, 2010) The formula and in particular the operator ∼ are intended to describe what has been defined by Asher and Lascarides as “a partially isomorphic mapping between these structures” (Asher and Lascarides, 2003: 168). The authors furthermore state: “There must be a common theme between Kα and Kβ . [. . . ] The more informative the common theme, the better the Parallel relation” (Asher and Lascarides 2003: 465). The axiom in equation 2.41 is therefore modulated for film similarly to the other axioms: (?(α, β, λ) ∧ semantic similarity(α, β)) > P arallel(α, β, λ)

(2.41)

A considerable number of examples for Parallel-relations is given in the analysis of the first shots of the short film WORDS above. The eventualities which can be described with the predicate ‘play’ are related to each other by Parallel because of their semantic similarity on the one hand and their similar structure on the other hand. A concrete example is depicted in table 2.11. According to Kress and van Leeuwen (1996), both shots show a bidirectional transactional process with two interactors. The other modalities do not play an important role for the interpretation of the eventuality ‘play’. However, they do share very similar configurations. Contrast (α, β) The second text structuring relation introduced by Asher and Lascarides is Contrast. It differs from Parallel in that it requires semantic dissimilarity, but needs structural similarity as well. Subsequently, the constraint given for Parallel above is also valid for Contrast: • Constraint on Contrast φContrast(α,β) ⇒ (Kα ∼ Kβ )

(2.42)

i i

i

74 The Logic of Film Discourse Interpretation

The default axiom for Contrast, however, differs from the one for Parallel. Asher and Lascarides point out that “the semantics of Contrast(α, β) entails a partial isomorphism between the DRS-structures Kα and Kβ that in turn generates contrasting themes” (Asher and Lascarides 2003: 208). These contrasting themes can more generally be summarised as semantic dissimilarity. (?(α, β, λ) ∧ semantic dissimilarity(α, β)) > Contrast(α, β, λ)

(2.43)

Unfortunately, the axiom for inferring Contrast is not very elaborated. What is described here with “semantic dissimilarity” is rather a vast periphrasis for circumstances which cannot be associated with each other by other relations. Key words such as “but” or “although” which have been listed for relations in verbal discourse (cf. Asher and Lascarides 2003: 208) clearly express the needed antithesis or opposition. However, in filmic discourse, these key words are in most cases not available. Instead, contrasting themes are conveyed nonverbally; they have to be inferred as eventualities of each event. As has been outlined within the analysis above, the relation between the events in shots 9 and 10 of WORDS (see table 2.12) is at first sight interpreted as a Contrast-relation in the discourse structure, since the action process in shot 10 cannot be associated with the predicate ‘play’ which is the dominant predicate in the preceding shots. #

9.

10.

shot

shot description close-up of the man’s mouth blowing into the trumpet a man blowing out candles on a cake

audio track

music

sound of blowing heavily

calm piano and guitar music

sound of blowing out

calm piano and guitar music

Table 2.12: Shots 9 and 10 of the short film WORDS (Everynone, 2010) The DRS in the two shots is similar, but the meaning potential of the respective shots is quite different, so that the two established themes are generally contrasting. From a more global view of the whole discourse

i i

i

i

i

Film Discourse Interpretation 75

structure, it then becomes clear that the two shots are also related to each other by Parallel (see section 2.1 for further information). It is assumed that further analysis of this sort of relation in concrete film extracts will give a better insight into the conditions which establish a contrasting situation between shots. This is in fact taken to be the case for all the types of relations introduced here. 2.2.5

The Syntax for Segmented Film Discourse Representation Structures

So far, the vocabulary for constructing logical forms of the film discourse segments and the set of film discourse relations have been described in terms of their effects on discourse interpretation. With regard to the aim of analysing wholly specified discourse structures which include both the logical forms and the relations, it is now necessary to outline the syntax of the formal language for film discourse interpretation. Together, they enable the description of SFDRS as have already been used in section 2.1. A first combined depiction of several logical forms of the discourse segments and their relations is given in table 2.13. This structure is formuπ0 π1 , π2 eπ1 = play eπ2 = play “play” (m) [v] button [v] unknown actor (o) π1 : [v] unknown actor (n) π2 : [v] unknown actor (p) π0 : [a] rhythmic music starts [a] calm, rhythmic music m, n |∼ play (eπ1 ) o, p |∼ play (eπ2 ) P arallel(π1 , π2 )

Table 2.13: The logical form of shots 2 and 3 of WORDS (Everynone, 2010) lated in the box-style notation form adapted from the kind of description given in DRT and SDRT, but uses the box-style now mainly to outline the different contributions the modalities make in order to result in any particular DRS Kπn . This depiction is also valid for the inference of the relation where temporal conditions, if available between the segments, are specified. In the current example description, however, these conditions are not mentioned because the meaning postulate for Parallel does not

i i

i

76 The Logic of Film Discourse Interpretation

impose temporal requirements. In case of their existence, for example in a Narration- or Explanation-relation, they are simply added to the logical form in terms of variables tn and a formalised description of the temporal situation, as for example in bef ore(t1 , t2 ). The definition of the structure is adapted from the definition of verbal discourse structure as given by Asher and Lascarides (2003: 138): A (film) discourse structure or S(F)DRS is a triple hA, F, LAST i, where: • A is a set of labels π, π1 , π2 , . . . • LAST is a label in A (intuitively, this is the label of the content of the last discourse segment that was added to the logical form); and • F is a function which assigns each member of A a member of Φ, the set of well-formed SDRS-formulae (Ki ) assumed for the interpretation. This definition entails the use of labels which tag the logical forms of the discourse segments. It is thus possible to tag different discourse contents with these labels and to use them as arguments within the representation scheme of the film discourse relations as has already been shown in section 2.2.4. As Asher and Lascarides point out: “Since a discourse relation can connect the label of one text segment to the label of another, where these in turn have content expressed in terms of further discourse relations, discourse relations impose a hierarchical structure.” (Asher and Lascarides 2003: 138) One of these labels within the discourse structure is called LAST , since it is the label which has been added finally to the logical form. However, the description of discourse structure, in particular of verbal discourse, is in most cases open-ended, since it always presents extracts from a dynamically unfolding discourse which could be continued at any time. Film discourse, in contrast, is a temporally delimited, but still unfolding artefact. Nevertheless, the analysis can only take into account film extracts connected within the overall text. The analysis thus takes that label as the last label which has been added to the respective extract and which still participates in the interpretation process. A set Ψ makes available the logical forms for the discourse segments. Φ in the definition is then the subset of Ψ that stands for the set of wellformed SDRS-formulae assumed. Furthermore, it defines R(φ1 , . . . , φn ) as its elements, in which R is a symbol ranging over discourse relations. On the basis of the hierarchical structure generated by the discourse

i i

i

i

i

Film Discourse Interpretation 77

relations, Asher and Lascarides define these relations as being binary (cf. Asher and Lascarides 2003: 138), which means that R is always R(πn , πn0 ). This means as well “that a label π immediately outscopes a label π 0 if F(π) includes as a conjunct a formula of the form R(π 0 , π 00 ) or R(π 00 , π 0 ) for some discourse relation R” (Asher and Lascarides 2003: 138). π 0 is then said to be a sub-constituent of the hierarchically superior label π and π 00 that of π 0 . hA, F, LAST i, where : • A = {π0 , π1 , π2 , π3 , π4 , π5 , π 0 , π6 , π7 , π8 } • F(π1 ) = Kπ1 F(π2 ) = Kπ2 F(π3 ) = Kπ3 F(π4 ) = Kπ4 F(π5 ) = Kπ5 F(π6 ) = Kπ6

(2.44)

F(π7 ) = Kπ7 F(π8 ) = Kπ8 F(π0 ) = P arallel(π1 , π2 ) ∧ P arallel(π2 , π3 ) ∧ P arallel(π3 , π4 ) ∧P arallel(π4 , π5 ) ∧ P arallel(π5 , π 0 ) 0

F(π ) = Elaboration(π6 , π7 ) ∧ Contrast(π7 , π8 ) • LAST = π8 The formula in equation 2.44 depicts the evolving discourse structure of the beginning of the film WORDS. The notation of this structure closely follows the representation forms given for verbal discourse structures in Asher and Lascarides (2003). On the one hand, the formula outlines the labels which are included in the interpretation process. In this example, these are the labels π0 –π8 given in A. These labels are assigned by F to the FDRS Kπ1 –Kπ8 . On the other hand, the description determines that most of the labels are subject to Parallel-relations, but that there is at the same time a hierarchical structure that subordinates π 0 as a label which is elaborated through other labels. The last label added in this structure is π8 which, in the example, has in no way the function of completing the discourse. Instead, the separation of this extract is simply the result of dividing the comprehensive film discourse into pieces which can be clearly handled. The depiction of the discourse structure therefore is an open one which could be extended by further labels and their relations which follow the content tagged with π8 .

i i

i

78 The Logic of Film Discourse Interpretation

This also means that A, as the set of labels which is given for the respective discourse structure, generally consists of more labels than depicted. In sum, it contains the total number of labels which can be found for the various discourse segments of the whole film. As Asher and Lascarides point out: “The function F is represented explicitly in the DRS-looking structures as conditions of the form π : φ— this reflects the fact that F = φ” (Asher and Lascarides 2003: 140). In the formal language for film discourse interpretation, φ is used within the meaning postulates of the discourse relations as the antecedent: φR(α,β) . This means that it reflects the order in which the axioms are applied in the dynamic logic (cf. section 2.2.4; Asher and Lascarides 2003: 159). As a second representation scheme, Asher and Lascarides introduce the extended DRS box notation which represents in particular the hierarchical structure with its subordination of several labels. This scheme is also adopted for the analysis of filmic discourse to outline the different relations found between the labels and to describe the subordination and dependencies in the discourse structure. In table 2.14, the SFDRS for the first shots of WORDS is given. It gives evidence to the fact that every label πn in the structure is related to a DRS Kπn and that it is possible to represent the content of the particular label in a specific structure. π0 π1 , π2 , π3 , π4 , π5 , π 0 π1 : Kπ1 , π2 : Kπ2 , π3 : Kπ3 , π4 : Kπ4 , π5 : Kπ5 P arallel(π1 , π2 ) P arallel(π2 , π3 ) P arallel(π3 , π4 ) P arallel(π4 , π5 ) P arallel(π5 , π 0 ) π0 : π6 , π7 , π8 π6 : Kπ6 , π7 : Kπ7 , π8 : Kπ8 π0 :

Elaboration(π6 , π7 ) Contrast(π7 , π8 )

Table 2.14: Segmented Film Discourse Representation Structure of the beginning of WORDS (Everynone, 2010)

i i

i

i

i

Film Discourse Interpretation 79

2.2.6

Glue Language and Discourse Update in Filmic Discourse

The previous sections have presented until now how filmic discourse segments can be transferred into a logical form and how these logical forms are intended to be combined into SFDRS. The language for the transfer into logical forms has been provided mainly by the logic of information content, although section 2.2.4 additionally gave a view of the default axioms which are needed for inferring film discourse relations. These axioms are already part of the logic of information packaging and discourse update which outlines the concrete interpretational process and which will now be elucidated in detail. The second logic of film discourse interpretation generally consists of two parts: the glue logic and discourse update. Whereas the glue logic mainly provides the processes needed for building the logical forms of discourse segments, discourse update computes the process of binding together different information sources with a rhetorical relation. According to Asher and Lascarides, the interpretation of the logical forms of discourse segments is definitely a more comprehensive process than that of building these forms (cf. Asher and Lascarides 2003: 183), since it includes diverse interpretation patterns. For the interpretation of filmic discourse, the following two features represent the most decisive patterns involved: the Principle of Nonmonotonic, Defeasible Reasoning and the MDC principle. Both principles will be described in detail in the following. The Principle of Nonmonotonic, Defeasible Reasoning In order to construct film discourse structure in terms of S(F)DRS, inference processes have to operate on two different choices: on the one hand, a decision has to be made concerning which discourse segments will be connected by a rhetorical relation; on the other hand, it also has to be chosen which relation holds between the elements. As Eickmann (2003) points out, these processes are a matter of pragmatic reasoning, since the semantic information provided by the discourse segments is often not very rich and has to be combined with world knowledge and further information sources. This is notably the case in multimodal, filmic discourse which in most cases does not provide explicit semantic content, but instead needs additional information for the interpretation of its meaning-making potential (see section 1.1 for further information). Defeasible reasoning in film interpretation is thus a matter of finding arguments for meaning that can normally be constructed by combining the semantic information provided by the discourse segments with world knowledge and further information sources such as film knowledge. However, this combination of different information sources is nevertheless

i i

i

80 The Logic of Film Discourse Interpretation

constrained by default axioms, as explained in the following essential conclusion of Asher and Lascarides: “Computing rhetorical connections is a nonmonotonic process (making discourse update nonmonotonic), based on axioms in a glue logic that encapsulate information from several knowledge resources: lexical and compositional semantics; domain knowledge; and cognitive states, among others.” (Asher and Lascarides 2003: 246) Gluing together the different knowledge sources thus represents the process of reasoning for normally concluded consequences based on the default axioms. For instance, in the example of the short film WORDS, it is necessary to know that the fish which is shown in shot 16 is called a blowfish and therefore can be associated with the dominant predicate ‘blow’ in its context; see table 2.15. #

15.

shot

shot audio description track sound of a blow 2 men against boxing the headgear

16.

a blow fish

17.

a bulb with a blowout

music calm piano and guitar music

sound of bubbling water

calm piano and guitar music

clicking

calm piano and guitar music

Table 2.15: Shots 15–17 of the short film WORDS (Everynone, 2010) A Parallel-relation is thus inferred due to the semantic similarity which can be seen in the inferred eventualities of the discourse segments and which is the default axiom needed for the rhetorical relation. As already outlined within the description of the default axioms for inferring rhetorical relations in filmic discourse, these defaults use the nonmonotonic conditional operator >. The underlying logic for the process of binding together the logical forms is thus a nonmonotonic logic relying on defeasible reasoning which translates the formula A > B as if A then normally

i i

i

i

i

Film Discourse Interpretation 81

B. It is therefore assumed that the coherence of this excerpt is normally inferred by the recipient knowing that the fish is called a blowfish and this is then a parallel eventuality to the predicates given in the shots before. This inference would naturally not be possible if the recipient did not know this fact, which could be the case for example if the recipient’s native language is not English. The principle thus follows the principle of abductive interpretation taken as the main basis for the interpretation of multimodal discourse. Following Peirce, abduction as the third principle of reasoning is most notably a matter of forming hypotheses because of a certain knowledge the interpreter already has (cf. Peirce et al. 1979: 5.171; see section 1.1). This can for example also be hypotheses about how segments in a discourse can be related to each other. Within the interpretation process this hypothesis construction is regulated by the process of discourse update, which is defined as a sequence of update operations depicted as +. + directly relates to the constraints given within the glue logic. Operating a simple update means that a set σ of S(F)DRS is combined with a new piece of information, for example Kβ . The output is a new set of discourse structures, depicted as σ 0 , which combines old and new information with each other. A sequence of such updates is then the updateSDRT which interrelates the different simple updates (cf. Asher and Lascarides 2003: 218): ΣX (σ, Kβ ) is the sequence of updates: σ + Kβ + ?(α1 , β, λ1 ) + . . . + ?(αi , β, λi ) + . . . where hαi , λi i ∈ X is the ith element of X and X ∈ Sσ as the set of all possible sequences of all possible subsets of avail-pairs (σ). This shows that it is not necessarily an already defined discourse structure in terms of Kn which is related to σ. Further assumptions depicted as ?(α, β, λ) can be added as well. However, the update mechanism itself does not give any indication of which discourse segment or structure is actually preferred and will therefore be attached to σ. Although the different default axioms described within the glue logic initially constrain the relational connections and determine the inference process, discourse update is also profoundly effected by the second principle, which will now be described. The Principle of Maximise Discourse Coherence (MDC) Asher and Lascarides note that “there is as yet no systematic way of choosing among the available labels those that the new information should actually attach to [. . . ]” (Asher and Lascarides 2003: 213). It is thus not possible to give clear and largely universally valid mechanisms for the process of gluing

i i

i

82 The Logic of Film Discourse Interpretation

together the segments and constructing the final discourse structure. Instead, this is in each case a distinct and independent process guided by the principle of finding the best and most preferable structure. This is the MDC principle (see section 2.2.2). According to Prevot and Vieu, this can be defined as follows: “MDC is based on a coherence partial order on discourse structures. Maximizing coherence amounts to prefer discourse structures with the smallest number of nodes, the fewest semantic and pragmatic clashes, the largest number of rhetorical relations and the fewest number of specifications.” (Prevot and Vieu 2008: 57f.) A pragmatically preferred discourse structure is for example the one depicted in the graphical representation given in section 2.1. As described in the example analysis, the structure does not contain the Contrast-relation given in the first graphical representation. Instead, the pragmatic clash between the eventualities π7 and π8 is resolved in favour of inferring a Parallel-relation due to the context which follows. Consequently, and because of the number of similar relations, this structure is then preferred to the one which includes the Contrast-relation. It enables a more simple and counterbalanced interpretation of the discourse and therefore maximises its coherence. It is then the conceivably best update which can be given for this excerpt. According to Asher and Lascarides (2003: 234), the MDC-principle can formally be expressed as given in the following equation: • Maximise Discourse Coherence (MDC) Best-updateSDRT (σ, Kβ ) = {τ ∈ updateSDRT (σ, Kβ ) : τ is ≤σ,β -maximal}

(2.45)

σ stands for the discourse context which is updated with the new information β or, as a DRS, Kβ . The Best-updateSDRT of this context and the new information is then given by the generated context in which it would be preferable to interpret the respective relation. In the example given in table 2.15, this context is the one in which a Parallel-relation is preferred to a Contrast-relation. ≤ therefore specifies the so-called partial order on discourse structures in that it has maximal elements which define this order. For the current example, τ : P arallel ≺ Contrast is valid, since the Parallel-relation is the preferred relation in this context. Concrete analysis in the following chapter will outline other descriptions of pragmatically preferred discourse structures in which, for example, labels are minimised or stronger relations between discourse segments can be found. It will

i i

i

i

i

Film Discourse Interpretation 83

then be possible to elucidate different values for the relations depending on their respective context. The two principles depicted in this chapter give a general overview of how the interpretation process of film discourse interpretation is influenced. The overall aim is to generate logical forms of the discourse which outline its well-formedness in terms of its structure and its coherence, which is established through a number of rhetorical relations between the discourse segments. Meaning postulates and default axioms constrain this process of logical construction. However, the multitude of different information sources often leads to conflicting clues about the discourse update (cf. Asher and Lascarides 2003: 247). Since the second logic of film discourse interpretation depends on much more information than the first logic of information content, various knowledge sources cooperate in reasoning about the interpretation preferred. Asher and Lascarides do not go into detail in their description of these resources and only enumerate references such as lexical and compositional semantics, domain knowledge and cognitive states (cf. Asher and Lascarides 2003: 246). These resources for verbal discourse do not account fully for filmic discourse. Examining the semantic representation of the film discourse context already displays in most cases an inferential process which then has to be updated again with new information brought into the context. This incoming information is bound to the existing content by discourse relations which act as mediators that guide hypothesis construction by filling in gaps of under-specified information (cf. Bateman and Schmidt 2011: 42). It is thus the role of film discourse relations to constrain and guide meaning construction within the process of discourse interpretation. Asher and Lascarides emphasise that this interpretation results “from several interacting but separate logics [. . . ]” (Asher and Lascarides 2003: 430; see section 2.2.2) which undertake the task of filling in the different semantic holes raised within the discourse. For filmic as well as verbal discourse, these holes cannot only be filled with linguistic information, but have to be enriched with knowledge about the world, about films and narrative texts, etc. Whereas the glue logic for verbal discourse mainly combines a number of logics that concentrate on linguistic information (i.e., the logic of information content and of (partial) description of content), the logic of film discourse interpretation has to concentrate on those parts of the logics which mainly refer to pragmatically derived information. These are for example the logics of world knowledge and cognitive modelling (see figure 2.2 in section 2.2.2 for more information on the logics that contribute to verbal discourse interpretation). Since film discourse relations generally refer to a wide range of communicative purposes, it is assumed that the relations represent a selection of alternatives from different levels of meaning which can be optionally

i i

i

84 The Logic of Film Discourse Interpretation

associated within a variety of structures. Within these structures, their use can be elucidated further from a functional perspective which tries to work out the different functions the relations fulfil in their respective contexts. This book will therefore combine the formal framework that has now been described in detail with a functional consideration of the relations’ respective use within the extracts and their discourse structure. Therefore, it will use functional accounts to discourse in order to elaborate on the communicative purposes the relations can be associated with (see chapter 4 for more information). According to Bateman and Rondhuis (1997), for example, the discourse relations provided within the SDRT-framework are to be classified within an abstract stratum because of the high level macro organisation they provide for a text or discourse. Since the axioms given for each discourse relation describe relatively neutral and mainly nonhierarchical relationships, they can be ranked as relations that attach communicative purposes to their definition, but are not directly related to the stratum of lexicogrammar. This is true as well, and particularly for the choice of film discourse relations that have been described above. Since these relations combine discourse segments which are already a product of the recipient’s inference process, they are abstract constructions with a considerable distance from concrete filmic phenomena. However, they are assumed to fulfil different functions in different contexts. Their syntagmatic use in a concrete discourse structure gives evidence for these different functions. A commonly attributed organisation of semiotic systems is that of metafunctions (cf. Halliday 2004). Tseng emphasises that a “systematic modelling of metafunctions both within (intrinsic) and beyond (contextual) language systems [. . . ] can lead us to a consideration of how intrinsic filmic discourse patterns directly reflect contextual (generic, historical, stylistic, etc.) conventions” (Tseng 2009: 56). This aspect of reflecting contextual conventions is a crucial factor within the SFDRSframework as well. With regard to the pragmatic preferences which influence for example the choice of relations within a particular structure, a metafunctional organisation of these mechanisms may at the same time offer a better insight into the semantic circumstances. This will be elucidated in subsequent chapters.

2.3

Conclusion of Chapter 2

The aim of the preceding chapter has been to develop and formulate a concrete methodological approach to film discourse interpretation on the basis of structure and coherence in filmic text. The first comprehensive analysis of the short film WORDS in section 2.1 has shown that a detailed consideration of the inference processes is absolutely necessary to be able to profile filmic meaning construction. Solely

i i

i

i

i

Film Discourse Interpretation 85

with regard to the different inferential strategies and knowledge sources influencing the interpretation, it is possible to describe each particular contribution to the meaning-making process. As has been demonstrated within the analysis shown, the framework of discourse relations as described by formal accounts in discourse semantics delivers a strongly formalised depiction of the properties of filmic text and enables at the same time the examination of how these relations can be characterised on the grounds of inferences and the principle of abduction. On the basis of this analysis, section 2.2 has given a detailed outline of the analytical method developed in this book: the logic of film discourse interpretation. With this framework, it is possible to generate logical forms of the discourse which are related to each other by the set of film discourse relations described on the basis of default axioms and meaning postulates, i.e., logical principles. An overview of the proposed framework is given in figure 2.4. Simultaneously, the figure describes the overall process of interpretation. In a first step, the filmic data as the basis of the analysis has to be transcribed as depicted in section 2.2.1. The second step of the analysis is then governed by the logic of information content, which makes it possible to examine the logical forms of the discourse and to describe them in the box-style notation introduced. Subsequently, the logic of constructing the logical form of discourse via the glue language and discourse update produces the overall discourse structure, the final SFDRS (see sections 2.2.5 and 2.2.6). As has already become clear, the framework offers a way to bridge the gap between a basic description and analysis of filmic devices and a detailed outline of the interpretational process of inferring a film’s meaning. It is thus possible to combine the examination of semiotic codes within a multimodal text or discourse with the description of the inferences needed for their interpretation—a task which has hitherto proved intensely challenging. Following on this first application in section 2.1, the experiential analyses in the next chapter will demonstrate further how useful the framework can be for examining filmic meaning construction. For a concluding depiction, figure 2.4 summarises the described framework and its different levels of analysis. It points out that the transcription of the filmic data will serve as a basis for the analysis, but will not be seen as its own stratum within film discourse analysis. Instead, it is intended to highlight the different modalities as purposes of the analysis, which will then be examined by the two logics within the logic of film discourse interpretation. The results of this examination then build a foundation for an overall interpretation of any film.

i i

Figure 2.4: The Logic of Film Discourse Interpretation developed in this book

i

86

i

i

i

i

i

3 Analysis

On the basis of the analytical model presented in section 2.2, this chapter will display comprehensive analyses of filmic text extracts which elucidate their structure and coherence based on relational meaning-making. The analyses of larger excerpts will put emphasis on the unfolding film discourse structures which are established by the discourse relations. The aim is to give examples for every film discourse relation presented in section 2.2.4. This general view of the use of the different relations in several contexts will allow a comprehensive examination of film discourse coherence. The broad applicability of the framework has already been demonstrated within the analysis of the short and nonnarrative film WORDS. The extracts which have been chosen for comprehensive analysis in the following now display a broader range of examples from narrative films from different genres. As the most common form of fictional films and also as used in documentaries and experimental films, “[n]arrative art ruthlessly exploits the tentative, probabilistic nature of mental activity” (Bordwell 1985: 39). It is the basic assumption of a story or event-sequence based on space, time and causality which organises the information of and in our world and which fundamentally initiates the recipient’s meaning construction (cf. Bordwell and Thompson 2001). The analysis of narrative films will thus examine how the recipient generally makes sense of the information depicted in these and other film forms and how the textual unfolding of a film influences this meaning-making process. Although the framework of discourse semantics generally applies for all types of texts, the set of film discourse relations is explicitly designated for narrative (or expository) filmic texts (see section 2.2.4) and so allows a clearly structured examination of film interpretation. This will be exemplified by the comprehensive analysis of extracts from the following films of the 2000s decade: The Lives of Others (2006) Vanilla Sky (2001) Am´elie (2001) Requiem for a Dream (2000)

87 i i

i

88 Analysis

The choice of these films is motivated by several factors. Whereas WORDS does not contain an overall narration or a coherent figuration, the following excerpts are taken from feature films which normally have a length of about two hours and which narrate a coherent and completed story. The excerpts from Requiem for a Dream and Vanilla Sky depict the beginning of the respective movies; the sequences from The Lives of Others and Am´elie are in each case taken from somewhere in the middle of the film. All extracts contain completed scenes which describe one aspect or episode of the overall narration. A short summary is given in each section; the complete transcription of the extracts is given in Appendix A. From an aesthetic perspective, the excerpts represent common film techniques established and now generally accepted in film. They feature a range of specific design elements and typical filmic arrangements which suggest the diversity of current montage and film editing compositions, as in, for example, the parallelised action via cross-cutting in The Lives of Others. The examination in the following sections is intended to outline the textual foundation of these compositions and to describe in detail how meaning construction takes place in these terms. The main focus within the analysis will lie on the arrangement of the auditory and visual level in the film extracts, and thus on the intersemiotic patterns which dominate the meaning construction. The chosen excerpts are therefore discourses which use both sensory channels in intended combination and in which either the visual or the auditory level may play a significant role within a specific part of the meaning-making process. The analyses will be structured according to the following patterns. First, there will be an introduction to the film itself which gives an overview of the filmic data and a short summary of the topic. In a second step, the extensive analysis of the textual “criteria” of the respective extract will outline the different logical forms of its discourse segments. The eventualities employed in these logical forms will account for the significance and relevance of the different modalities and will consequently characterise in detail the intersemiotic meaning construction involved. The analysis will furthermore specify how and with which knowledge source the meaning postulates and default axioms are satisfied within the context and how discourse segments can be related to one another on this basis. Finally, it will become clear how these relations are then combined into Segmented Film Discourse Representation Structures (SFDRS) which display the textual unfolding of the discourse.

3.1 The Lives of Others (2006) The Lives of Others is a movie about the monitoring routine of the secret police (“Staatssicherheit”, abbr. “Stasi”) in the German Democratic Republic in the 1980s. Observation scenes are forcefully shaped by the description

i i

i

i

i

Film Discourse Interpretation 89

of interrogation methods and the controlling machinery. The protagonist and Stasi captain Wiesler, played by Ulrich Muhe, ¨ is assigned to spy on playwright Georg Dreyman (Sebastian Koch), who is suspected of pro-Western sympathies. The image of Wiesler observing Dreyman and listening with a headset to his conversations in his flat dominates the whole film. During this observation, Wiesler ends up in moral conflicts; his participation in the playwright’s daily life changes his attitude and has fatal consequences for his work. Directed by Florian Henckel von Donnersmarck, the film was first released in Germany in spring 2006. In 2007, it won the Academy Award for Best Foreign Language Film. 3.1.1

Summary of the Extract

The chosen extract depicts a key scene of the film (situated approximately after the first half) in which Dreyman comes to know by a telephone call that his good friend and colleague Jerska, who had been blacklisted for being critical of the system, has committed suicide. Wiesler listens in on the call with a second receiver from the attic storey above Dreyman’s flat. His situation is cross-cut with shots from the flat. The noticeable colour palette as a sub-modality of the visual level separates the two settings from each other in that it gives illustrations of Wiesler and the Stasi in grey and green colours and arranges the scenes illustrating Dreyman’s life in brown or beige. The monotonic and consistent audio track of the caller runs through both shot types and is motivated in that the Stasi captain participates in the telephone circuit with a second receiver. He hears the caller’s voice from off in the same way that Dreyman hears it. When the caller concludes the conversation, Dreyman slowly sits down at the piano and reaches for the last gift from his friend Jerska, the score “Sonata for a Good Man”, whose title is depicted in a relatively long shot. Before Dreyman’s hands are shown on the keyboard, piano playing is already audible. Only after a few bars are played is there a cut between the shot of the music book and Dreyman, who is finally playing the piano. This piano playing can be seen as diegetic sound.1 Wiesler listens in on the music with the headset at the same time. The viewer can realise an internal auricularisation,2 although 1 A diegetic sound differs from a so-called extradiegetic off-sound in that the source of the sound or voice is visible in the shot. Off-sounds are thus also background sounds or music which cannot be heard by the film’s characters. A voice-over narration without any reference to a character is also an off-sound. However, there is equally a differentiation between on- and off-sound or voice-on and voice-off: Although the source of the sound or voice might not be visible at first, it can for example be interpreted because of a later depiction. On-sound and voice-on directly refer to a source which is visible on the visual level (cf. Wildfeuer 2013). 2 The French term ‘auricularisation’ was introduced by Franc ¸ ois Jost with regard to the tonal perspective (‘point de vue sonore’). It is analogous to the term ‘focussing’ or ‘focal-

i i

i

90 Analysis

the sonata sounds with the same intensity and quality throughout. As there is no change between on- and off-sound, a technical indication of the auricularisation is not marked. The continuous piano playing highlights a dynamic composition of the two settings and the characters. A semicircle tracking shot starts from a medium close-up of Dreyman and continues within the attic space depicting Wiesler from behind listening to the music in his headset. The dynamic, but slow movements of the montage end in a front view which makes visible that a teardrop runs over Wiesler’s face. The last shot again shows Dreyman at the piano. The playwright concludes playing with the comment: “You know what Lenin said about Beethoven’s Appassionata: — “If I keep listening to it, I won’t finish the revolution.” — Can anyone who has heard this music, I mean truly heard it, really be a bad person?”3 3.1.2

Analysis

The extract from The Lives of Others uses the well-known method of crosscutting two scenes with each other that happen at the same time in different settings. Since the main characters as well as general characteristics of the film, for example the colour palette which clearly distinguishes the different settings from each other, have already been introduced and the main points of the story-line displayed, the sequence focuses on the usage of the piano sonata as a musical element. In addition, the differences between the audio and video track play an important role. It will become clear that it is particularly the intersemiotic composition which produces a very regular discourse structure whose relations give reason for the cross-cutting. This montage is first established by the contrasting shots of the two locations. Whereas shot 1 shows the well-known setting of Dreyman’s flat, shot 2 depicts the second protagonist Wiesler listening with the headset. The two shots differ from each other most notably because of the colouring which disposes the recipient to recognise the already introduced monitoring situation. The shots are connected by the continuous orchestral music in the background which fades out from the sequence before. In addition, the telephone ringing as the most dominant sound in this scene, which recurs again in the third shot, maintains a direct connection. The first ringing (eπ1 ) gives a reason for the event depicted in the second shot (eπ2 ). It calls Wiesler’s attention to the telephone line to which he then isation’ (cf. Jost 1989: 46ff.). Whereas ‘ocularisation’ is used for the visual subjectification, ‘auricularisation’ refers to the differences in the audibility of sounds for characters in a film. It has to be differentiated between a so-called zero-auricularisation and the internal primary or secondary auricularisation. The latter is available when a sound is audible from the character’s perspective and then becomes a diegetic sound due to montage or image-text-allocations (cf. footnote 1; Jost 1989: 57). 3 A complete transcription of the extract is given in section A.2 in the appendix.

i i

i

i

i

Film Discourse Interpretation 91

switches (eπ3 , depicted in shot 3). The following shot thus gives a detailed point-of-view-shot of the listening unit that Wiesler operates with. Finally, shot 4 depicts how Wiesler now takes the second receiver to be able to listen to the phone call (eπ4 ), see table 3.1. eπ1 = ring [v] telephone (m) [v] colouring: brown/beige (n) [a] telephone ringing (o) [a] orchestral music (p) m, o |∼ ring (eπ1 ) eπ2 = listen [v] Wiesler with headset (q) [v] colouring: green/grey (r) [a] orchestral music (s) q |∼ listen (eπ2 ) eπ3 = activate [v] Wiesler’s hand (t) [v] listening device (u) [v] colouring: green/grey (v) [a] telephone ringing (w) [a] orchestral music (x) t, u |∼ activate (eπ3 ) eπ4 = listen [v] Wiesler with receiver (y) [v] colouring: green/grey (z) [a] orchestral music (a) y |∼ listen (eπ4 ) Table 3.1: The logical forms of the events in shots 1–4 of the The Lives of Others extract Although the logical forms of the events which can be interpreted within the shots represent a certain similarity mainly on the auditory level, the visual level illustrates a cause-effect association. Since processes of listening and activating follow the event of ringing, they are in a temporal succession as defined within the Temporal Condition for Result in section 2.2.4. Furthermore, the ringing phone in shot 1 depicts a cause for the event of the following shots; shot 2 and 3 show the reaction to this cause. The relation which can then best be inferred due to these conditions is a Result-relation: (?(eπ1 , eπ0 , λ) ∧ causeD (eπ1 , eπ0 )) > Result(eπ1 , eπ0 , λ)

(3.1)

i i

i

92 Analysis

π 0 in the equation represents shots 2–4 which together constitute a subordination to the first discourse segment and therefore can be understood as one single event. Nevertheless, the different eventualities depicted in table 3.1 must also be inferred due to fine-grained differences in the action processes. Shot 3, which gives Wiesler’s point-of-view perspective on the listening device, can be related to the preceding segments by an Elaboration-relation and therefore is part of the second entity which is necessary for the Result-relation. This depicts a kind of specification of the general setting of the attic storey which is opposed to Dreyman’s flat. In addition, the shot is a close-up moving in on a previously established setting, which in general gives a more detailed view of the depicted event. Since the process of activating the listening device is generally focused by this closeup, the shot can be interpreted as having the more specific function of concentrating on this aspect. If this function were not visible, shots 2 and 3 would be interpreted as one single event. Instead, an Elaboration-relation is inferred here because of the described condition of specifying the view of the setting and highlighting the monitoring situation again: (?(eπ2 , eπ3 , λ) ∧ specif icationD (eπ3 , eπ2 )) > Elaboration(eπ2 , eπ3 , λ) (3.2) Shot 4 can be related to shot 3 in that together they satisfy a further Resultrelation. Shot 4 shows Wiesler with the receiver, now being able to monitor the telephone call. The cause for this ability is given in shot 3 which shows the activation of the telephone line. The same conditions given for the first Result-relation, including the temporal condition, are again fulfilled: (?(eπ3 , eπ4 , λ) ∧ causeD (eπ3 , eπ4 )) > Result(eπ3 , eπ4 , λ)

(3.3)

The orchestral music in the background here operates supportingly in that it continues and connects all four shots to each other. A general coherence is thus already given on the basis of this continuous sound track. It facilitates the inference process and is therefore added to the logical forms of the discourse segments.4 The discourse structure then unfolds as the SFDRS given in table 3.2. The structure does not yet reflect the parallelism given by the cross-cutting montage. In fact, and due to the subordination of the three shots which continuously depict the same setting, there are no direct Parallel-relations which would maintain the similarities which normally construct parallel action. This changes with regard to the following discourse segments. The transcription of shots 5–9 gives evidence in this case for the strong parallels 4 The soundtrack thus effects the most likely attachment sites that influence the updateSDRT -process in this context (cf. Asher and Lascarides 2003: 212ff.). However, a precise analysis of these attachment sites would go beyond the scope of this book and needs considerable further research (see chapter 5 for a short outlook).

i i

i

i

i

Film Discourse Interpretation 93

π0 π1 , π 0 π1 : Kπ1 Result(π1 , π 0 ) π0 :

π2 , π3 , π4 π : Kπ2 , π3 : Kπ3 , π4 : Kπ4 π0 : 2 Elaboration(π2 , π3 ) Result(π3 , π4 )

Table 3.2: Segmented Film Discourse Representation Structure of the first shots (1–4) of the The Lives of Others extract between the different events which can be inferred here; see table 3.3. The logical forms of the events make it evident that their semantic characteristics show strong analogies and therefore can easily be related to each other; see table 3.4. The eventualities of shot 5–8 are all inferred as listen, since both actors, Dreyman and Wiesler, are shown in medium close-up, listening to the caller on the telephone. Their shapes dominate the visual level, no sounds other than the voice of the caller are audible. Although the colouring of the shots contrasts the two settings and the vectors which emanate from the participants’ viewing direction are in opposition to each other, the events share common features and in particular the same topic. The events thus are both structurally as well as semantically similar, since the voice track, which runs through all four shots, creates a continuous topic and semantic analogy. The relations that can be inferred for this sequence are thus Parallel-relations which uphold a typical cross-cutting montage. Their logical description of these relations, as introduced in section 2.2.4, is given in the following: φP arallel(eπ5 ,eπ6 ) ⇒ (Kπ5 ∼ Kπ6 )

(3.4)

φP arallel(eπ6 ,eπ7 ) ⇒ (Kπ6 ∼ Kπ7 )

(3.5)

φP arallel(eπ7 ,eπ8 ) ⇒ (Kπ7 ∼ Kπ8 )

(3.6)

Since eπ5 in shot 5 also shows a strong similarity to the other discourse segments, it can be related to the preceding events by a Parallel-relation and the SFDRS of this extract can be expanded as given in table 3.5. This makes it evident that the first subordination establishes the essential cross-cutting montage which is then maintained by Parallel-relations. The

i i

i

94 Analysis

shot description

spoken language

5.

Dreyman’s flat: Dreyman appears, answers the telephone

“Yes?” — “Georg, Wallner here.”

6.

attic storey: Stasi captain listens to the telephone conversation

“What’s up?” — “Georg, it’s about Jerska. He hanged himself last night.”

7.

Dreyman’s flat: Dreyman listens to the caller

“Georg?”

8.

attic storey: Stasi captain listens to the conversation

“I’m going to hang up, okay?”

9.

Dreyman’s flat: Dreyman hangs up slowly

/

#

shot

Table 3.3: Transcription of shots 5–9 of The Lives of Others (Henckel von Donnersmarck, 2006) relations here create a typical filmic design element and therefore mainly serve as text structuring entities which arrange the discourse’s texture. Nevertheless, the alternating sequence of shots depicts as well a narrative continuity in that it expresses a chain of events which occur in a temporal succession. The audio track also keeps up this continuity by providing the strong common topic of the telephone call. Both protagonists in this scene, albeit spatially separated from one another, listen to the caller and his voice track in the same way. Whereas the monitoring situation in general is already a well-known situation which has been depicted repeatedly before, the content of the voice track now is exclusively in the foreground of the filmic representation. Neither is any other sound audible nor are the characters’ reactions of further importance at this point and the camera hardly moves. The semantic content of the voice track therefore becomes

i i

i

i

i

Film Discourse Interpretation 95

eπ5 = listen eπ6 = listen [v] Dreyman on the telephone (m) [v] Wiesler with receiver (p) [v] colouring: brown/beige (n) [v] colouring: green/grey (q) [a] “Georg? Wallner here” (o) [a] “Georg, it’s about. . . ” (r) m, o |∼ listen (eπ5 ) p, r |∼ listen (eπ6 ) eπ7 = listen eπ8 = listen [v] Dreyman on the telephone (s) [v] Wiesler with receiver (v) [v] colouring: brown/beige (t) [v] colouring: green/grey (w) [a] “Georg?” (u) [a] “. . . going to hang up, ok?” (x) s, u |∼ listen (eπ7 ) v, x |∼ listen (eπ8 ) eπ9 = hang up [v] Dreyman in his flat (y) [v] telephone on the dresser (z) [v] colouring: brown/beige (a) y, z |∼ hang up (eπ9 ) Table 3.4: The logical forms of shots 5–9 of the The Lives of Others extract

π0 π1 , π 0 π1 : Kπ1 Result(π1 , π 0 ) π2 , π3 , π4 , π5 , π6 , π7 , π8 π2 : Kπ2 , π3 : Kπ3 , π4 : Kπ4 , π5 : Kπ5 , π6 : Kπ6 , π7 : Kπ7 , π8 : Kπ8 π0 : Elaboration(π2 , π3 ) π 0 : Result(π3 , π4 ) P arallel(π4 , π5 ) P arallel(π5 , π6 ) P arallel(π6 , π7 ) P arallel(π7 , π8 )

Table 3.5: Segmented Film Discourse Representation Structure of the first shots (1–8) of the The Lives of Others extract

i i

i

96 Analysis

the most important aspect in this strand of the film’s plot and is even more highlighted by the rather simple discourse structure. The cross-cutting montage can thus be interpreted more generally as one event within this extract which is designed by the separation of several sub-events described as the different eventualities depicted in table 3.5. These eventualities are the result of a rather fine-grained segmentation because of physical changes, but do not assume a conceptual change (cf. Zacks and Magliano 2011: 6). Hence, the cuts in this scene enable the interpretation of two different cross-cut events which again are separated into several ‘fine events’ (cf. Zacks and Magliano 2011), as suggested in figure 3.1.

Figure 3.1: The cross-cutting montage in The Lives of Others The relations displayed in table 3.5, for example, hold between the events which directly follow each other, as is normally the case in linearly unfolding discourse. However, the description of the eventualities makes visible that not only the alternating shots resemble each other in their structure, but also that the respectively disconnected ‘fine events’ exhibit a kind of relation between each other. Separate from the alternation, they maintain a spatiotemporal consequence which depicts the process of listening to the caller from either the flat or the attic storey. Besides the binary relationships between the two participants, there is thus in addition a continuation within the discourse segments in their respective spatiotemporal locales (cf. Bateman and Schmidt 2011: 218f.), as the dashed arrows in the graphical representation indicate. The bold arrows, in contrast, illustrate the film discourse relations which hold between the alternating events. Although the monitoring situation persists throughout the whole scene, the coherence of the following shots suspends the alternation in that the text focuses on the setting of Dreyman’s flat in the following three, comparatively long running shots. Subsequent to several close-ups on the characters, there follows in shot 10 a long shot of the setting which, on

i i

i

i

i

Film Discourse Interpretation 97

the one hand, describes a parallel to the first shot of this scene and, on the other hand, then focuses on the reaction of the protagonist. The logical forms of the events which can be inferred from these shots are given in table 3.6. eπ10 = sit down [v] Dreyman in his flat (m) [v] piano (n) [v] colouring: brown/beige (o) [a] silence (p) m, n |∼ sit down (at) (eπ10 ) eπ11 = grasp [v] Dreyman in his flat (q) [v] fem. protagonist appears (r) [v] Dreyman’s hands (s) [v] score sheet “Sonata” (t) [v] colour: brown/beige (u) [a] piano playing starts (v) s, t |∼ grasp (eπ11 ) eπ12 = play [v] hands on the piano (w) [v] colouring: brown/beige (x) [a] piano playing (y) [v] camera panning (z) [v] D.’s stature (a) w, y |∼ play (eπ12 ) Table 3.6: The logical forms of the events in shots 10-12 of the The Lives of Others extract eπ10 describes the event happening in the comparatively long shot 10. As an immediate reaction to the phone call, the protagonist turns to the piano and sits down there. Since the event of the phone call depicted before can be interpreted as a reason for the protagonist’s reaction, a cause-effect relationship may be inferred. Nevertheless, a direct cause as the fulfilled condition for Result cannot be found; hang up does not cause sit down. The relation which can be inferred here because of the spatiotemporal consequence is then Narration, see table 3.7.

i i

i π9 , π10

π10

eπ10 = sit down [v] Dreyman in his flat (m) [v] piano (n) [v] colour: brown/beige (o) : [a] silence (p) m, n |∼ sit down (eπ10 ) holds (eπ10 , t2 ) overlap(t1 , t2 )

Table 3.7: The logical form of the events in shots 9 and 10 of the The Lives of Others extract

N arration(π9 , π10 )

eπ9 = hang up [v] Dreyman in his flat (w) [v] phone on dresser (x) π : π0 : 9 [v] colour: brown/beige (y) w, x |∼ hang up (eπ9 ) holds (eπ9 , t1 )

π0

i

98

i

i

i

i

Film Discourse Interpretation 99

Simultaneously, at the moment when the piano comes to the fore, piano playing begins and marks a small, but significant variation in the semantic description of the content which initiates the separation of a further eventuality, eπ11 . The noticeable difference between silence in the first eventuality and the piano playing beginning in the second is marked on the auditory level [a]. At the same time, the female protagonist appears in the background, which can also be seen as a change in the action and therefore an indication for the separation of a further event. In addition, the cut to a close-up of Dreyman’s hands highlights a third discourse referent, the score sheet of the sonata. This close-up is the reason for separating a further eventuality within the coarse event of sitting down at the piano, since the fine event in eπ11 focuses especially on the sonata. In addition, it establishes a direct relation to the piano playing by the title of the score sheet, which identifies the music in the background. The fine-grained segmentation of the two events is depicted in figure 3.2.

Figure 3.2: The interference between shot and fine event boundaries in The Lives of Others In contrast to the cut between the two shots, the boundary between the two fine events is postulated to occur at that moment when the piano playing begins in the background. The eventuality grasp directs attention to a specific aspect of the situation and therefore changes the granularity of description by highlighting the score sheet and its title. The close-up in eπ11 and the clearly visible title thus mark an extension of information given by the previous discourse segment. The name of the score sheet, which serves as an insert that normally identifies objects on the visual level, is directly related to the auditory level which progressively becomes more important. The change of the camera perspective can thus be understood here as a specification and enables the inference of an Elaboration-relation. The Temporal Consequence as well as the default axiom for Elaboration are fulfilled and so block the inference of any other discourse relation; see table 3.8.

i i

i Elaboration(π10 , π11 )

π10

eπ10 = sit down [v] Dreyman in his flat (m) [v] piano (n) : [v] colour: brown/beige (o) [a] silence (p) m, n |∼ sit down (eπ10 ) holds (eπ10 , t1 )

π10 , π11

π11

eπ11 = grasp [v] Dreyman in his flat (q) [v] fem. protag. appears (r) [v] Dreyman’s hands (s) [v] score sheet “Sonata” (t) : [v] colour: brown/beige (u) [a] piano playing starts (v) s, t |∼ grasp (eπ11 ) holds (eπ11 , t2 ) contains (t1 , t2 )

Table 3.8: The logical form of the events in shots 10 and 11 of the The Lives of Others extract

π0 :

π0

i

100

i

i

i

i

Film Discourse Interpretation 101

As described in table 3.6, the explicit significance of the piano playing does not appear until eventuality eπ12 is inferred. Then, however, the entity [piano playing] becomes decisive for the construction of the logical form and its variable needs to be part of the propositional depiction; see table 3.6. Consequently, the auditory level also affects the process of relating the discourse segments to each other. Shot 12 finally shows how Dreyman is playing the piano and therefore gives evidence for the playing audible before. The piano sound thus can be seen as an on-sound whose source is depicted later, as set out in footnote 1 on page 89. The temporal succession of the two events and their causal relation enable a reasonable conjunction between the visual and the auditory level and allow the inference of an Explanation-relation between eπ11 and eπ12 following: φExplanation(eπ11 ,eπ12 ) ⇒ bef ore (eπ11 , eπ12 ) (?(eπ11 , eπ12 , λ) ∧ causeD (eπ11 , eπ12 )) > Explanation(eπ11 , eπ12 )

(3.7) (3.8)

The logical form of the two eventualities eπ11 and eπ11 related by Explanation is then as given in table 3.9. The fine-grained segmentation of the two events and their relationship again give evidence for the intersemiotic meaning construction in this extract. Its specificity originates at least partially in the overall structure of the discourse. With regard to the definitions given in footnote 1 on page 89, the structural composition thus enables a first examination of these differences on the auditory level. Nevertheless, it is not possible to outline the particular relations between the different levels of the audio and video track. It cannot be described, for example, that the visual depiction of the playing is not actually totally synchronous to the audio track, although it is interpreted as an on-sound which is produced by the character. However, the separation of the individual events has made clear that the beginning of the piano playing here plays an important role for the segmentation and is furthermore supported by the change in the camera perspective. From a more general perspective, the continuous piano playing which also overlaps the shots following is the crucial factor for the maintenance of the cross-cutting montage. It has thus the same function that the telephone call has in the beginning of this extract, namely to create an overall dynamic association of the two settings and the characters. Therefore, it is a determining part of the logical forms of the following shots and constrains the inference process effectively.

i i

i Explanation(π11 , π12 )

π11

eπ11 = grasp [v] Dreyman in his flat (q) [v] fem.protag.appears (r) [v] D.’s hands (s) : [v] score sheet “Sonata” (t) [v] colour: brown/beige (u) [a] piano playing starts (v) s, t |∼ grasp (eπ11 ) holds (eπ11 , t1 )

π11 , π12

π12

eπ12 = play [v] hands on piano (w) [v] colour: brown/beige (x) [a] piano playing (y) [v] camera panning (z) : [v] D.’s stature (a) [v] side view (b) w, y |∼ play (eπ12 ) holds (eπ12 , t2 ) before (t1 , t2 )

Table 3.9: The logical form of the events in shots 11 and 12 of the The Lives of Others extract

π0 :

π0

i

102

i

i

i

i

Film Discourse Interpretation 103

A second influential resource besides the piano sound is the semicircle tracking which begins in the second part of shot 12 to focus on Dreyman’s figure and which continues in the attic. The camera first moves towards Dreyman, who is then shown in a close-up which finally identifies him as the one playing the piano. The movement towards the character’s face establishes that the piano playing is a diegetic on-sound whose symbolic function in combination with its score sheets also plays an important role for the interpretation of the whole scene. Its proceeding as a semicircle tracking shot gives reason for the following cut to the second setting where the Stasi captain Wiesler listens to the playing via his headset. The movement from right to left is extended over both shots which depict the captain from behind and in a front view. The second shot showing the attic storey then gives a medium close-up of the captain and makes it visible that a teardrop runs down his face. It is again the music which not only supports the dynamic camera movement, but also dominates the whole sequence. Its quality is still equal to the on-sound quality in the preceding shots, although Wiesler hears it with the headset. Normally, the audio track in this case would be designed as an auricularisation of the character (see footnote 2 on page 89). The composition of the logical forms of these shots evokes a Parallel-relation, since it is anticipated that the cross-cutting is repeated here and has been established by this kind of relation before (see for example table 3.5). The shared topic of the piano playing indeed argues for this relation and the structural compositions of the shots are relatively similar. It is inferred that the Stasi captain Wiesler is still listening to the events in Dreyman’s flat and now comes to the fore again. Since the parallelism between the two protagonists is already established by the previous cross-cutting montage, the situation is now repeated similarly. The proceeding audio track as well as the semicircle tracking shot accompany this parallelism and create a temporal consequence between the shots. It may be assumed that the best update of the discourse structure is essentially already given by inferring the Parallel-relation between the segments, for which a temporal consequence is not required. Although the logical forms of the eventualities differ from each other in more than one entity, they share a general semantic similarity which is the condition for Parallel and which can be seen in the comparison of the two shots in figure 3.3. On the basis of the already established cross-cutting montage, it can here be inferred again that the two events happen simultaneously. The condition semantic similarity is thus fulfilled because of the contextual knowledge the recipient has already built up. The temporal consequence, which is for example part of the meaning postulates for Background or Narration, is overwritten by this argument and rather can be seen as generally highlighting and connecting the parallelism between the events. In the previous cross-cutting montage, this continuity was created by

i i

i

104 Analysis

Figure 3.3: General similarity between shots 12 and 13 in the The Lives of Others extract the voice track of the caller which accompanies the shots. It is thus in particular the preceding context of the film which dominates the inference process here. The Parallel-relation is thus inferred as holding between the two shots that depict the monitoring situation in a cross-cutting montage. A second indication for the preferred update is the given context of the following shots which, on the one hand, give a detailed insight into the character Wiesler and, on the other hand, go back to the setting of Dreyman’s flat where he himself is still playing the piano. From a more general point of view, the insertion of the two shots of the attic storey therefore displays a short navigation to the occurrences which happen simultaneously. According to Zacks and Magliano (2011), the following front view shot depicting Wiesler in a medium close-up would not normally be inferred as a further event within this scene, because there is at first sight no remarkable change in the action visible. Nevertheless, the change in the camera perspective has the function here to elaborate on the parallelism between the two eventualities and to expand the preceding content in that it gives a very close detail of the character’s reaction. The medium close-up of the protagonist then makes visible the teardrop which runs down his face. It can be seen as an effect of the listening situation, which happens as a consequence of the piano playing, and therefore is inferred as a further event within this scene. Therefore, it may be inferred that the event of crying is caused by the preceding discourse content and evokes the inference of a Result-relation. (?(eπ13 , eπ14 , λ) ∧ causeD (eπ13 , eπ14 )) > Result(eπ13 , eπ14 , λ)

(3.9)

The hearing of the piano playing, interpreted as listen (eπ13 ), represents the cause for the eventuality cry (eπ14 ). With regard to the overall content of the film, this Result-relation seems to be the best update in the inference process. The interpretation of a cause-effect relationship between the piano playing and Wiesler’s reaction has a high symbolic value for the development of the film’s plot, which describes how Wiesler changes his attitude towards Stasi principles. Assuming that the unfolding of the discourse structure in a specific way directly displays the development of

i i

i

i

i

Film Discourse Interpretation 105

the story-line, the arrangement of the described sequence gives evidence for this assumption and is further supported by the following context. This depicts again the protagonist Dreyman playing the piano. Hence, the shots showing Wiesler in the attic are framed by type-equal eventualities: eπ12 (=play) and eπ15 (= play). The eventualities eπ14 and eπ15 can therefore be related by a Parallel-relation which again points to the cross-cutting montage. For a detailed view of the semantic composition of the shots, see their logical forms in table 3.10. eπ14 = cry [v] camera panning (g) [v] Wiesler at attic storey (h) [v] teardrop on his face (i) [v] colouring: grey/green (j) [a] piano playing (k) h, i |∼ cry (eπ14 ) eπ15 = play [v] Dreyman at the piano (l) [v] Dreyman’s girlfriend standing behind him (m) [v] colouring: brown/beige (n) [a] piano playing (o) l, o |∼ play (eπ15 ) eπ16 = comment [v] Dreyman at the piano (p) [v] Dreyman’s girlfriend standing behind him (q) [v] colouring: brown/beige (r) [a] “You know what Lenin said. . . ” (s) p, s |∼ comment (eπ16 ) Table 3.10: The logical form of the events in shots 14 and 15 of the The Lives of Others extract This makes visible that the eventualitites eπ15 and eπ16 are eventualities within the last shot of this scene, which are inferred because of the change of action in the discourse. The beginning of the voice track (s) marks the boundary of eπ15 and the beginning of eπ16 . Both eventualities that can be inferred in this longer shot, play and comment, depict a natural event-sequence of two events which temporally overlap. The relation that can be inferred is thus Narration. With regard to the hypothesis that the discourse structure effects the overall interpretation and gives reason for evaluating the sequence as a key scene of the movie, the collection of eventualities has to be analysed in more detail—for example in terms of different topics which have been established. However, the description of the logical forms and their

i i

i

106 Analysis

connections makes evident that the coherence in this extract is in most cases maintained by relations which function ideationally, i.e., which reconstruct and uphold representations of the film’s content. The succession of the shots as is on hand here thus does not only influence the texture of the extract, but also carries the development of the story-line. h A, F, LAST i, where : 0

• A = {π0 , π1 , π , π2 , π3 , π4 , π5 , π6 , π7 , π8 π9 , π10 , π11 , π12 , π13 , π14 , π15 , π16 } • F(π1 ) = Kπ1 F(π2 ) = Kπ2 F(π3 ) = Kπ3 F(π4 ) = Kπ4 F(π5 ) = Kπ5 F(π6 ) = Kπ6 F(π7 ) = Kπ7 F(π8 ) = Kπ8 F(π9 ) = Kπ9 F(π10 ) = Kπ10 F(π11 ) = Kπ11 F(π12 ) = Kπ12 F(π13 ) = Kπ13 F(π14 ) = Kπ14 F(π15 ) = Kπ15 F(π16 ) = Kπ16 F(π0 ) = Result(π1 , π 0 ) 0

F(π ) = Elaboration(π2 , π3 ) ∧ Result(π3 , π4 ) ∧ P arallel(π4 , π5 ) ∧P arallel(π5 , π6 ) ∧ P arallel(π6 , π7 ) ∧ P arallel(π7 , π8 ) ∧ P arallel(π8 , π9 ) ∧N arration(π9 , π10 ) ∧ Elaboration(π10 , π11 ) ∧ Explanation(π11 , π12 ) ∧P arallel(π12 , π13 ) ∧ Result(π13 , π14 ) ∧ P arallel(π14 , π15 ) ∧N arration(π15 , π16 ) • LAST = π16 The logical form of eπ16 is the last eventuality in this extract. π16 is thus the last label which is added to the discourse structure (see the equation above). Since all shots can be formulated as logical forms of the discourse, these Film Discourse Representation Structures (FDRS)

i i

i

i

i

Film Discourse Interpretation 107

Kπ1 –Kπ16 are assigned to members of the overall structure hA, F, LAST i. Their coherence is formally proven by the discourse relations listed in the equation above, which gives as well an insight into the structure’s hierarchical unfolding. The Discourse Representation Structure (DRS) box notation of the SFDRS is then as given in table 3.11. This makes visible again that the structure of the excerpt contains only one subordination which constitutes the depicted hierarchy. It is therefore a relatively simple and straightforward discourse structure whose coherence is established by a multitude of different relations which, however, in most instances fulfil the function of creating and construing the filmic representation of the story-line. The sequence of Parallel-relations, in contrast, gives evidence for the specific filmic configuration of the cross-cutting montage. These relations therefore serve as text-structuring entities, nearly analogously to those given for verbal discourse by Asher and Lascarides. A final graphical representation of this structure is given in figure 3.4. 3.1.3

Conclusion

The extract chosen from The Lives of Others by Florian Henckel von Donnersmarck depicts a clearly structured discourse. The SFDRS described within the analysis prove the coherence of the excerpt and show its particularly intersemiotic meaning construction. This is maintained by noticeable interrelationships between the auditory and the visual level on the one hand and the use of the musical element on the other hand. The detailed analysis of the logical forms of the different shots reveals that in most cases both sensory channels are involved in the inference process. This becomes evident in the propositional description of the eventualities, which displays in each case which entities of the logical form influence the meaning construction. In particular the piano playing dominates the extract and has a high symbolic value for the development of the film’s plot. Since the playing itself is a crucial factor for inferring the semantic configuration of some discourse segments, its overall structure supports the accentuation of this symbolic function. Whereas it first serves as background music, the playing then comes to the fore and can be interpreted as diegetic sound because of its direct relation to the visual level. It is, for example, directly connected to the information given by close-ups on the score sheet. The textual composition made visible by the examination of the unfolding discourse structure thus makes it possible to pinpoint important filmic practices and means of design. This applies as well with regard to the cross-cutting montage as a filmic specificity. The overview of the complete discourse structure, given in table 3.11, points out that this kind of montage is directly established by relations which mainly function on the textual perspective. The order

i i

i

108 Analysis

π0 π1 , π 0 π1 : Kπ1 Result(π1 , π 0 ) π2 , π3 , π4 , π5 , π6 , π7 , π8 , π9 , π10 , π11 , π12 , π13 , π14 , π15 , π16 π2 : Kπ2 , π3 : Kπ3 , π4 : Kπ4 , π5 : Kπ5 , π6 : Kπ6 , π7 : Kπ7 , π8 : Kπ8 , π9 : Kπ9 , π10 : Kπ10 , π11 : Kπ11 , π12 : Kπ12 , π13 : Kπ13 , π14 : Kπ14 , π15 : Kπ15 , π16 : Kπ16 Elaboration(π2 , π3 ) Result(π3 , π4 ) π0 : P arallel(π4 , π5 ) π 0 : P arallel(π5 , π6 ) P arallel(π6 , π7 ) P arallel(π7 , π8 ) P arallel(π8 , π9 ) N arration(π9 , π10 ) Elaboration(π10 , π11 ) Explanation(π11 , π12 ) P arallel(π12 , π13 ) Result(π13 , π14 ) P arallel(π14 , π15 ) N arration(π15 , π16 )

Table 3.11: Segmented Film Discourse Representation Structure of the entire extract from of The Lives of Others

i i

i

i

i

Film Discourse Interpretation 109

π0

π1 Result π0

π2

Elaboration

π3 Result π4 Parallel (π5 , π6 ) ∧ (π6 , π7 ) ∧ (π7 , π8 ) π9 Narration π10 Elaboration π11 Explanation π12 Parallel π13 Result π14 Parallel π15 Narration π16

Figure 3.4: Graphical representation of the SDRS of the extract from The Lives of Others. Due to space constraints, the eventualities π5 –π8 are summarised. They are related by a sequence of Parallel-relations. i i

i

110 Analysis

of Parallel-relations gives evidence for the meaningful opposition of semantically and structurally similar discourse segments which do not only create a varied and interesting text segment, but which have considerable influence on the filmic representation of the plot. However, the structural depiction also reveals that this sequence is not only created by text-structuring relations. The alternation in the first part of the extract is then broken up by relations that normally give more insight into a particular setting or situation, such as, for example, the Elaboration- and Explanation-relation as cause-effect relationships. The Narration-relation at the end of the structure demonstrates that a more fine-grained analysis of the particular events within the scene is definitely necessary to make available each meaningful aspect of the discourse segments. Furthermore, the use of the relation here indicates another specific design element: the maintenance of the atmosphere which is created within the last shot precisely because of the natural event-sequence and the spatiotemporal consequence which is upheld by the Narrationrelation. The analysis of the chosen extract thus not only gives a detailed examination of the intersemiotic composition of this discourse and its meaning construction, but allows as well the consideration of filmic specificities which can at least partly be analysed with linguistic tools. Their textual and discursive composition can then support interpretations on a higher level, for example with regard to symbolic values.

3.2 Vanilla Sky (2001) Cameron Crowe’s Vanilla Sky is an American remake of the Spanish movie Abre los ojos (1997) by Alejandro Amen´abar. It describes the identification process of the protagonist David (played by Tom Cruise) after having been involved in a car crash and being horribly disfigured. He struggles to gain control over his life and stumbles across the fine line between reality and dream. The film was first released in 2001 and stars amongst others Pen´elope Cruz, who also played the female protagonist in Amen´abar’s Abre los ojos. An important aspect of the movie in general is the particular use of music. Recent pop songs as well as rock music compositions have considerable influence on the film’s character and its meaning potential. The chosen extract is the beginning of the movie, which means that neither characters nor settings have been introduced before. The transcription of the sequence is given in section A.3 in the appendix.

i i

i

i

i

Film Discourse Interpretation 111

3.2.1

Summary of the Extract

The extract begins with a cross-cutting of a black screen and images of a city from above. The display of the city, first directly from above in extreme high shots, later shown by long pans which are repeatedly interrupted by the black screen, is labelled by noises of cars (horns, engine noise, etc.); an indistinct echo is audible for a few seconds. The audio track is interrupted by silence, when the black screen is visible. A second audio track with the noise of blowing wind is always in evidence. The interruption by black screens causes a noncontiguous sequence of shots. The pan is never directly continued when the black screen disappears. Instead, there is a second pan to a specific building on the street. A further voice track, given by a Spanish female voice, is audible (“Abre los ojos. Open your eyes”). The English version is coloured by a Spanish accent. While the last black screen is visible, the song “Everything in Its Right Place”, performed by the band Radiohead, starts in the background. The following shot gives a wide pan of a room with a television running on mute and a man sleeping in a bed. Again, the Spanish voice is audible (“Open your eyes. Open your eyes.”). It is interrupted when the man switches off the CD-player alarm next to his bed. At the time when the CD-player becomes visible in the shot, there is a change between voice-off and voice-on, since the source of the voice becomes visible. The following sequence of images shows the protagonist’s daily routine after waking up. The camera follows him while the music in the background gets louder. When he turns the light on in the bathroom, there is a cut to a closer shot of his face. The camera then goes back to the other room where it catches the protagonist’s reflection in a mirror. The shots show subsequently, but with some small temporal ellipses, his routine in the morning, packing together his things and leaving the flat. The music in this scene plays a special role. While the melody is very calm and rather serves as a backdrop, the lyrics are in the foreground. One sentence (“Everything in its right place”) is repeated several times. The song continues in the following shots and underlines the consecutiveness of the events. It then fades out after a few seconds, when a car is leaving a garage and only the roar of the engine is audible. The following sequence shows the protagonist’s car ride through empty streets. Long pans of the street and abandoned sidewalks alternate with close-up views of the protagonist in his car which emphasise his facial expression. Some of the shots of the street or the traffic lights, for example, are point-of-view shots. The same applies to the image of his watch shown in detail. A very long shot of a completely empty Times Square in New York again underlines the accentuated situation. A zoom in on the protagonist getting out of his car and looking around anxiously follows. Then the camera zooms out to the whole setting. The protagonist starts to run along the streets and

i i

i

112 Analysis

an aerial shot gives a very large view of the setting. His running is shown by a montage of rapidly alternating shots of his face (camera in front of or besides him, partly low angle), of the street in front of him and of the animated neon and television-style signage around him. The images succeed each other rapidly; they are nearly unidentifiable. In detailed analysis, however, it becomes clear that there are also images which do not correspond to the Times Square signs. For example, there is a close-up view of an LP cover and different images of magazines appear occasionally. The scene is accompanied by the sound of a sitar and rhythmic drums. Their rhythm gets faster and the sound intensity grows synchronously to the increasing number of shots. A 360-degree tracking shot from a low camera angle then closes the Times Square sequence and there follows a cut to the flat where the protagonist is lying on the bed. The beginning voice track of a female voice (“Open your eyes. Open your eyes.”) now does not have any noticeable accent, but there is a difference in the pitch of voice. In contrast to the beginning of the scene, the voice track now names the protagonist (“David, open your. . . ”) and identifies him. Again, he switches off the CD-player alarm next to his bed. The sequence then recurs as was shown in the beginning of the sequence: the protagonist gets up, enters the bathroom, etc. In contrast to the sound track in the beginning, it is now a special voice track from off which dominates the scene. However, these shots are not part of the following analysis. 3.2.2

Analysis

The beginning of this extract from Vanilla Sky, which is as well the beginning of the movie, starts with a montage of shots of the city taken directly from above which are interrupted by black screens without any further visual information. These black screens count as interruptions which construct small temporal as well as spatial ellipses, since the pans and camera movements are never directly continued. In contrast to inserts that provide verbal content with background information about the film, for example, these interruptions do not provide further information for the propositional content. The introductory view from above continues as a slow zoom in on a specific building, while the noise of the blowing wind remains the dominant modality in all shots and also underlies the black screens. Furthermore, an indistinct echo is audible until the voice track “Abre los ojos” is heard. The whole sequence of zooming in can be taken as one single event which is described in the logical form in table 3.12. Since there are no conceptual changes within this scene, no further events or sub-events are inferred. However, the propositional description of the eventuality is rather difficult, because there is no concrete action visible. The beginning of the extract can be described as a typical introduction which first shows

i i

i

i

i

Film Discourse Interpretation 113

a very general view of the setting. It can be inferred that the following action happens in a bigger city which cannot be directly identified simply because of the images of the streets. Nevertheless, paratextual information about the film may let the recipient know that it is the city of New York. eπ1 = wake up [v] city/streets (m) [v] pan to the left (n) [v] represented participant: TV (o) [v] male actor in bed (p) [v] alarm (CD player) on bed table (q) [a] noise of blowing wind (continuous) (r) [a] background noise from the street (interrupted) (s) [a] indistinct echo (t) [a] “Abre los ojos. Open your eyes.” (u) p, q, u |∼ wake up (eπ1 ) Table 3.12 At the moment when the voice track becomes more dominant, music also begins and creates a transition to the following shots. Subsequent to a further black screen, a cut follows to a medium close-up of a television in a room. The camera then pans to a man lying in a bed and turning off the alarm; see table 3.13. Thus, the preceding introduction here becomes more concrete in that a character is finally introduced and the setting is more explicitly given. It now becomes clear that the voice track from the beginning is the voice of the alarm which wakes the protagonist up in his bed. The inference process therefore can be described as interpreting the whole sequence as one single event, eπ1 , described as waking up; see table 3.12. The logical form makes visible the dynamic unfolding of the content by the different discourse referents which all play a role within the meaningmaking process. In comparison to the logical forms described in the analysis of the extract from The Lives of Others, the forms here include a larger number of referents. Since the first shots all support the inference of wake up, they have to be included in the description of this first event within the unfolding discourse structure. The evolving context is an important factor for the inference and it becomes evident that units like the shot and the different black screens do not lead to the interpretation of different events here. A first separation then can be made when the voice track is interrupted and the process of waking up as eπ1 is terminated. In this case, there is no cut that could create a small ellipsis or show another perspective on the following action. Instead, the sound track in the background comes

i i

i

114 Analysis

#

15.

16.

shot

shot description

audio track

black screen

music begins

a television in a room showing a black-and-white film (Sabrina (Wilder, 1954))

dominant music, sound track

pan to a man lying in a bed, turning off the alarm

dominant music, sound track: Everything Is in Its Right Place by Radiohead

the man gets up and turns off the TV, walks through the room

sound track: “Everything, everything, everything. . . ”

Table 3.13: Transcription of shot 16 of Vanilla Sky to the fore and focuses attention on the action happening in the video track. The camera pans from and to different entities in the long shot and describes how the protagonist gets up and turns off the TV. This plan sequence represents a very typical description of different small incidents happening in a spatiotemporal sequence and will normally be inferred as one single ‘coarse-grained’ event. The different camera perspectives give a view of several small details, such as the pair of tweezers that the protagonist uses in the bathroom, but these do not play a crucial role within the unfolding story-line. The whole sequence of shots therefore can be described as the protagonist’s morning routine. Its eventuality, which again contains a high number of discourse referents, can be inferred as get up, as shown in table 3.14. This eventuality is thus a consequence of the event described before. Since both eventualities also stand in a temporal succession and the process of waking up can be inferred as immediately following, the relation which fits here is Narration. No other conditions which could override the default Narration-relation are fulfilled here. φN arration(eπ1 ,eπ2 ) ⇒ (eπ1 u eπ2 )

(3.10)

i i

i

i

i

Film Discourse Interpretation 115

eπ2 = get up [v] male actor (v) [v] represented participant: TV (w) [v] mirror in bathroom (x) [v] reflection: protagonist in living room (y) [v] key and documents on a chair (z) [v] bag with stuff in his hands (a) [a] sound track: “Everything is in its right place. . . ” (b) v |∼ get up (eπ2 ) Table 3.14 The common topic shared by the two events has at least the minimal content of something happening in the morning at the same setting. It can be inferred that the first event, waking up, occasions the second one, getting up, as formally captured in the following. (?(eπ1 , eπ2 , λ) ∧ occasion(eπ1 , eπ2 )) > N arration(eπ1 , eπ2 , λ)

(3.11)

It is not possible to find a cause or another condition holding in this context and it can therefore be proven that Result, for example, does not hold here. ¬cause(eπ1 , eπ2 ) ⇒ ¬Result(eπ1 , eπ2 ) (3.12) The only auditory source in this plan sequence is the sound track which, on the one hand, has an accompanying function and, on the other hand, may also play a more important role for meaning construction, since it offers the possibility to find an intersemiotic reference to the visual level. The song text “Everything is in its right place” may be seen as a comment on the described events, if it is interpreted coherently. On the basis of a very typical sequence of events which does not require a very complex inference process, the recipient’s attention can be drawn to the song text itself. It may then come to the fore and an intersemiotic reference may be found between the song and the depicted event of the morning routine. This reference, however, is not very explicit in the filmic text and the song text is rather extended and drawn-out by the melody and rhythm of the song. Its semantic content therefore has to be filtered out of its surroundings rather finely. From the textual perspective, the sound track has a purely accompanying function; there are no other cues which direct attention to this reference. The sound track becomes silent when the scene is cut to a setting outside the building where a car is leaving the garage. On the basis of the events described before, it can be inferred and will be verified immediately that it is the protagonist who leaves the house in the morning. It is thus an event

i i

i

116 Analysis

which is occasioned by the story-line before. Although it is not evident that the setting is the house which was zoomed in on at the beginning of the sequence, the spatial reference is also readily inferred. The following identification of the protagonist assures again this logical consequence in that it verifies that it is him leaving the house with the car. Showing him in this car makes it plausible to infer that this happens in direct succession. In table 3.15, the further unfolding of the discourse is transcribed. The first two shots in this scene for example give the protagonist’s perspective on the streets around him when he is leaving the garage with the car. They display his perception of the world around him. Since the sound does not play an important role here and only gives synchronous noises corresponding to the visual level, the impressions given by the different shots are the only meaning-making entities. Subsequently, the shots alternate between, on the one hand, descriptions of the protagonist in his car, which mainly concentrate on his facial expression, and point-of-view shots of the streets and the environment on the other hand. The whole montage creates a general overview of the depicted situation. Therefore, it can be inferred as the event of a car ride, augmented by a high number of point-of-view shots which, however, do not bring about a separation into further coarse-grained sub-events. Although the focus is sometimes directed to small details within this sequence, as is the case for example in the two shots depicted in figure 3.5, these details give more information concerning the single overall event. The logical form of this event is described in table 3.16.

Figure 3.5: Shots 44 and 45 in Vanilla Sky As already pointed out above, the descriptions here also include more fine-grained incidents of the dynamically unfolding discourse. However, these incidents do not influence the segmentation into several events, but rather count as discourse referents within one logical form. Whereas the analysis of The Lives of Others depicts a sequence of shots which all describe sub-events of one larger event, such as the parallel montage which combines different strands of the story, the shots here create montage sequences which feature only one single event. The three eventualities eπ1 –eπ3 described above can, however, be separated from each other because of the conceptual changes that can be found in the diegesis. Their succession is on the one hand motivated by a cause-effect relationship and on the other hand maintained by a simple Narration-relation. In sum, the

i i

i

i

i

Film Discourse Interpretation 117

shot description

audio track

30.

POV-shot of the street (to the right)

roar of car engine

31.

POV-shot of the street (to the left)

roar of car engine

32.

close-up of man in the car

roar of car engine

33.

the car on the street

roar of car engine

34.

close-up of man in the car, driving

roar of car engine

35.

POV-shot of the empty street and the traffic lights

roar of car engine

36.

medium close-up of the man from the co-driver’s seat perspective

roar of car engine

#

shot

Table 3.15: Transcription of shots 30–36 of Vanilla Sky

i i

i

118 Analysis

eπ3 = drive [v] car on the street (m) [v] male actor in his car (n) [v] street, traffic lights (o) [v] car on Times Square (p) [a] sound of the car engine (q) m, n, o, p |∼ drive (eπ3 ) Table 3.16 different events depict a natural event-sequence. Their SFDRS is given in table 3.17. π0 π1 , π2 , π3 π1 : Kπ1 , π2 : Kπ2 , π3 : Kπ3 π0 : N arration(π1 , π2 ) N arration(π2 , π3 )

Table 3.17: Segmented Film Discourse Representation Structure of the beginning of Vanilla Sky The music in this scene does not play an important role for the meaning construction. It primarily comes to the fore at the end of this scene, which describes a long shot of the car driving on a completely empty Times Square in New York and approaching the position of the camera, which then turns through 360 degrees (see shot 46 and the stills in lines 47 and 48 in the transcription in the appendix). At that time, however, the music highlights the specificity of the described situation in that it becomes a very rhythmic and more dynamic melody with growing intensity and volume. The camera first zooms in on the protagonist and focuses on his anxious facial expression and then zooms out again into a long shot from above. The shot depicts the end of eπ3 and therefore is still associated with the process of driving the car. A separation can then be made when the camera focuses on the protagonist and the music gets louder. The camera tracking as a filmic design element may support the transition between the two events and emphasises the emotional portrayal of the protagonist as indicated by his facial expression. The dominant music and the camera movements highlight the threatening situation and indicate its exceptionality, but do not effect the basic discourse structure construction.

i i

i

i

i

Film Discourse Interpretation 119

They still play an important role for the establishment of the following scene which is inferred as eπ4 , see table 3.18. eπ4 = panic [v] protagonist at Times Square (m) [v] camera: 360-degree tracking shot (n) [v] camera: zooming out (o) [v] signs (p) [v] CD, magazine covers (q) [v] photographs (r) [a] sitar, rhythmic drums (s) [a] volume: increasing (t) [a] rhythm: getting faster (u) m, n, t, u |∼ panic (eπ4 ) Table 3.18 This eventuality contains as well the large number of different, partly nonidentifiable shots and images which succeed each other rapidly; see figure 3.6 and section A.3 in the appendix.

Figure 3.6: Selected frames from the end of the extract from Vanilla Sky Whereas the point-of-view shots of the signs and buildings around Times Square have a concrete textual motivation, namely the description of the protagonist’s perception, the inserted images cannot be directly related to the other discourse segments. Due to the extremely fast alternation of the images, it is in most cases hardly possible for a viewer to see and identify them individually. Sometimes, even the focus is missing, so that the content of the image stays diffuse and indistinct. It is thus rather the interplay of these impressions which establishes the overall meaning of the montage. The accompanying music underlines the coherence of the different shots and supports at the same time the dynamic of this montage. Its intensity grows according to the increasing volume and rhythm.

i i

i

120 Analysis

At the end of this montage, the protagonist stops and begins to scream while the camera is tracking around him. This tracking shot is then abruptly stopped and there follows a cut to the protagonist lying on his bed in the flat which was introduced at the beginning of this extract. It can be inferred that the scene before was a dream sequence from which he now wakes. The voice-over gives a further indication for this projection: “Open your eyes.” The logical form of the event is given in table 3.19. eπ5 = wake up [v] protagonist on bed (v) [a] female voice: “Open. . . ” (w) v, w |∼ wake up (eπ5 ) Table 3.19 Since the shot thus strongly suggests that the protagonist dreamed the events that happened before, they have to be interpreted differently. This change in the interpretation gives a good example of the Context Change Potential (CCP), which was described in detail in section 2.2.2. At the beginning of the extract, there are no concrete cues that point to a dream or projection. Only the abrupt cut between the long montage in Times Square and the setting of the flat at the end marks the end of this dream sequence and establishes the unfolding of the now following discourse. By this explanation, it can be inferred that the projection has to be subordinated to the actual event of waking up. Thereby, the Explanationrelation has an important function for the overall discourse in that it explains the development of the story-line and thus works ideationally. The first inference that the events seen before follow each other in a sequence of Narration-relations can indeed be kept up, but the whole SFDRS of the extract has now to include the subordination. The depiction of this SFDRS is given in table 3.20. Here, it becomes visible that the Explanation-relation actually holds between eπ0 , representing the eventualities eπ1 –eπ4 , and eπ5 , which is the second waking up in this sequence. π5 as the last label added to this discourse structure can consequently be seen as a ‘discourse pop’ (cf. Asher and Lascarides 2003: 229f.; see section 2.2.2), which considerably affects the interpretation of the structure. Similar to the example described in section 2.2.2, it results here from choosing the updated SFDRS according to those attachment sites that maximise the overall coherence and explain dependencies and circumstances within the story-line. The preferred structure is thus the one with the rather exceptional subordination, which is again depicted in the graphical representation in figure 3.7. At the end of the extract, a repetition of the shots shown in the beginning follows, which depict the morning routine of the protagonist. The main

i i

i

i

i

Film Discourse Interpretation 121

π0 π 0 , π5 π1 , π2 , π3 , π4 π1 : Kπ1 , π2 : Kπ2 , π3 : Kπ3 , π4 : Kπ4 N arration(π1 , π2 ) π0 : N arration(π2 , π3 ) π0 : N arration(π3 , π4 )

π5 : Kπ5 Explanation(π 0 , π5 )

Table 3.20: Segmented Film Discourse Representation Structure of the extract from Vanilla Sky π0

π0

Explanation

π5

Narration π3

xt Ch a

π2

Conte

Narration

nge

π1

Narration π4 Figure 3.7: Graphical representation of the discourse structure in the extract from Vanilla Sky

i i

i

122 Analysis

difference lies on the auditory level: on the one hand, the voice track from the alarm differs from the one in the beginning mainly in its timbre and the missing Spanish accent. Furthermore, there is a second voice track as a voice-over which accompanies the following shots and which directly refers to the dream sequence. Its source first remains unknown and is not resolved until the setting changes. 3.2.3

Conclusion

As has become visible throughout the analysis, the unfolding discourse structure of the extract from Vanilla Sky differs from that of The Lives of Others in that it features only five different events, although the two extracts have almost the same length. This is a result of the different design elements used in the two films. Whereas The Lives of Others uses the cross-cutting montage to compare and juxtapose two different strands of the plot, the different shots and changes in the perspective in Vanilla Sky always represent one single inferrable event. The third part of the extract describes, for example, the car-ride scene, which is mainly characterised by the alternation of close-up shots on the protagonist and point-of-view shots from his perspective. This alternation puts emphasis on pecularities in the streets which indeed influence the general atmosphere and the interpretation of the scene, but they do not allow the separation of further events or sub-events. Therefore, they are not described as contributing further to the logical structure to the discourse. Nevertheless, this textual unfolding directly influences the development of the story-line in that it supports the display of these small details within the event-structure. An important aspect within this analysis is the CCP of filmic discourse and its influence on the interpretation of the structure during the update process. Whereas the beginning of the extract is normally inferred as a sequence of relations upholding an apparently simple story-line, the last label added to the discourse structure changes the interpretation of this sequence considerably. The repetition of the unfolding story makes clear that the preceding structure depicts a projection or dream sequence, which, as a consequence has to be subordinated to the form following. The move from the ‘mental world’ of the protagonist (cf. Bateman 2007: 43) to the general story-line of the film is created by a single transition from the shot at Times Square to the close-up of the protagonist in his bed which, on the textual level, depicts the point of change during the update process, the ‘discourse pop’ (see above). The result is the pragmatically preferred SFDRS given in table 3.20 and figure 3.7, which stands in contrast to the previously developed structure in table 3.17, for example. This example not only illustrates the dynamic unfolding of the discourse structure which is clearly depending on new information added to the discourse, but also gives evidence for a specific filmic technique which

i i

i

i

i

Film Discourse Interpretation 123

is often used to describe levels of the story that do not cohere in space and time, such as flashbacks, dreams and memories (cf. Bateman 2007: 45). These kinds of projections are sometimes embedded without a noticeable change in the description (in contrast to clear indications, such as, for example, inserts or the change of colour or visual quality) and the move from one level to the other has to be inferred because of the logical relations holding between the different events, as given in the example described here. Although the description of the sleeping protagonist is a very usual indication of establishing a dream sequence, this description is included in the projection and therefore does not lead to the right inference of this sequence. Only its repetition makes it possible to find a causal relationship between the discourse segments, which gives the missing explanation for the described circumstances. The preferred discourse structure thus gives evidence of how the recipient is guided by the text in order to find the right interpretation by gaining maximal coherence (cf. Asher and Lascarides 2003: 234). This makes it possible to describe a filmic montage specificity from the discourse analytical perspective and therefore illustrates the clearly textual composition of the unfolding story in this extract, which can then be used for further interpretive work.

3.3 Am´elie (2001) The film Am´elie whose original french title is Le Fabuleux Destin d’Am´elie Poulain (The Fabulous Destiny of Am´elie Poulain) is a romantic comedy directed by Jean-Pierre Jeunet. The main actress, Audrey Tautou, plays a waitress in Paris, who decides to improve the lives of the people around her. While for example taking revenge on the authoritative greengrocer in her neighbourhood and pairing her friend Georgette with Joseph, she herself struggles with her own solitude and does not have the heart to approach the young man she is secretly attracted to. Finally, with the insightful help of her neighbour and good friend, she overcomes her shyness. The movie mainly stands out due to its exceptional graphical illustration. The intensive play of colours and some extraordinary cuts and editing techniques are accompanied by a remarkably designed audio and sound track. 3.3.1

Summary of the Extract

The extract to be analysed is from the second third of the film (01:22:50–01:24:17) and depicts a sequence in which Am´elie’s neighbour and concierge, Mrs. Wells, receives a letter from her missed, but long-deceased husband. It contains only three different shots showing the

i i

i

124 Analysis

entrance hall and her flat’s living room. A detailed transcription of this scene is given in section A.4 in the appendix. While Mrs. Wells is cleaning the entrance hall, the postman hands over a letter to her. She opens the letter and reads it while slowly returning to her flat. A voice-over from a male voice reads out a short message from customer services that a mailbag has been recovered from a plane crash in 1969 (about 30 years before the story time). The director of the customer services now wants to forward a letter addressed to her. There follows the first cut between the long shot in the entrance hall and a second shot now depicting Mrs. Wells sitting on her sofa and reading the letter. The camera is slowly zooming in. A second male voice-over replays the letter’s wording: “Darling Mado, I’m in exile. I can’t sleep, can’t eat. I think of you endlessly. I know I’ve made the biggest mistake of my life. I turned down that woman’s money. If all goes well, I’ll soon be able to afford a house. I dream of better times ahead when you’ll forgive me and join me here . . . one orange-coloured day. Your ever-loving Adrien.” The voice-over is accompanied by different and partly undefinable noises such as for example a trumpet sound or the rush of water. They do not refer to any visible sources in the shots and therefore are generally off-sounds which, at first sight, are not connected to other modalities in the shot. The third and last shot in this sequence then focuses on Mrs. Wells and how she is kissing and cleaning a framed picture of her husband. Piano playing starts in the background. 3.3.2

Analysis

As the following analysis will outline, the interpretation of the chosen extract and its rather complex discourse structure depends on several factors. On the one hand, the voice track, arranged by two different voiceovers, plays an important role for the meaning-making process. On the other hand, the audio track in the background, presenting a variety of sounds, is of further importance and interest for the interpretation. On the visual level, there are no strong particularities. In comparison to many other movies, the shots in this example appear to be comparatively long, but in relation to the film as a whole, both longer shots and sequence shots are common. Hence, the analysis will show that the logical forms of the discourse describe eventualities on the level of the shot which can be inferred from conceptual changes within one shot. Furthermore, it will become clear that the two voice-overs represent subordinate discourse structures which are dominated by the preceding logical forms. This subordination is not established until the letter, as the dominant motif in this sequence, has been handed over to the main character of the scene, Mrs. Wells.

i i

i

i

i

Film Discourse Interpretation 125

The first logical forms of the extract’s beginning are given in table 3.21. They outline two different events happening in the first shot, which are separated from each other due to the narrative processes which can be recognised. eπ1 = dialogue eπ1a = say [v] entrance hall (m) [v] male actor: postman (n) [a] “How’s it going?” (o) n, o |∼ say (eπ1a )

eπ1b = say [v] entrance hall (p) [v] fem. actor: Mrs. Wells (q) [a] “It’s always better. . . ” (r) q, r |∼ say (eπ1b )

eπ2 = handover eπ2a = say [v] entrance hall (s) [v] male actor: postman (t) [a] “This is for you.” (u) t, u |∼ say (eπ2a ) eπ2c = hand over [v] entrance hall (y) [v] male actor: postman (z) [v] fem. actor: Mrs. Wells (a) [v] letter (b) z, a, b |∼ hand over (eπ2c )

eπ2b = say [v] entrance hall (v) [v] fem. actor: Mrs. Wells (w) [a] “Me?” (x) w, x |∼ say (eπ2b )

Table 3.21: The logical forms of the first eventualities in the extract from Am´elie Since the setting and the main character are already established and well known, an identification here is not needed. The first eventuality can be inferred as a dialogue between the two characters in this setting who talk to each other. Their voice tracks lead to the formulation of the two sub-events of saying, which directly refer to the characters’ action and which are formulated as eπ1a and eπ1b . The second eventuality continues as a dialogue between the two characters, but is influenced by the subevent of handing over (eπ2c ) the letter to Mrs. Wells. The second part of this dialogue concentrates on the delivery process, introduced by “This is for you”, and can therefore be separated from the first part. The letter itself plays an important role for the unfolding narration and comes to the fore as described in the logical form of the last sub-event in table 3.21.

i i

i

126 Analysis

The two main events, the dialogue and the handover, stand in a spatiotemporal sequence and generally share the common topic of the letter or its delivery. The relation which can be inferred here is thus the default Narration-relation which is supported by the fact that the second eventuality is motivated by the first one: i.e., eπ1 occasions eπ2 . (?(eπ1 , eπ2 , λ) ∧ occasion(eπ1 , eπ2 )) > N arration(eπ1 , eπ2 , λ)

(3.13)

The same applies to the following eventuality which is still part of the first shot and which shows Mrs. Wells now as the only character, opening the letter and reading it. Its logical form is described in table 3.22. eπ3 = open [v] entrance hall (c) [v] fem. actor: Mrs. Wells (d) [v] letter (e) d, e |∼ open (eπ3 ) Table 3.22 The three eventualities depict a so-called natural event-sequence which is established by Narration-relations and whose overall common topic is still the letter. The SFDRS of the beginning of the extract is therefore a relatively simple structure and can be described as given in table 3.23. π0 π1 , π2 , π3 π1 : Kπ1 , π2 : Kπ2 , π3 : Kπ3 π0 : N arration(π1 , π2 ) N arration(π2 , π3 )

Table 3.23: Segmented Film Discourse Representation Structure of the beginning of the extract from Am´elie However, this SFDRS then has to be modified with regard to the following eventualities, which can be inferred mainly due to the voice track as the most dominant modality. Whereas the dialogic voice tracks in the beginning are part of the overall description of the processes described in the logical forms above, the voice track in the following is a voice-over which has a different function. Here, it represents a further discourse structure which is on the one hand subordinated to the one described in

i i

i

i

i

Film Discourse Interpretation 127

table 3.23, but on the other hand runs parallel to the continuing processes depicted in the image. It is thus a twofold structural unfolding which has to be described on different levels, although there are no explicit changes in the granularity of description on the visual level and the voice seems to dominate the whole setting. There is thus firstly the structure which depicts the protagonist in the entrance hall, reading the letter and thereby moving slowly towards her flat. Secondly, there is the event of the unfolding voice track which describes a significant actuality that happened a long time ago. The voice track includes several sub-eventualities which illustrate the verbal content of the letter and which can be described as further sub-events. The graphical representation in figure 3.8 demonstrates these parallel unfolding structures.

π3 Narration π0

π4

π5 “Dear Mrs. Wells”

“we recently recovered”

Figure 3.8: Graphical representation of the first voice track in the Am´elie extract The first structure, which has been labelled with π4 in figure 3.8, can be described as the logical form in table 3.24, which is relatively similar to eπ3 . There are only small movements of the female character entering her flat which are followed by a similarly slow-moving camera. In the background, there is gentle whispering audible, which can be inferred as Mrs. Wells’ voice reading the letter. The wording of the letter as the voice track is repeated in the following.

i i

i

128 Analysis

eπ4 = read [v] slowly moving camera (f) [v] entrance hall (g) [v] female actor: Mrs. Wells (h) [v] letter (i) [a] gentle whispering (j) h, i, j |∼ read (eπ4 ) Table 3.24 Dear Mrs. Wells, we recently recovered a mailbag that was lost in a plane crash on October 12th 1969 on Mont Blanc. We are forwarding the enclosed letter to your address. Please accept our apologies for the uncustomary delay. Jacques Grosjean, Customer Services. With regard to the verbal content explicitly given here, its logical forms can naturally be described using Asher and Lascarides’ original Segmented Discourse Representation Theory (SDRT) framework, since this is simply a verbal text. This is shown for the first sentence in table 3.25. The eventualities represent the second unfolding sub-structure, as depicted in figure 3.8. They can also be related to each other by the set of verbal discourse relations described in the SDRT framework by Asher and Lascarides (2003) and therefore are labelled with π5a or π5b . There is, for example, a cause-effect relationship between the fact that the letter was found and then, as a consequence, forwarded to the protagonist. Furthermore, the exact date in the relative clause at the beginning clearly defines the temporal dimensions of this discourse. π5a , π5b

π5a

eπ5a = recover we (x) : mailbag (y) recover (eπ5a , x, y)

π5b

eπ5b = being lost z=x : plane crash (a) M ont Blanc (b) being lost (eπ5b , z)

Table 3.25: The logical form of the first sentence of the first voice-over in the extract from Am´elie in original SDRS-form. This shows that the main difference in the description of the logical forms for verbal discourse lies in the formulation of the eventuality in the

i i

i

i

i

Film Discourse Interpretation 129

last line. Asher and Lascarides use a distinct description of the propositional content in terms of predicates followed by discourse referents, as in being lost (eπ5b , z). The formulation of the eventuality in this book differs from this notation in that it highlights the defeasible result of reasoning about the intersemiotic meaning constructed from the various modalities and their interplay. In film interpretation, these predicates are not simply provided by a compositional semantics. Moreover, in the present example, the logical forms simply illustrate the verbal content of the voice track without any further information such as timbre, volume or quality of the voice. It is thus a mono-modal description of how meaning can be inferred due to the very explicit semantic content. On the basis of the given filmic context, however, the viewer in total has considerably more background information and is aware of the letter’s original source. It is de facto the main protagonist, Am´elie, who has composed and sent the letter to her neighbour to please her (which is shown in detail earlier in the film). The actualities from the letter text are thus fictitious and only pretend to sustain the causal relationship described above. Actually, within the story-world, there has not been a plane crash and no letter has been found. Since the context has already delivered the necessary information to verify and evaluate the propositional content, which here comes to the fore, the viewer is able to interpret the constructed story-line with direct reference to what has happened before. The voice track as a whole operates on a different level of diegesis which cannot be related to the first unfolding discourse structure. It is thus not possible to infer a discourse relation which holds between the two discourse structures. In contrast to the montage in Vanilla Sky, the projection here is not explicitly established by one of the discourse relations described in section 2.2.4, but is instead incorporated in the description of two parallelised entities within the overall discourse structure of the movie.5 This structure unfolds as given in table 3.26. The voice track ends by identifying the cover note’s supposed author, who is named in the last sentence. Here, the intersemiotic meaning construction is visible: The identification of the addresser operates on the auditory level, but is based on two different modalities, the male voice and the spoken language, and its semantic content, the name. This meaning construction takes place only on the level of the second discourse structure, the unfolding voice track, and does not have any relation to the parallel structure. It again becomes visible that two different strands of the discourse here are combined within a sequence, which, in the following, is elaborated by further devices on the auditory level. With the following first cut within this scene, the setting changes to the protagonist’s flat. 5 This then corresponds to the paratactic organisation possible for projection as purposed by Bateman (2007) and Bateman and Schmidt (2011).

i i

i

130 Analysis

π0 π1 , π2 , π3 , π4 π1 : Kπ1 , π2 : Kπ2 π3 : Kπ3 , π4 : Kπ4 N arration(π1 , π2 ) N arration(π2 , π3 ) N arration(π3 , π 0 ) π4 , π5

π0 : π0 :

π4 π4 : Kπ4

π5 π5a : Kπ5a , π5b : Kπ5b , π5c : Kπ5c , π5d : Kπ5d

Table 3.26: Segmented Film Discourse Representation Structure of the beginning of the extract from Am´elie At the same time, the first voice track stops and a second one with a different voice starts. This voice-over directly succeeds the first one, but differs mainly in the general voice quality and its timbre, so that there is a contrast between the two voices. Consequently, another speaker is identified. The second discourse structure (Kπ5 ) thus ends with the last eventuality, which can be inferred from the verbal content of the voice track, whereas the first structure (Kπ4 ) continues unfolding. Although there is a small ellipsis because of the fact that the protagonist has entered her flat and sat down on her sofa, it becomes evident that it is still the same event of reading the letter. The change of the setting here does not bring about the separation of a further event. Instead, it is the change of the voice track which now makes clear that the protagonist is reading the second, forwarded letter. The verbal content of this second letter is again a sub-structure which evolves as a parallel subordination. Simultaneously, various sounds and noises are audible in the background. These are recorded sounds which do not have a recognisable source on the visual level. From this shot alone, it is not clear whether the character hears the sounds, whether they come from her imagination or whether they are filmic specificities or design elements. In table 3.27, the transcription depicts the different noises and sounds of the audio track.

i i

i

i

i

Film Discourse Interpretation 131

#

shot

audio track

spoken language [2nd voice-over:] Darling Mado, I’m in exile. I can’t sleep, can’t eat. I think of you endlessly. I know I’ve made the biggest mistake of my life.

6.

trumpet sound (fanfare) — drumming — rush of water — bellringing

7.

chirping birds — drumming — chirping birds — baby crying — rush of water — chirping — orchestra

I turned down that woman’s money. If all goes well, I’ll soon be able to afford a house.

8.

drumming — bellringing — chirping

I dream of better times ahead when you’ll forgive me and join me here. . .

9.

chime of a bell — chirping — ticking

. . . one orangecoloured day. Your ever-loving Adrien.

Table 3.27: Transcription of the second shot of the extract from Am´elie Since the noises do not show any reference either to the voice track described above or to the description of the primary story-line, they have to be depicted as a further, third structure unfolding synchronously; see figure 3.9. The three structures do of course influence each other with regard to the total meaning construction, but they cannot directly be related via discourse relations of the kind being developed here. They illustrate different diegetic levels which create the overall, but specific atmosphere. Nevertheless, a motivation for mixing up these structures is not visible. The situation within the story-line does not account for background noises in this vein nor does the propositional content occasion sounds like these. What can be suggested, however, is that these sounds may be recognised by the viewer as being repeated from a sequence before. They are used very similarly as background sounds in the scene where Am´elie first

i i

i

132 Analysis

π3 Narration π0

π4

π5a “Dear . . . ” π5b “we recently” π5c

Narration

π6 Narration

π 00

—

π7

π8

“Darling Mado”

[trumpet sound]

Figure 3.9: Graphical representation of the second voice track in the extract

i i

i

i

i

Film Discourse Interpretation 133

reads the various letters she stole from her neighbour and later copies and cuts up in order to send a new one to her. Each noise repeated here accompanies the extracts from several letters which are given as a voiceover while the protagonist is reading them. The repetition in the example extract taken for analysis is thus a reproduction and reduction of the sounds in a form similar to the reproduction of the letter itself. The sounds seem to be combined equally to the sentences of the various letters which Am´elie glued together. In a more detailed view of the synchronously unfolding structures, the intersemiosis of the different modalities also allows the association of some interrelations between the filmic elements: The information that the writer of the letter is in exile and the relatively dominant sound of a trumpet can be brought together if the latter is associated with military music for example. The same happened in the scene before where the trumpet playing can be interpreted as definitely accompanying the letter that informs the receiver about the exile situation. The off-sound can thus operate supportively with regard to particulars of the content of the letter which is the subject matter of the other sub-structure. The eventualities which can be inferred from the voice track as a sequence of propositions are given in table 3.28.

eπ7a−c eπ7b eπ7c eπ7d

eventualities be in exile, ¬sleep, ¬eat think of make mistake turn down

eπ7e eπ7f

go well afford

eπ7g eπ7h eπ7i−k

dream of forgive join

synchronous sound trumpet sound (fanfare) drumming, rush of water bell-ringing chirping birds, drumming chirping baby crying, rush of water, chirping orchestra drumming bell-ringing, chime of a bell, chirping

Table 3.28 None of the sounds directly refers to these predicates, but they do allow a few associations which influence the general atmosphere of the scene. Basically, the sounds are in all cases totally out of the context of the letter’s diegetic level as well as the one describing the female protagonist. Imported without any fade or overlay, they do not seem to be heard or imagined by the protagonist, but are likely to create a certain atmosphere for the viewer. Similar to the situation of the first voice track before,

i i

i

134 Analysis

the three structures are textually juxtaposed and unfold independently. The SFDRS of this part of the extract representing these parallelised substructures is given in table 3.29. π0 π4 , π 00 π4 : Kπ4 N arration(π4 , π 00 ) π6 , π7 , π8 π0 : π 00 :

π6 π6 : Kπ6

π7 π7 : Kπ7

π8 π8 : Kπ8

Table 3.29: Segmented Film Discourse Representation Structure of the three parallel unfolding structures in the Am´elie extract This parallel unfolding stops when the letter is finished by a typical closing. Simultaneously, the sound track ends with the noise of chirping and fades out slowly. The camera as an element of the first structure still zooms in on the protagonist very slowly and then stops moving as well. The event at the end of this sequence can thus be inferred as the last eventuality of the main diegetic level, as given in table 3.30. The now audible sound is a diegetic sound caused by the protagonist and is therefore inferred within eπ9 . eπ9 = weep [v] living room (m) [v] female actor: Mrs. Wells (n) [v] letter (o) [a] sound of weeping (p) n, p |∼ weep (eπ9 ) Table 3.30 A cut to the last shot of this sequence follows where Mrs. Wells is standing in front of a framed picture which she kisses. It can be inferred that eπ9 (weep) occasions the last eventuality added to the overall discourse structure, eπ10 (kiss). The relation which holds between the two events is

i i

i

i

i

Film Discourse Interpretation 135

thus most plausibly Narration. (?(eπ9 , eπ10 , λ) ∧ occasion(eπ9 , eπ10 )) > N arration(eπ9 , eπ10 , λ)

(3.14)

Since there is no other meaning postulate which gives reason for the inference of another discourse relation, Narration here is the default and preferred update and so is not overwritten. At the same time, the intersemiotic meaning construction in this shot enables an identification of the letter’s author. It is inferred that the voice of the voice track refers to the framed picture in the last shot and that the person in the picture is likely to be the intended writer. Due to context knowledge from circumstances described earlier in the film, the writer is already introduced as the beloved person. The piano playing in the last added eventuality does not play a significant role for the meaning construction, but leads over and fades to the following scene where it continues for a while. Although the two scenes depict different settings and story-lines, they are connected to each other via the music. However, in both situations, this has only an accompanying function. 3.3.3

Conclusion

As has become apparent throughout the analysis, the discourse structure of the extract from Am´elie is relatively complex and the two subordinations create a dense network of interconnections. As a complete overview of the entire extract, its overall and final discourse structure is depicted in table 3.31. Although the individual discourse segments are mainly related to each other by Narration-relations, the subordinated structures also include a number of relations for verbal discourse which are not described in detail in table 3.31. However, in this case and in contrast to the analysed structures in the preceding sections, it is possible to describe several sub-events of the subordinated structures. The most dominant sensory channel in this extract is the auditory level with the voice tracks. The video track, in contrast, displays the typical graphical illustration of the movie in general. Relatively small camera movements such as the slow zoom on the scene’s protagonist mainly sustain the film’s dynamic. Since the sequence focuses on the letter as a general motif, the zoom rather supports this focus and does not direct attention to something else. No further editing techniques or particularities are on hand on the visual level and thus do not exert any important influences on the eventualities. The three shots are relatively long and contain various events which have to be described separately.

i i

i π0 :

π0

π6 π6 : Kπ6

π9 : Kπ9 , π10 : Kπ10 N arration(π6 , π9 ) N arration(π9 , π10 )

π 00 : π7 π7 : K π 7

π8 π8 : K π 8

π5 π5 : Kπ5

Table 3.31: Final Segmented Film Discourse Representation Structure of the extract from Am´elie

π0 :

π6 , π7 , π8 N arration(π4 , π6 )

π4 π4 : Kπ4 Narration(π4 , π 00 )

π4 , π5 , π6 , π7 , π8 , π9 , π10

π1 , π2 , π3 , π 0 π1 : Kπ1 , π2 : Kπ2 , π3 : Kπ3 Narration(π1 , π2 ) Narration(π2 , π3 ) Narration(π3 , π 0 )

i

136

i

i

i

i

Film Discourse Interpretation 137

As has already been suggested in general, the audio and sound tracks are designed remarkably within the whole film. The extract exemplifies this specificity from two different perspectives: On the one hand, the dominant voice tracks account to a considerable extent for the meaning potential of the whole extract. On the other hand, the arrangement of the residual audio track in the background highlights the film’s specific characteristics. Especially the use of various sounds, noises and voices in this extract makes evident how and to what extent the auditory level either influences the meaning construction or, in contrast, is deployed totally in counterpoint. The overall discourse structure unfolds due to the establishment of two sub-structures Kπ0 and Kπ00 , which are established by the main diegetic level. In table 3.31, these two subordinative structures and their dependency are clearly depicted. It becomes evident that both auditory levels unfold independently, but are, however, juxtaposed. They are thus textually connected to create a coherent and meaningful text, albeit one drawing on a somewhat richer source of interconnections than simply discourse relations. The various sound elements support the narration, although they are only partly connected to the actual context of the main diegetic level. Therefore, they can rather be seen as strategies of design and creativity which diversify the textual composition and which are made part of the multimodal configuration to enrich the general constitution of the discourse. They interfere with the discourse referents on the main diegetic level, but do not stand in relation to the conditions of inferring the eventualities. However, they have to be classified as a further structure of the discourse which unfolds independently. They may effect connotations and associations which exceed the particular meaning potential of the extract. As has already been pointed out in section 3.3.2, a conceivable association would be the reference to a situation at war while hearing the trumpet sound. The noise of a crying baby and chirping birds in the end stand in contrast to the rather threatening sounds in the beginning and underline the promising message of the letter. The sounds can thus be understood as a kind of illustration of the content of the letter on the auditory level whose realisation can be manifested on the textual level. With regard to the film as a whole, the audio track does not only refer back to the use of these sounds in a scene before, but is also representative of the general and in any case exceptional design of the film’s auditory level. Sounds and noises are often part of the story-line. For example, before Am´elie meets Nino, the male protagonist, he collected and taped other people’s laughs. In another scene, Am´elie hears an old record playing whose source is not visible for a long time. In the end, it becomes evident that an old blind man is listening to a gramophone which plays the music for the whole metro station. The sounds are diegetic sounds which do not

i i

i

138 Analysis

come from off very often, but which are used as a background pattern that creates a sometimes surreal filmic atmosphere. As is the case in the current example, these specific cinematic devices give voice to the artificial character of the film. Am´elie received enthusiastic reviews concerning its magical atmosphere and the overall imagery (cf. Mitchell 2011, NOvember 2). Besides the graphical illustration and the diversified editing techniques, this atmosphere is also created by the different sounds, which not only correspond to and accompany the montage scenes, but also establish individual symbolic patterns for which the analysed scene is one example.

3.4 Requiem for a Dream (2000) Directed by Darren Aronofsky, Requiem for a Dream is an adaptation of a novel by Hubert Selby, Jr., and was first released in the year 2000. The film portrays different forms of addiction and self-delusion in the lives of the protagonist Harry (played by Jared Leto), his mother Sara (Ellen Burstyn) and some friends. The chosen film extract is taken from the film’s beginning. 3.4.1

Summary of the Extract

As has already been described in section 2.2.4, the movie starts with a replay of a TV infomercial. This replay is cross-cut with the opening credits of the film as white letters on black background which give a visual contrast to the replay. The replay is stopped abruptly and the image cuts to a close-up of a connector pulled out of a socket. In the following shot, an older female person and a younger male person are talking to each other about the TV standing in the living room. The dialogue identifies the two persons: “Harold, please, not again the TV.” — “Aw, come on, Ma.” Both characters are named by each other, although Harold only names his mother with “Ma”. A split screen follows, which is established by a wipe to the left when the woman is leaving the room and closing the door. Both screens show different activities in temporal sequence; the audio and voice track is highlighted by gentle music in the background. Sometimes a point-ofview shot is used on the left side of the split screen showing the mother’s perspective while observing her son through the keyhole. He tries to take away his mother’s TV, but it is chained to the heater. While Harold is shouting at his mother, she takes off a key from her neck and passes it under the door. When the mother is talking about the chain, the second screen shows a close-up view of the keyhole from Harold’s point of view. The split screen is dissolved by a wipe to the right, as Harold is leaving the room with the TV table. Simultaneously, the music in the background

i i

i

i

i

Film Discourse Interpretation 139

stops. The mother is focused in her room and shown in detail in a very close-up view. In the next shot, the letters of the film title are shown tumbling down in a vertical wipe and dispersing to a black screen, accompanied by a noise of a closing gate and strong strokes of a violin bow which continues with the following images. Harold is shown with the TV table in front of the door of the room he has just left. There he meets his friend, Tyrone, who is waiting for him. The music in the background gets louder and highlights the following sequence of shots which show the transportation of the TV. The images are interrupted by insertions of credits. In contrast to the close-up views before, the scene is now often shown in long shots. The insertion “Summer” tumbles down with a vertical wipe. The following alternation of images shows an ongoing scene; partly in detail with close-up views of the protagonists, partly in long shots giving more background and context information. Again, there is a hard cut between this scene and the images thereafter with the sound fading out. The last part of the extract represents a montage of images and sounds, cut in rapid succession, but without any direct continuity. The shots show the preparation and intake of drugs: an image of a lighter, a bubbling fluid, an extreme close-up of drawing up the injection, the dilation of a pupil. The last shot of the extract shows a long shot of a room where the two protagonists are listening to music and dancing. 3.4.2

Analysis

Although the film extract uses discontinuous effects such as the difference between the replay and the film itself or the split screen which separates two different strands of the plot from each other, it is a coherent text excerpt that introduces the main characters of the story. Its coherence can be proven by the following analysis of the logical forms of the segments and the combination of the logical forms into discourse structures. To show this, the first 50 seconds will be analysed initially. The transcription is given in table 3.32. #

1.

shot

shot description

TV replay with inserts

audio track applause, background noise of the TV show

spoken language “Juice for Tappy. Oah. Juice for Tappy!”

i i

i

140 Analysis

2.

3.

credits

TV replay with inserts

4.

credits

5.

TV replay, the show master

6.

TV replay with inserts

7.

credits

8.

TV replay, the show master

applause, background noise of the TV show applause, background noise of the TV show applause, background noise of the TV show applause, background noise of the TV show applause, background noise of the TV show applause, background noise of the TV show applause, background noise of the TV show

“Ooaahh.”

“Juice, juice, juice. Juice by you, juice by you.” “Thank you, thank you, thank you.” “We got a winner. I said we got a winner.” “We got a winner. We got a winner.”

“Our first winner is a flight attendant.”

“Welcome Mary Kellington!”

i i

i

i

i

Film Discourse Interpretation 141

applause, background noise of the TV show applause, background noise of the TV show

9.

TV replay, candidate of the show, subtitle with name

10.

TV replay, candidate and show master, subtitle with name

11.

a connector pulled out of a socket

sound of pulling out the connector

“Ooaahh. . . ” (stopped abruptly)

12.

an older female and a younger male person in a living room, talking about the TV

steps on the ground, sound of closing the door

“Harold please, not again the TV.” — “Aw Ma. Come on, Ma.”

“Juice by Mary, juice by Mary!”

“Juice by Mary, juice by Mary! Ooaahh.”

Table 3.32: Transcription of the beginning of Requiem for a Dream (Aronofsky, 2000) The first shots of the extract show typical aspects which characterise a TV infomercial. They depict unstructured analytical processes with several actors, which are interpreted as parts of a whole that is not represented in the shot (cf. Kress and van Leeuwen 1996: 104). The inserts of short slogans within these shots are synchronous to the audio track that replays the ongoing applause and jubilation of the participants of the show. The visual quality of the shots is rather poor and stands in direct contrast to the alternating shots which show a black screen with white letters that give information about the production companies of the film, Artisan Entertainment and Thousand Words. The audio track is continued from the shot before and in no way corresponds to the visual level. The alternating sequence can thus be interpreted as showing several different discourse structures which unfold on different levels of description. Since, on the

i i

i

142 Analysis

one hand, the shots of the infomercial are interpreted as being shown on TV, it can be inferred that there is probably someone watching this replay. This event of watching is then again located on another level of the diegesis. On the other hand, there are the opening credit shots which give background information about the film as a whole, but do not refer to the event shown in the alternating shots. Thus, they cannot be directly associated with the TV replay, although the black background fades in slowly and the shots following overlay the preceding ones. In contrast to the black screens inserted in the extract from Vanilla Sky, the credit shots here give clear verbal information. Due to general knowledge of how films normally begin, it is possible to infer that this is typical meta-information which appears at the beginning of the film. It is not to be interpreted as events of the main diegetic level, but is instead organised on a different text level which unfolds as a further discourse structure. The alternation ends with a shot of an electrical connector, which then finally is interpreted as depicting the film’s actual plot because of its visual quality and the introduction of the main protagonists in the shot following. As figure 3.10 suggests, there are thus three different discourse structures unfolding at the beginning of the extract: the first structure which depicts the opening credit sequence starting at the very beginning, the replay structure which itself features shots from different perspectives and inserts, and the parallel unfolding structure of the film’s story-line which is first realised and comes to the fore in shot 11.6 The event of watching is indeed not depicted in the images, but can still be inferred because of the context. Although the individual structures alternate, they unfold in parallel, since it is assumed that the opening credits do not have any influence on the replay, for example. It can thus be inferred that the film is constructing three different and parallel events whose logical forms are depicted in table 3.32. The opening credits will be described here by the predicate name, since the shots list the names of the two production companies as well as the director. The various sub-events which can be inferred within this structure will not be distinguished here. The overall event is the opening credit sequence. The same applies to the unfolding of the TV replay, which is described as eπ1 in table 3.33. Both discourse segments can be formulated as the FDRS Kπ1 and Kπ2 as usual accompanied by a third FDRS Kπ3 unfolding in the background. Together, they generate the overall discourse structure of the extract, as given in table 3.34. 6 In film, multiple diegetic levels like those depicted in figure 3.10 are quite common and generally easy to understand. For verbal discourse, in contrast, it is not very usual to have different levels of diegesis which are connected to each other so closely. On this account, it is necessary here to call for an extension of the traditional SDRS in order to be able to depict every structural unfolding of the film.

i i

i

i

i

Film Discourse Interpretation 143

Figure 3.10: Parallel unfolding discourse structures in Requiem for a Dream Since all three structures are neither structurally nor semantically similar and operate on different diegetic levels of the discourse, they cannot be related to each other by a film discourse relation. From a more general point of view, they are of course ‘parallelised’, but the meaning postulate and default axiom for the Parallel-relation cannot be fulfilled in this context. The three levels of description are different layers of the film’s diegesis and therefore are juxtaposed without any further filmic discourse structure relationship. In a more fine-grained analysis with each of the individual structures, it would, however, be possible to find relations between the different sub-events, which can be identified in the TV replay, for example, as welcoming the audience and introducing the candidates. These sub-events are then related to each other by Narration or Elaboration, which depicts the complexity of the overall discourse structure. They will, however, not be described in detail here.

i i

i

144 Analysis

eπ1 = replay eπ2 = name [v] TV show (m) [v] black screen (q) [v] bad visual quality (n) [v] white letters:“Artisan” (r) [v] inserts:“Tappy!”(o) [a] “Ooaahh. . . ” (s) [a] “Juice for Tappy” (p) m, n, o, p |∼ replay (eπ1 ) q, r |∼ name (eπ2 ) eπ3 = watch [v] replay (u) u |∼ watch (eπ3 ) Table 3.33: Logical forms of the first part of the extract from Requiem for a Dream π0 π0 :

π1 π1 : K π 1

π2 π2 : Kπ2

π3 π3 : K π 3

Table 3.34 Nevertheless, the last sub-event of the TV replay is of further importance, since its eventuality is directly connected to the third discourse structure. Since the image track is cut abruptly and the audio track fades out immediately, it can be inferred that the replay is interrupted. The following shot of the connector and its event give a cause for this interruption, because it can be inferred that pulling out the connector is likely to remove electricity from the device whose connector is shown. In figure 3.11, a fine-grained view of the particular frames which create the ending of the replay is given. Here, it becomes visible that the depicted transition must be a part of the event of watching TV, because the process of going out can directly be seen in the image track, which keeps up for a moment and then fades out. Its eventuality can thus be described as eπ4 (go out) which follows eπ3 (watch) on the same level of the diegesis. The two last frames in figure 3.11 then describe the event of pulling out the connector (eπ5 ). The logical forms of the discourse segments are given in table 3.35. Both events are in a temporal consequence and the second event gives a cause for the event depicted before, so that it is possible to infer an Explanation-relation. Normally, the event of pulling out the connector precedes the event of going out and therefore is the reason for the fading

i i

i

i

i

Film Discourse Interpretation 145

Figure 3.11: Frames from shots 9 and 10 of Requiem for a Dream (frames 1381–1389)

π4 , π5 eπ4 = go out [v] TV show (v) [v] cut (w) π4 : [a] “Ooaahh. . . ” (x) [a] fade out (y) v, w, x, y |∼ go off (eπ4 ) holds (eπ4 , t1 )

eπ5 = pull [v] connector (z) [v] pulling (a) π5 : z, a |∼ pull (eπ5 ) holds (eπ2 , t2 ) bef ore (t1 , t2 )

Explanation(π4 , π5 ) Table 3.35: The logical form of the events in shots 10 and 11 of Requiem for a Dream

i i

i

146 Analysis

image. (?(eπ4 , eπ5 , λ) ∧ causeD (eπ4 , eπ5 )) > Explanation(eπ4 , eπ5 , λ)

(3.15)

causeD in equation 3.15 indicates that it is the discourse which gives evidence for the cause. Moreover, λ means that the two eventualities are part of the same unfolding structure, namely the third one which is now the dominating discourse, and therefore indicates that the relation does not hold between eventualities on different diegetic layers. The unfolding of the structure in the following makes clear that the film’s main story-line is now introduced. Shot 12 gives further information about the setting and the characters and establishes the main topic of the ongoing plot. The logical form of the event which can be inferred here is given in table 3.36. eπ6 = talk [v] actor: male person (m) [v] actor: female person (n) [v] represented participant: TV (o) [a] “Harold, not again the TV!” (p) [a] “Aw Ma. Come on, Ma!” (q) m, n, p, q |∼ talk (eπ6 ) Table 3.36 The event of eπ6 describes a dialogue between the characters who become visible in the image track and who are identified by the voice track. Furthermore, their dialogue identifies the connector as part of a television and therefore gives reason for the replay seen before. It can also be inferred from the voice track that this situation is not happening for the first time. The event therefore gives a lot of background information which was not available for the recipient before, and describes the circumstances of the preceding event. These circumstances are quantitatively and qualitatively specifying the overall discourse. The relation between the two segments is thus not only a spatiotemporal consequence, but can also be seen as introducing the main characters of the story and one of the main settings which will be important in the following. Consequently, the default axiom for Narration may be overwritten by the circumstantial information given in eπ6 . The relation which can be inferred in this context is thus a Backgroundrelation. Although the two events related by this relation do not resemble each other very strongly, they share the common topic of the TV and its transportation. The update-process may therefore most plausibly lead to

i i

i

i

i

Film Discourse Interpretation 147

the inference of Background in favour of Narration. (?(eπ5 , eπ6 , λ) ∧ circumstantial information(eπ5 , eπ6 )) > Background(eπ5 , eπ6 , λ)

(3.16)

Since the shot is taken as a medium long shot and not only introduces the main characters of the film, but also its general setting and an important topic, it can be interpreted as having the function of a so-called establishing shot, which sets up the scene and its context at the film’s beginning (cf. Bordwell et al. 1985: 202). In sum, the overall discourse structure of the first part of the extract is then as depicted in table 3.37. π0

π0 :

π1 π1 : Kπ1

π2 π2 : Kπ2

π3 , π4 , π5 , π6 π3 : Kπ3 , π4 : Kπ4 , π5 : Kπ5 , π6 : Kπ6 N arration(π3 , π4 ) Explanation(π4 , π5 ) Background(π5 , π6 )

Table 3.37: Segmented Film Discourse Representation Structure of the first part of the extract from Requiem for a Dream Whereas the first two structures unfolding in parallel are not described in great detail here, the third structure is the one which establishes the film’s main diegesis. Therefore, in table 3.37, it becomes visible how the topic develops due to the discourse relations holding between the eventualities. In figure 3.12, the evolving structure is also finally depicted as a graphical representation. The shots following depict a split screen situation which shows the plot from two perspectives. On the left, it shows the mother in a second room, sometimes observing her son through the keyhole, expressed by a point-of-view shot. On the right, the setting is the living room where the son attends to the TV and talks to his mother. In table 3.39, this second part of the film extract is transcribed. Subsequent to eπ6 (talk), which shows the two characters in the living room, talking about the TV, there is a second eventuality (eπ7 ) which can be inferred within shot 12 and which depicts the mother entering a second room and closing the door. Its logical form is given in table 3.38.

i i

i

148 Analysis

π0

π1

π2

π3 Narration π4 Explanation π5 Background π6

Figure 3.12: Graphical representation of the first shots (1–11) of Requiem for a Dream (Aronofsky, 2000)

eπ7 = close [v] actor: female person (r) [v] participant: door (s) [a] closing (t) r, s, t |∼ close (eπ7 ) Table 3.38

i i

i

i

i

Film Discourse Interpretation 149

#

12.

13.

14.

15.

16.

17.

18.

shot

shot description older female and younger male person in living room, talking about the TV split screen (caused by a wipe): mother in front of the door, son in living room split screen: POV shot through keyhole, son acting with TV in living room split screen: mother watching her son, a close-up of a chain split screen: mother watching her son, son talking to her split screen: close-up of mother taking the key off her neck, son talking to her split screen: POV shot through keyhole, close-up of key

spoken language “Harold please, not again the TV.” — “Aw Ma. Come on, Ma.” “Why do you have to make such a big deal out of this, Ma? You know you’ll get the set back in a couple of hours.”

/

“What are you trying to do? You trying to get me to break my own mother’s set?” “Or break the radiator?’ “And maybe blow up the whole house, Ma. Your own son, your own flesh and blood, Ma?” “ls that what you’re trying to do? Your own son?”

i i

i

150 Analysis

19.

split screen: close-up of mother, son taking the key from the ground

“Why do you always gotta play games with my head?”

20.

split screen: close-up of mother, door handle

“Harold, I wouldn’t do that. The chain isn’t for you. It’s for the robbers.”

21.

split screen: close-up of mother, son talking to her

“Then why won’t you. . . ”

22.

23.

24.

25.

26.

split screen: POV shot through keyhole, son talking to her split screen: close-up of mother; son shaking the door split screen: POV shot through keyhole, extreme close-up of son split screen: POV shot through keyhole, medium shot of son split screen: close-up of mother, son in living room

“. . . come out?”

“See what I mean? See how you always gotta upset me, Ma?”

/

/

“Ma. Ma. C’mon out?”

i i

i

i

i

Film Discourse Interpretation 151

27.

28.

split screen: close-up of mother, son acting in living room

/

close-up of mother watching her son through keyhole

“This isn’t happening. And if it should be happening it would be alright. So don’t worry, Seymour. It’ll all work out. You’ll see already. In the end it’s all nice.”

Table 3.39: Transcription of the second sequence (shots 12–28) of Requiem for a Dream Since the end of eπ6 is succeeded by the beginning of eπ7 , the latter temporally and spatially overlaps the first one in that they are directly attached to each other. They therefore depict a natural event-sequence whose first eventuality occasions the second one. They can thus be related to one another by Narration. (?(eπ6 , eπ7 , λ) ∧ occasion(eπ6 , eπ7 )) > N arration(eπ6 , eπ7 , λ)

(3.17)

The discourse relation here holds between events in a single shot. It is on the one hand the exceptional length of the shot with more than 120 frames which gives reason for the necessary segmentation into several eventualities. Moreover, the small conceptual change between the dialogue as the topic of eπ6 and the action process of closing the door in eπ7 is decisive for the structural unfolding. It serves as the basis for establishing the following split screen which, from a structural perspective, represents itself as a subordinate discourse segment. The split screen maintains two strands of the plot which both unfold in time and space and therefore both evolve a particular discourse structure. The structures are both subordinated to the eventuality which motivates the split screen, in which the mother leaves the living room by entering a second room and closing the door. This process of closing is realised as a wipe to the left with which the split screen is established; see figure 3.13. Moreover, the filmic text here directly gives instructions to split the screen and the two strands of the plot by using the wipe as a formal resource which functions to influence the film’s unfolding. On the textual

i i

i

152 Analysis

Figure 3.13: Frames of the second part of shot 12 of Requiem for a Dream (frames 1510–1525) (Due to space constraints, every third frame of this sequence is included in the figure.) level, this resource is partly comparable to so-called cue phrases in verbal language, such as and then for Narration as a verbal discourse relation (e.g., “Kim watched TV. And then she went out.”; cf. Asher and Lascarides 2003: 202), which monotonically yield the relation so that abductive reasoning about the best discourse update here is not necessary. In the example presented here, the situation is different in that the resource does not entail the relation, but at least produces the two structures which are subordinated in the following. This can be expressed formally as follows: (?(eπ7 , eπ0 , λ) ∧ [horizontal wipe halfway(eπ7 , eπ0 )] → (Kπ8 ∧ Kπ9 ) (3.18) The split screen as a result of this process is thus not a defeasible consequence which may be interpreted differently, but is directly given within the filmic discourse and therefore can straightforwardly be derived from the conditions within the text. Hence, the operator in the equation above is →, which indicates that the logical forms in combination with [horizontal wipe halfway] imply the two sub-structures Kπ8 and Kπ9 , which again unfold in parallel as two juxtaposed events. In contrast to the parallelised structures in the beginning of the extract, the events are not cross-cut within an alternating sequence, but are shown simultaneously on two screens within one image. The camera perspective regularly alternates between close-ups on the protagonists and point-of-view shots from the two protagonists’ perspective, for example through the keyhole when the mother is observing her son or from the son’s perspective when he is trying to take away the TV. The relation which can be inferred as establishing this subordination is Result, since the eventuality of closing gives a cause for the separation of

i i

i

i

i

Film Discourse Interpretation 153

the two structures. (?(eπ7 , eπ0 ), λ) ∧ causeD (eπ7 , eπ0 )) > Result(eπ7 , eπ0 , λ)

(3.19)

Since the two sub-structures Kπ8 and Kπ9 resemble each other in different aspects and are both accompanied by the continuous voice track, a Parallel-relation can be inferred between them. In contrast to the alternating sequence in the beginning of the extract, the two strands unfold on the same diegetic level and it is this which allows the inference of a discourse relation. The SFDRS given in table 3.40 describes the subordination and their parallel unfolding. π0 π6 , π7 , π 0 π6 : Kπ6 , π7 : Kπ7 N arration(π6 , π7 ) Result(π7 , π 0 ) π 0 , Kπ8 , Kπ9 P arallel(π8 , π9 )

π0 : π0 :

π8 π8 : Kπ8

π9 π9 : Kπ9

Table 3.40: Segmented Film Discourse Representation Structure of the split screen sequence in the extract from Requiem for a Dream The two structures can then again be differentiated because of different sub-events. There are for example on both screens at least two different events of talking and observing, which, however, are not directly parallelised. While the mother is observing her son through the keyhole, he is talking to her. Later, the mother is answering her son’s questions and the son is listening to her while at the same time observing the door. The individual point-of-view shots do therefore not influence the segmentation of further events. This highly dense structure and the split screen are dissolved with a wipe to the right when Harold is leaving the room with the TV table. The single shot of the mother is thus again a monotonically resulting consequence which is directly produced by the filmic device. The mother is focused in her room by a very close-up view, while her voice track

i i

i

154 Analysis

comments on the situation: “This isn’t happening. And if it should be happening, it would be alright. So don’t worry, Seymour. It’ll all work out. You’ll see already. In the end it’s all nice” (see the transcription in table 3.39). The logical form of this event is given in table 3.41. eπ10 = comment [v] actor: female person (m) [a] “This isn’t happening. And if. . . ” (n) m, n |∼ comment (eπ1 ) Table 3.41 The eventuality of this discourse segment is described as eπ10 , since it is part of a natural event-sequence maintained by Narration, which immediately follows the split screen sequence. The relation thus holds between eπ8 as the label for the first subordinated structure depicting the mother in her room and eπ10 as the next inferrable event outside the subordination. (?(eπ8 , eπ10 , λ) ∧ occasion(eπ8 , eπ10 )) > N arration(eπ8 , eπ10 , λ)

(3.20)

π7 Result π0

π8

π9

Narration

π10 Figure 3.14: Graphical representation of the split screen sequence in Requiem for a Dream

i i

i

i

i

Film Discourse Interpretation 155

The graphical representation in figure 3.14 makes the dissolving of the parallelised structures visible and depicts how the structure further unfolds. Simultaneously to the second wipe on the visual level, the music in the background stops. The whole scene is then interrupted by a title tumbling down in a vertical wipe and then dissolving to a black screen. It is accompanied by a noise of a closing gate and strong strokes of a violin bow. This transition represents a short break within the text which separates two different scenes by an insert. However, the music carries over to the following shots and creates the only continuity between these discourse segments. In the following, a third character is shown in the hall talking to Harold in front of the room. When the two characters leave the house, the music in the background gets louder. It highlights the following sequence in which Harold and his friend are walking along the street where some women are sitting in deck chairs. A first long shot of the setting outside is followed by alternating close-ups of the women and the two young men. The shots can be seen from the perspectives of the respective characters. Whereas the camera moves to the right showing the women sitting on the street, it follows the movements of the two men to the left when they are shown in detail; see table 3.42. This alternation of shot–reverse shots is again very regular and it is assumed that the sequence is inferred as one event which features several sub-events, namely the transportation of the TV and the process of observing the two men. In sum, the whole sequence can be described, on the one hand, as a so-called disconnected narrative process (cf. Kress and van Leeuwen 1996: 259) which shows a kind of dialogue between the participants; on the other hand, the particular shots can be seen as point-of-view shots in an alternating, ‘reflexive’ structure, as described by Branigan (1975: 63). Whereas the individual shots within the first sub-event depict unidirectional transactional actions in which the two actors transport the television as a represented participant, but look as well at phenomena which are not shown in the shot itself, the respectively subsequent shots represent reactors which form eyeline vectors to the phenomena given in the other shots. According to Branigan, “the character need not stare directly into the camera (for this involves another convention) but the eyes must be very near the line of the camera” (Branigan 1975: 63). This is the case in this example. Since the shots which show the two actors are taken from the opposite side of the imaginery line running between the participants, they strongly interrelate with the opposed shots and therefore give evidence for the interpretation of only one single event. Furthermore, the shots share specific commonalities, for example the same lighting and the continuous sound track in the background. Their logical form, described as eπ12 and two sub-events (eπ12a and eπ12b ), is given in table 3.43.

i i

i

156 Analysis

shot description

spoken language

34.

the son and the black protagonist moving the TV along the street

/

35.

medium shot of women

/

36.

son and black protagonist walking along the street, talking to the women

“Hello.”

37.

medium shot of women

“Yeah. . . hello.”

38.

son and black protagonist walking along the street

/

39.

medium shot of women

/

#

shot

Table 3.42: Transcription of the third sequence (shots 34–39) of Requiem for a Dream

i i

i

i

i

Film Discourse Interpretation 157

eπ12 = dialogue eπ12a = look at [v] actors: Harold, Tyrone (m) [v] repr. participant: TV (n) [v] female persons (o |∼ p) m, n, o |∼ look at (eπ12a )

eπ12b = look at [v] actors: female persons (p) [v] male actors (q |∼ m) p, q |∼ look at (eπ12b )

Table 3.43: The logical form of the dialogue sequence in Requiem for a Dream As a very typical montage sequence which is highlighted by a dominant sound track, the following shots mainly use medium long shots and long shots to show the two protagonists on their way walking along different settings and transporting the television. Insertions of credits interrupt the shots and function as temporal and spatial ellipses within the sequence. The credits again give background information on the film and its producer and director and make the scene substantially similar to the film’s principal beginning. Its discursive structure is correspondingly analogous to that of the first shots (see figure 3.12) and will not be reconstructed here. Particular interest is finally given to the last part of the film extract which represents a montage of images and sounds, cut in rapid succession. The transcription of this sequence is given in table 3.44. It depicts a number of at first sight not coherent shots which can hardly be recognised due to their rapid succession. Besides a few rather incomprehensible sounds, there is no continuous audio track or background music. Every shot is a close-up or extreme close-up which is neither faded in nor wiped out. shot description

audio track

64.

close-up of a mouth

/

65.

a white bubble on black background

moaning

#

shot

i i

i

158 Analysis

66.

close-up of a lighter

/

67.

extreme close-up of bubbles

crackling noise

68.

close-up of a fluid in an injection

/

69.

close-up of a pupil, dilating

shortly increasing noise

70.

close-up of a second mouth (darker skin)

/

71.

close-up of a bottle’s cap

brewing

72.

close-up of a steaming fluid in the cap

/

73.

close-up of a fluid in an injection

/

i i

i

i

i

Film Discourse Interpretation 159

74.

abstract image of the infiltration of the fluid into the blood

moaning

75.

close-up of a pupil, dilating (darker skin)

/

76.

long shot from above: the two men listening to music and dancing

“Sheeit. . . that’s some boss scag, baby.” / music

Table 3.44: Transcription of shots 64–75 of Requiem for a Dream Normally, the whole sequence will be inferred as one event, namely the preparation and intake of drugs and their effects on the participants. How the different shots are identified as sub-events is illustrated by the logical forms given in table 3.45. Since these sub-events can be related to each other by discourse relations, they are described as eπ16 – eπ21 , subsequent to the eventualities described before. eπ16 = open [v] mouth of white actor (m) [v] sachet with white powder (n) m, n |∼ open (eπ16 ) eπ18 = heat [v] lighter (q) [v] unknown person (r) q, r |∼ heat (eπ18 ) eπ20 = draw [v] fluid in an injection (u) u |∼ draw (eπ20 )

eπ17 = increase [v] white bubble (o) [a] noise of moaning (p) o, p |∼ increase (eπ17 ) eπ19 = bubble [v] bubbles (s) [a] crackling noise (t) s, t |∼ bubble (eπ19 ) eπ21 = dilate [v] pupil (v) [a] noise of dilation of pupil (w) v, w |∼ dilate (eπ21 )

Table 3.45 Most of the shots depict nontransactional action processes supported by corresponding noises. The crackling noise in shot 67, for example, synchronises with the eventuality of the bubbling fluid. The same applies

i i

i

160 Analysis

to the noise synchronous to the dilation of the pupil which emphasises the process depicted in shot 69. As a natural event-sequence, the releasing of the bubbles depicted in shot 65 follows the process of opening the small sachet with drugs. It is inferred that the powder seen in the sachet now is diffused somewhere. The eventuality in shot 64, labelled with π16 , thus occasions the eventuality in shot 65. The relation which can be inferred here is Narration: (?(eπ16 , eπ17 , λ) ∧ occasion(eπ16 , eπ17 )) > N arration(eπ16 , eπ17 , λ) (3.21) In the following, the shots are associated with each other by logical causeeffect relationships which overwrite the default Narration-relation. The eventualities heat and bubble as well as draw and dilate can be related by a Result-relation, since they all depict logical cause-effect connections: the lighter is used to heat an unspecified object which is then shown in the following shot as a bubbling fluid, and the injection causes the dilation of the pupil. The SFDRS of the six shots can thus be depicted as given in table 3.46. π0 π16 , π17 , π18 , π19 , π20 , π21 π16 : Kπ16 , π17 : Kπ17 , π18 : Kπ18 , π19 : Kπ19 , π20 : Kπ20 , π21 : Kπ21 N arration(π16 , π17 ) π0 : Contrast(π17 , π18 ) Result(π18 , π19 ) Contrast(π19 , π20 ) Result(π20 , π21 )

Table 3.46: Segmented Film Discourse Representation Structure of shots 64–69 of Requiem for a Dream Since the transitions between the eventualities eπ17 and eπ18 as well as eπ19 and eπ20 do not depict similar circumstances, they can only be related to each other by Contrast-relations. Although they have a similar DRS, they are semantically similar, but generate contrasting themes. φContrast(π17 ,π18 ) ⇒ (Kπ17 ∼ Kπ18 )

(3.22)

φContrast(π19 ,π20 ) ⇒ (Kπ19 ∼ Kπ20 )

(3.23)

i i

i

i

i

Film Discourse Interpretation 161

A second sequence of close-ups immediately follows whose eventualities closely resemble the preceding ones. Shot 70, for example, shows the same nontransactional process of opening a sachet with the mouth. In this case, it is a person with darker skin which is depicted in the close-up and which allows an identification with the second protagonist in the sequence before who is, in contrast to shot 64, an actor with darker skin. Likewise, shot 75 shows another pupil which dilates because of the infiltration of the fluid into the blood shown in shot 74; see table 3.47. eπ22 = open [v] mouth of 2nd actor (x) [v] sachet (y) x, y |∼ open (eπ22 ) eπ24 = draw [v] fluid in injection (c) c |∼ draw (eπ24 )

eπ23 = steam [v] cap, fluid (z) [v] steaming fluid (a) [a] noise of brewing (b) z, a, b |∼ steam (eπ23 ) eπ25 = inf iltrate [v] vein, fluid (d) [a] noise of moaning (e) d, e |∼ infiltrate (eπ25 )

eπ26 = dilate [v] pupil (f) [a] noise of dilation of pupil (g) f, g |∼ dilate (eπ26 ) Table 3.47 The SFDRS in table 3.46 can thus be expanded as given in table 3.48. Whereas the first part of the structure (π16 –π21 ) is constantly alternating between Contrast- and Result-relations, the subordinated structure (π22 – π26 ) contains Narration- and Result-relations. As already outlined above, there are as well two Parallel-relations which relate the eventualities to preceding segments of the first part. Since there is a parallel between π16 and π22 , a subordination is established that unfolds as the second strand showing again the preparation from the beginning. Shots 71 and 72 are inferred as one sub-event (eπ23 ), depicted by different camera perspectives. Although there is a small temporal ellipsis between the two shots (the fluid in the latter shot changes its consistency and is then steaming), the process described in both images is nearly the same. The structure is finally expanded by the last label π27 which can again be associated with the events before by a cause-effect relationship. The long shot from above shows the two protagonists after having taken the drugs. The voice track comments on the effect the drugs have: “Sheeit. . . that’s some boss scag, baby.” Consequently, the predicate which can be inferred via the logical form of this discourse segment is identified as comment; see table 3.49.

i i

i

162 Analysis

π0 π16 , π17 , π18 , π19 , π20 , π21 π16 : Kπ16 , π17 : Kπ17 , π18 : Kπ18 , π19 : Kπ19 , π20 : Kπ20 , π21 : Kπ21 , N arration(π16 , π17 ) Contrast(π17 , π18 ) Result(π18 , π19 ) Contrast(π19 , π20 ) Result(π20 , π21 ) P arallel(π16 , π 0 ) π0 :

π22 , π23 , π24 , π25 , π26 π22 : Kπ22 , π23 : Kπ23 , π24 : Kπ24 , π25 : Kπ25 , π26 : Kπ26 N arration(π22 , π23 ) π0 : N arration(π23 , π24 ) Result(π24 , π25 ) P arallel(π20 , π24 ) Result(π25 , π26 )

Table 3.48: Segmented Film Discourse Representation Structure of shots 64–75 of Requiem for a Dream

eπ27 = comment [v] two men (h) [a] “Sheeit. . . that’s some boss scag”(i) [a] music from the record player (j) h, i |∼ comment (eπ27 ) Table 3.49

i i

i

i

i

Film Discourse Interpretation 163

It is inferred that the situation in shot 76 gives a general overview of how the intake of the drugs affected the protagonists’ state and mood. This interpretation is possible due to the inferences which have been drawn before. Given on the one hand the knowledge that the protagonists have traded in the television for money to buy drugs and on the other hand the detailed description of the consumption, the depiction in shot 76 gives a very general outcome of the preceding events and represents at the same time a final conclusion of the whole extract. The main protagonists of the movie as well as the main topics of the story have now been introduced; the first strand of the plot is completed. 3.4.3

Conclusion

It has become clear in section 3.4.2 that the film extract taken from Requiem for a Dream represents a complex and highly structured discourse with several sub-structures. In contrast to many other movies, the film is constituted by a high number of extremely short, almost subliminal shots which are often cut together in rapid succession. Yet in the beginning sequence, this stylistic device is used frequently and therefore gives insight into the general composition and design of the rest of the film. Important interpretation patterns are established which influence the recipient’s attitude towards their viewing of the film as a whole. As revealed in the tables and figures which represent the SFDRS, the discourse’s partly extravagant structure can mostly be characterised by relations which, on the one hand, focus on the content of the discourse and which, on the other hand, influence its texture. The very beginning of the excerpt is portrayed by an alternation of credit shots and those shots which identify the main protagonists of the film’s story. To distinguish these fundamentally different topics, the discourse structure had to be divided into further sub-structures which unfold in parallel. In this regard, however, it is highly coherent. In particular, the different subordinations within the overall SFDRS, for example depicted in table 3.40, outline the multilayered organisation of the text. A further stylistic device is the extensive use of close-ups and extreme close-ups which, in their high number, maintain as well strong parallels in their textual structuring. It becomes clear that the different shots bear a high and in most cases related, coherent meaning potential that is captured by the eventualities within the logical forms. A very interesting example is the construction of the split screen which establishes two parallel sub-structures. The subordination of these structures can be monotonically inferred due to the horizontal wipe on the visual level. The discourse here makes it possible to derive the formal structure without defeasible reasoning, but due to a textual implication. This construction goes beyond the examples of verbal discourse structures

i i

i

164 Analysis

given by Asher and Lascarides (2003) (see section 2.2.2) and therefore depicts a filmic specificity. It is the technical features of the filmic text, which give rise to the structure and therefore to a further possibility to construct a narrative, which is not comparable to verbal discourse. Furthermore, apart from typical introductory scenes which present the main protagonists of the film and its setting as well as the main points of the story, the sequence of π16 –π26 showing the drug consumption events constitutes an important key scene which recurs several times and in different styles throughout the movie. The analysis of this scene points out that its interpretation is based on a clear structure which is constructed by a regular use of relations. However, the shots depict rather unknown processes in extreme close-ups whose meaning potential first has to be identified. It is assumed that the more often the scene is shown, the better the interpretation succeeds. The processes shown in this montage are thus elaborated progressively, since the different discourse segments can be taken as given in subsequent film scenes.

3.5

Conclusion of Chapter 3

The comprehensive analyses of the four film extracts have demonstrated the applicability of the framework presented in chapter 2. With the help of the logic of film discourse interpretation, it was possible to build concrete logical forms of filmic discourse that, on the one hand, give evidence for the intersemiotic meaning construction of the discourse segments and, on the other hand, outline structure and coherence of the extracts by describing film discourse relations holding between these segments. This directly corresponds to the three characteristics of a narrative Bordwell discusses in his introduction to “Narration in the Fiction Film” (Bordwell 1985): narrative as a representation, as a structure, as a process. The derivation of the logical forms of the discourse within the framework of the logic of information content results in the representation of the film’s story-line. This representation can as well be described by and with its discursive structure, which is constructed by the interrelationships between the representational entities. The meaning construction thereby is a dynamic process within a spatially and temporally unfolding discourse. The first logic within the presented framework, the logic of information content, allows a transformation of filmic devices into more abstract representations of the propositional content of a film. Therefore, the framework delivers a concrete schema for the description of the technical and formal entities used in the filmic text and gives details for a summary of this content as the verbalisation of the eventuality. Each eventuality which can be inferred is illustrated by the DRS box-style notation introduced in section 2.2.3. The fact that every modality and submodality on the visual

i i

i

i

i

Film Discourse Interpretation 165

and auditory level can be included enables a very fine-grained depiction of the various entities involved. The representations of the eventualities resemble the formalisms used in projects which aim at modelling discourse production and the generation of narratives in various fields of research (cf. Jasinskaja and Rossdeutscher 2009; Jasinskaja 2010; von Stutterheim et al. 2012; see section 2.2.2). In contrast to the notation style applied in the preceding analyses, the result in Jasinskaja’s work contains two different levels of representation which first feature “referents for objects that can be seen in the film, as well as time points and intervals for the time, and DRS-conditions for propositions that can be judged true on the basis of the picture and the sound track” (Jasinskaja 2010: 1). The second level includes relationships between states and events as well as emotions and perceptive ascriptions to the protagonist of the film. Although filtering out information from different sensory levels, the formalism used by Jasinskaja does not carry out a multimodal analysis of how this information can be inferred and which role it plays within the generation of the narrative. Its apparently comprehensive representation of the interpretation process thus omits how the described events and mental states are manifested within the filmic text and which contextual information guides their interpretation. The classification of the semiotic resources pursued in this book, in contrast, gives evidence for the framework’s specific benefit in terms of the examination of the intersemiotic meaning-making in filmic text, which will be elucidated in further detail in the chapter following. The emphasis of the analyses in this chapter lay on the particular discourse structures as syntagmatic realisations of the paradigmatically available discourse relations given in section 2.2.4. Consequently, the variety of these structures gives evidence for the various possibilities of a film’s unfolding in terms of montage and film editing techniques. It could be shown, for example, that filmic techniques such as the cross-cutting montage in The Lives of Others are directly based on their textual composition and that they can therefore be analysed with linguistic tools. Furthermore, the framework made it possible to describe the projection sequence in Vanilla Sky by demonstrating the CCP and the so-called discourse pop (see section 3.2.2) during the interpretation of this extract. The analysis of this film verified how the dynamic unfolding of filmic text in general can be examined systematically and in particular with regard to the logical principles that influence the recipient’s inference process. As a result of the analysis, the preferred discourse structure gives evidence of how the recipient is guided by the text and its technical devices. The two further analyses in sections 3.3 and 3.4 then gave examples of the complexity of film discourse structures and their narrative potential. It became clear that filmic diegetic levels may evolve on several structures which unfold in parallel. The analysis of these parallels in terms of the SFDRS made it

i i

i

166 Analysis

possible to elucidate the various subordinations and coordinations within the overall structure that maintain the films’ coherence. The logic of film discourse interpretation thus makes available an in-depth view of the relational meaning-making and the recipient’s interpretation process in terms of a logically based description. This allows an examination of both the inferential strategies of reasoning about the film’s meaning as well as the distinct constraints the text offers to control this interpretation. With regard to the short theoretical discussion in chapter 1, the preceding analyses presented exactly the systematic examination which had been required for a more comprehensive overview of filmic meaning construction. In addition, the detailed description already gave a first classification of the relations from a metafunctional perspective. The beginning of the unfolding discourse structure in the extract from The Lives of Others, for example, is mainly constructed by text-structuring relations such as Parallel or Contrast. This classification supports the formal description of the interpretation in that it refers to the functional dimension of meaning and tries to find correspondences of these two perspectives. However, this functional consideration of the filmic discourse now needs further examination. This will be pursued in the chapter following. On the basis of the formal level of description presented in section 2.2.2 and applied to example extracts above, it will be useful to organise the different entities and discourse segments according to their specific tasks within the meaning-making process and the influence they have on the inference process. The discussion in chapter 4 will therefore promote a metafunctional and stratificational organisation of the results found in the analyses in order to combine the bottom-up process of building logical forms of the discourse with a top-down process of describing the social processes in which filmic meaning construction is embedded. With the help of this organisation, it will then be possible to give an even more comprehensive examination of film discourse interpretation.

i i

i

i

i

4 Multimodal Narrative Construction in Filmic Text Building on the formal description of the inference processes during film interpretation in the previous chapter, the present chapter will now put forward the functional dimension of the framework in order to bring together both perspectives as the basis for a generalised approach to multimodal narrative construction in filmic discourse which can be elucidated in linguistic terms. The comprehensive analysis of the different film excerpts in chapter 3 has demonstrated that the films’ narrative content is constructed via a discursive texture which builds on relational meaning-making across different modes. It is thus a multimodal construction of filmic narrative which can be successfully examined with the framework presented in this book. Building the logical forms of the discourse and connecting them via the glue language and discourse update verifies that the combination of the different modes creates a coherent, meaningful narrative. According to Bordwell (1985) and as has been pointed out in section 1.3, narrative films are considered as a set of cues for building up a coherent story: “Comprehending a narrative requires assigning it some coherence. At a local level, the viewer must grasp character relations, lines of dialogue, relations between shots and so on. More broadly, the viewer must test the narrative information for consistency: does it hang together in a way we can identify?” (Bordwell 1985: 34) It is exactly this process of inferring the content of the discourse and relating it in a reasonable way which has been described in detail in the preceding examination. However, as has already been pointed out in section 1.1, this process is not only characterised by bottom-up operations of building logical forms, but also by “conceptual, strategic” top-down processes that furnish the “higher level textual data upon which various interpreters base their inferences” (Bordwell 1989: 3; see the quotation on page 5). These higher-level textual data will now be elucidated in more detail in the following. Various organisational levels of the discourse segments and their semiotic resources will be pointed out in order to find more systematic ways for describing and classifying their meaning potential. It has for example become clear that the logical forms which have been built for the various discourse segments in chapter 3 describe completely 167 i i

i

168 Multimodal Narrative Construction in Filmic Text

diverging events with different narrative processes. They enlist a number of modalities and submodalities which all contribute to the inferred event. However, it cannot be entirely determined which role the particular kinds of information exactly play for the meaning-making process. Their multiple articulations make it impossible to find generalisations about their use and meaning. Therefore, it is necessary to describe at least the various functions the devices have in the particular context and which could be observed during the analysis. This can also be traced back to the general assumption that ‘mode’ in a multimodal environment has to be treated as an entity which depends to a great extent on the context and its analytical purposes (cf. Tseng 2009: 85) and therefore has to be defined with regard to its actual use and realisation. It is then possible to ask how this mode works semiotically and in which way it might be decisive for the interpretation process. This is exactly what is still required in systematic film analysis: “If we can find ways of making connections between technical details and sources of interpretation more explicit and reliable, we would clearly be in a better position for pursuing analysis. But to support more systematic interpretative analysis, it is necessary to find systems of contrasts that organise and prestructure the filmic devices employed.” (Tseng and Bateman 2012: 5) Building on the analyses in the preceding chapter, the organisation and prestructuring of devices required is now possible. The following sections will therefore highlight different levels and perspectives within the logic of film discourse interpretation which organise the different types of resources and their meaning in the logical forms, and will show the role the discourse relations play in the unfolding discourse structure. In order to organise these different levels of filmic meaning, the notion of strata as one of the central components of systemic-functional linguistics (SFL) will be readopted in the following. The different levels of interpretation will be described as strata of film discourse analysis which are depicted in a meaning hierarchy that resembles the depictions of discourse strata in the general framework of SFL and social semiotics (see section 4.1). Such a stratificational organisation will help to describe meaning construction in more detail and with regard to contextual interpretation strategies which could not be elucidated systematically during the analysis in chapter 3. This will make it possible to work out the dynamic interaction between the operations of film interpretation and to elaborate on each element included in this process. The overall aim of this description and classification is a more integrated view of how meaning in filmic discourse is constructed by means of the recipient’s narrative comprehension and his/her interpretation within

i i

i

i

i

Film Discourse Interpretation 169

the social context. Consequently, this chapter will focus on a combined method for elucidating the semantic and pragmatic cues that help to interpret filmic text and that, in a further step, build a basis for bridging the gap between the conceptually different accounts of film analysis and finding combined and shared ways of interpretation. These shared ways have recently been focused on by David Bordwell in one of his online essays (Bordwell 2011) in which he pursues a “common-sense film theory”, i.e., a kind of meta-level of interpretation and explanation that is described as follows: “Recognizing that it [common-sense] is in play in narrative comprehension makes it something we need to analyze. We can understand filmic understanding better if we recognize what’s intuitively obvious, and then go on to ask what in the film, and in our psychological and social make-up, makes something obvious. And those factors may not be obvious in themselves. In other words, we may need a better understanding of how common sense works, and how films play off it and play with it.” (Bordwell 2011) In this comment, Bordwell explicitly asks for a combined view of those filmic devices, i.e., the technical details and textual qualities that are responsible for film understanding. It is thus necessary to describe exactly how these technical details work together with the textual qualities of film and the contextual interpretation strategies in order to clarify how filmic text can be interpreted. On the basis of the formal examination of filmic coherence and structure in chapter 3, the intended stratificational analysis in this chapter will therefore explore the various levels of narrative construction across different modes and strata of the filmic text. This will finally bring together the bottom-up processes of making meaning with top-down processes from higher strata that govern narrative meaning construction at the same time. On this basis, it will also be possible to conclude further results of the formal examination of filmic narrative in chapter 3. The variety of discourse structures examined gives a first overview of the diversity of narration in contemporary films and allows re-raising Bordwell’s question of “how common sense works” (see above). It is assumed that the strata of film discourse interpretation will help to get a closer look at the multimodal construction of filmic narrative which in turn sheds light on a more general comprehension and interpretation of film. It has for example become clear during the analysis that spatiotemporal consequence, generally taken as a basic assumption for narrative, was rarely constructed by the discourse structure itself, but instead is already inferred on another level of description, namely within the logic of information content. Consequently, it will be possible with the help of the intended stratificational organisation

i i

i

170 Multimodal Narrative Construction in Filmic Text

to elucidate in more detail the processes of the narrative logic and the construction of the film’s narrative. This chapter is therefore subdivided into the following sections: section 4.1 will briefly introduce the framework of SFL and social semiotics in order to bring the formal framework described above into relation with the functional approach to film and multimodality and to build a basis for the combination of both perspectives. The different levels of interpretation which will later be combined to yield the strata of film discourse analysis that are described in section 4.2. This section will give an overview of how meaning is constructed cross-modally within the logical forms of the discourse segments (section 4.2.1) and which metafunctional diversification can be found within these logical forms (section 4.2.2). Furthermore, it will outline the different knowledge sources that influence the interpretation process from a further layer (section 4.2.3). Finally, the strata of film discourse analysis will be shown in a comprehensive overview in section 4.2.4. The second part of the chapter is further subdivided. Section 4.3 will give an overview of how filmic narrative is generally constructed with the help of the logic of film discourse interpretation and which specificities of filmic discourse in comparison to verbal narrative text can be described with regard to the interpretation processes manifested within the different strata of film discourse analysis. On the basis of this description, section 4.4 will motivate a paradigmatic organisation of the set of film discourse relations in order to be able to describe more systematically how these relations make it possible to infer a film’s narrative content. Finally, section 4.5 will summarise how the framework provided in this book helps to analyse filmic narrative construction and thereby contributes to the above mentioned common-sense film theory.

4.1

The Functional Framework for Multimodal Film Analysis

Functional accounts of discourse interpretation build on the frameworks of SFL and social semiotics. These are mainly concerned with the interaction between producer and recipient of a text and therefore with the social processes that operate between them. They are based on the grounding statements of Michael Halliday and colleagues (Halliday and Hasan 1976; Halliday 1978; Martin 1992; Martin and Rose 2003; Halliday 2004), whose main assumption is that language is a social system in which meaning is made through interaction, i.e., by social activity. Hence, texts are the realisation of social contexts in sequences of clauses (see Martin and Rose 2003: 4).

i i

i

i

i

Film Discourse Interpretation 171

A central dimension which has first been assumed for language and is now also applied to filmic discourse is the notion of strata. Martin and Rose (2003) set out that the strata of language are built of the three levels of grammar, discourse and social context. Based on the Hjelmslevian distinction between content and expression plane (Hjelmslev 1961), Halliday extended the former into the levels of lexicogrammar and semantics, whereas the latter includes the levels of phonology and graphology. Martin (1992) then declared semantics to be specifically a discourse semantics which “both generalizes across grammatical resources and accounts for relations between as well as within clause complexes. The discourse semantics is thus more abstract than, and deals with larger units than, lexicogrammar” (Martin 1992: 19). Nevertheless, Martin also underlines the solidary relationship between the two levels. The typical SFL-diagram shown in figure 4.1 presents the correlations. Following Martin and Rose (2003), the level of discourse is embedded into a further level of social activity.

Figure 4.1: Combining the points of view of discourse (Martin and Rose 2003: 4) and the tri-stratal language system according to SFL (Tseng 2009: 44) The principle of realisation arranges the relations between strata. Martin and Rose (2003) describe this principle as “a kind of re-coding” that is associated with the concepts of ‘symbolising’, ‘encoding’, ‘expressing’ and ‘manifesting’ (Martin and Rose 2003: 5). Meanings are therefore realised across strata, actualised for example in a text on the level of discourse semantics, which again is realised by sentences. Lemke (1985) here focuses on the notion of metaredundancy, which sees cross-stratal relations not only

i i

i

172 Multimodal Narrative Construction in Filmic Text

pairwise to one another, as for example between the layers of lexicogrammar and semantics or between semantics and context, but moreover as the realisation of semantics by the ‘realisation of lexicogrammar by phonology’, for example. Metaredundancy then helps to comprehend the relation between language within a lower stratum and context within a higher stratum. Discourse semantics, hence, is definitely context-based, since texts are always embedded in a social activity; their meanings are construed within certain contexts. Discourse analysis then interfaces with the levels of language, since it “employs the tools of grammarians to identify the roles of wordings in passages of text, and employs the tools of social theorists to explain why they make the meanings they do” (Martin and Rose 2003: 4). This functional perspective includes the analysis of interrelationships between the textual structures and structures of the context (cf. van Dijk 1990: 27). In contrast to formal linguistic accounts that conceptualise language as a system of rules with restrictions on how to handle this system, the systemic-functional approach to language underlines its potential as a resource for making meanings “in terms of the choices that are available, the interconnection of these choices, and the conditions affecting their access” (Halliday 1978: 192). Language is thus not a system of rules, but a system of choices. Halliday here backgrounds the notion of structure that is formally seen as describing the rules for language in favour of the organising concept of a social system. He underlines: “Structure is the syntagmatic ordering in language: patterns, or regularities, in what goes together with what. System, by contrast, is ordering on the other axis: patterns in what could go instead of what. This is the paradigmatic ordering in language.” (Halliday 2004: 22) Going back to the linguistic framework of system-structure theory by Firth (1957) and its accentuation of the distinction between syntagmatic and paradigmatic axis, Halliday sets apart the paradigmatic axis as the primary mode of organisation from which the speaker makes choices to construct meaning. These choices are part of a ‘system network’ that provides all available choices drawn from the meaning potential. The system network lists proportionalities that are mutually interrelated and available in a social context. Consequently, the meaning potential of language is directly conjoined with the concept of system: “In order to interpret what is observed [the observed facts of ‘text-in-situation’ for example], however, we have to relate it to the system: (i) to the linguistic system, which it then helps to explain, and (ii) to the social context, and through that to the social system.” (Halliday 1978: 192)

i i

i

i

i

Film Discourse Interpretation 173

This is where the concept of system and that of social construction are finally connected: meaning, as is assumed, is established through interaction, through social behaviour in a society. This social behaviour is already observable within the process of language acquisition: “Language arises in the life of the individual through an ongoing exchange of meanings with significant others. A child creates, first his child tongue, then his mother tongue, in interaction with that little coterie of people who constitute this meaning group. In this sense, language is a product of the social process.” (Halliday 1978: 1) Meanings thus arise in a constantly ongoing process of exchange between the members of a culture. Those simultaneously reflect and act on things as well as affirm their status and roles in the culture and therefore establish the social structure while using language (cf. Halliday 1978: 2). Its potential as a semiotic resource evolves through the relationships that exist within this structure and that are established through the social interactions. But besides language as one resource of social activity, there are a great number of other semiotic systems operating simultaneously as is the case, for example, in film. Different semiotic resources, comparable to language in that they are all able to create meanings, work together in a text as an ongoing process of meaning construction. The concept of resource has therefore been extended to other semiotic modes which are then defined as “the actions and artefacts we use to communicate” (van Leeuwen 2005: 3; see also Kress and van Leeuwen 1996; Kress and van Leeuwen 2001) and to form meaningful texts such as films that offer a vast number of possible interpretations. These texts are not only composed of facial expressions and gestures, but also of music, images and other technically produced resources that have to be seen in the context of social semiotics. They are embedded in social communication and therefore have “a theoretical semiotic potential constituted by all their past uses and all their potential uses and an actual semiotic potential constituted by those past uses that are known to and considered relevant by the users of the resource, and by such potential uses as might be uncovered by the users on the basis of their specific needs and interests.” (van Leeuwen 2005: 4) The social context is then the environment that delivers principles or regularities that show the use of specific semiotic resources; it is thus the discursive dimension in which the semiotic potential of a resource can be analysed. This semiotic potential is described by van Leeuwen (2005) according to the Hallidayan concept of meaning potential, but differs from this in that it includes the dimension of ‘affordance’ which “also brings in

i i

i

174 Multimodal Narrative Construction in Filmic Text

meanings that have not yet been recognized, that lie, as it were, latent in the object, waiting to be discovered” (van Leeuwen 2005: 5). Furthermore, van Leeuwen underlines the plurality of discourses that has to be taken into account in a social semiotic view, since one aspect of reality can be established in different ways of making meaning depending on the situation and context. Hence, discourses derive from particular actions people perform in their social communication in particular situations. It is exactly this aspect which the approach presented in this book addresses and tries to follow by providing a framework for film discourse analysis. Discourse is even seen in a clearly wider sense than is expressed in terms of “extended text” including the procedural characteristics of communication and meaning-making, defining it as “socially constructed knowledge of some aspect of reality” (van Leeuwen 2005: 94) that comprehends not only clearly defined and predesigned meanings, but also the potential for new and nonspecific meanings. In film, these meanings are constructed dynamically in an ongoing process of constantly changing arrangements of different semiotic resources, which is at least a similar process to the described exchange of meanings in communication. Film is thus one example for the outcome of social behaviour in a culture that can broaden our comprehension of the social construction of the world, as has been pointed out by O’Halloran (2004): “While film is evidently staged and directed behaviour with sequences which have been edited to achieve particular effects, the analysis of this medium is at least a step in understanding semiosis in everyday life.” (O’Halloran 2004: 110) As has been shown in chapter 3, a detailed analysis of the filmic context thus gives information about the principles of using different semiotic resources in this context and their meaning potential. The second general perspective taken in the SFL-framework is that of attributing three universal functions to language in social contexts; this is commonly described as the metafunctional organisation of language resources.1 Halliday (1978) distinguishes the three functions ideational, interpersonal and textual, whereby the ideational metafunction has two subfunctions, experiential and logical. These functions organise the linguistic system, since it is “functional in origin and in orientation, so that in order to understand the nature of the linguistic system we have to explain it as having evolved in the context of this set of basic functions” (Halliday 1978: 47). The functions can each be illustrated as follows. The ideational metafunction describes the general resources that are activated to construct and represent experiences in and with the world. Its 1 “Meta” here refers to the abstractness of the functions: “they represent functions of language as incorporated into the linguistic system” (Halliday 1978: 50).

i i

i

i

i

Film Discourse Interpretation 175

subdivision into the experiential and logical metafunctions is related to the construction of these patterns of experience: The experiential function represents states and processes in the world as well as the actors and recipients that directly undergo these processes and events. In the context of functional grammar, the experiential function is a sentential function by which events, actions and states are expressed. The logical function, in contrast, features the possible logical relations between the different patterns of experiences. Those could be relations of coordination, addition, variation, etc. Halliday accentuates that these relations “constitute the logic of natural language [. . . ]” (Halliday 1978: 49). They specify how meanings are combined paratactically or hypotactically and therefore contribute to the textual unfolding. The interpersonal metafunction represents the social aspect of language that is described above. In a social context, speaker and listener are participants in a speech situation where language not only expresses reality, but also enacts relationships. Hence, it is an instrument to exchange attitudes and feelings among speakers. Halliday calls the corresponding sentential function the “modal element” which “expresses the particular role that the speaker has chosen to adopt in the situation and the role or role options that he has chosen to assign to the hearer” (Halliday 1978: 46). The textual metafunction provides resources for the discursive organisation of the ideational and interpersonal meanings in a text as it unfolds in an interaction. It is thus a unifying function that combines the two other metafunctions and can only be actualised in this combination. Halliday underlines that it is this function which is particularly responsible for the unfolding of a text: “The textual component represents the speaker’s text-forming potential; it is that which makes language relevant. This is the component which provides the texture [. . . ]” (Halliday 1978: 112f.). Martin and Rose (2003) point out that the three functions are inextricably connected; they work simultaneously. Hence, discourse can be analysed from these perspectives by analysing different patterns of meaning that realise these functions. Van Leeuwen (2005) explains further that the metafunctional organisation also plays a role in the development of the social semiotic paradigm: Kress and van Leeuwen (1996) for example assumed that all semiotic systems have the potential to fulfil the three metafunctions. Today, recent accounts in Multimodal Discourse Analysis (MDA) generally apply the metafunctionally organised paradigm to the analysis of different semiotic resources, for instance Meng (2004) for a museum exhibition, Jaworski and Thurlow (2009) for gestures and movement, and O’Halloran (2004) for film. It is underlined by several researchers that building on the idea of SFL and the metafunctional paradigm makes it possible to “model the relation between intrinsic organisation of language systems according to functional principles and the extrinsic contextual motivations” (Tseng

i i

i

176 Multimodal Narrative Construction in Filmic Text

2009: 56; see also Jewitt 2009). As Tseng further points out, the SFL framework enables us to transfer these parameters to other discourse kinds such as film and to analyse the interrelationships of their devices, contextual conventions and discourse dimensions. Since the framework operates cross-stratally, it allows “developing analytical methods for interrelating film devices, the corresponding discourse dimensions they configure to realise and each particular communicative function these devices fulfil” (Tseng 2009: 58). The dynamic view of discourse semantics as a metafunctionally diversified stratum is thus applied to the context of multimodal texts and it is assumed that the devices function as text-building resources which construct and maintain the coherence of the text. Pioneering works here are those of van Leeuwen (1991), Lemke (1998), O’Halloran (1999), Martinec and Salway (2005), Liu and O’Halloran (2009), and others who pursue the analysis of semantic relations in multimodal text as an extension of linguistic discourse analysis to nonverbal semiotic systems. This research often focuses mainly on two distinctive modes which are related to each other, for example image-text relations, and which are described with the Hallidayan categories of logico-semantic relations. The efforts in this context provide a basis for the development of a combined framework for film analysis that includes the different semiotic resources and their intersemiotic meaning-making. A further example for the application of SFL to film analysis is Tseng (2009, 2013), who takes the functional perspective of discourse semantics to analyse cohesive reference and identification in filmic discourse. Her framework is thus another approach to filmic coherence that systematically detects the cross-modal realisation of meaning in film through the interaction of modalities. Building on the functional linguistic research on cohesive reference by Halliday and Hasan (1976) and Martin (1992) on the one hand, and the cognitive film theoretical model by Bordwell and others (Bordwell 1985, 1989; see section 1.3) on the other, Tseng provides a method for analysing thematic configurations in different film extracts that reveals a content-based narrative structure with reference to the viewers’ inferential process of constructing meaning. The starting point for analysis is the discourse system of identification outlined by Martin (1992), which provides choices for the description of a character, of objects and settings within film images (cf. Tseng 2009, 2013). The functional perspective is then drawn upon to ask “whether we can also establish a filmic system of identification whereby an interpretation of the characters, objects and places (filmic participants) is established by relating them to something else, that is, how the interpretation of a reference item in film is constructed by searching for its identity in the text or in the context.” (Tseng 2009: 66f.)

i i

i

i

i

Film Discourse Interpretation 177

Tseng proposes a “meta-view of analytical strata” which takes the notion of discursive meaning as the analytical starting point and tries to combine contextual interpretation with systemic analysis. She organises the analytical strata of film discourse analysis as shown in figure 4.2.

Figure 4.2: The analytical strata of film discourse analysis adapted from Tseng (2009: 44) “Filmic data is transcribed (process x) for the use of discourse analysis (process y). At the higher level, the construction of interpretation then can be realised (process z) drawing on the process of discourse analysis, which is in turn realised by the data transcription meeting the need of discourse analysis.” (Tseng 2009: 46f.) This model demonstrates the need for detailed analysis of the filmic data in view of its discursive meaning potential and its necessary interconnection with the level of interpretation. The stratum of transcription presupposes an in-depth protocol of the individual film devices, or of the different modalities in film according to their purpose in the discourse stratum. The stratum of film discourse analysis then delivers the discourse analytical method to theorise the organisational features of film cues to construct discourse (cf. Tseng 2013). The underlying basis for Tseng’s thematic analysis is the Bordwellian model of film comprehension (Bordwell 1985, 1989), which has already been mentioned in section 1.3. In reference to this model, it is the materials and devices that establish discourse dimensions in film, i.e., dimensions of form and style which form, following Bordwell (2008), the ‘poetics of cinema’. From the social semiotic point of view that is taken by Tseng as well, the film data thus “are semiotic units/devices within a particular discourse dimension that is being investigated” (Tseng 2009: 48). These semiotic units then form socially constructed meaning patterns on the

i i

i

178 Multimodal Narrative Construction in Filmic Text

stratum of discourse dimensions which for Tseng is the starting point of SFL-based film analysis. Her account demonstrates that the application of the SFL-framework to the analysis of multimodal filmic text is an appropriate method for a systematic analysis of film discourse interpretation. Applying this framework to the research in this book, the metafunctional organisation of the filmic discourse and its characteristics as well as the stratification of the various levels of discourse analysis deliver the designated parameters for a systematic examination of the idea of filmic textuality. Since the framework allows the classification of discourse units in the field of several discourse systems and on the basis of their functional use, especially the patterns of structure and coherence can be identified systematically.

4.2

The Strata of Film Discourse Interpretation

In multimodal research, the need for approaches which are able to identify “the defining attributes of a multisemiotic text” (Liu and O’Halloran 2009: 368) is explicitly given. The overall aim of these approaches is on the one hand to investigate how semiotic resources integrate (cf. Eckkrammer 2004; Martinec and Salway 2005; Zhao 2009), and on the other hand to outline which resources form significant elements within the meaning-making process. With the help of the proposed formal and functional framework, this integration and interdependence of the semiotic resources can now be made visible in terms of the stratificational organisation mentioned above. This stratificational organisation will represent a direct alliance between formal and functional approaches to discourse semantics. In order to find a systematic meaning hierarchy that displays the different layers of organisation in film discourse analysis, it is now necessary to elucidate in detail from the functional perspective given in section 4.1 the different points of view of film discourse at work in the examples discussed in chapter 3. In section 4.2.1, it will therefore first be shown how the different semiotic resources work together within the logical forms of the discourse segments and how their intersemiosis can be considered in a functional analysis. The following section, section 4.2.2, will then give an overview of how the diverse functions of the resources in various contexts can be characterised more accurately by means of their metafunctional diversification in filmic discourse. The information sources which interact during the inference process of building the logical forms of the discourse segments will be described in more detail in section 4.2.3 in order to give an overview of this information that is not directly formulated, i.e., not obvious in the filmic text (see the quotation from Bordwell 2011 on page 169), but nevertheless influences meaning construction. Finally, in section 4.2.4, the various levels described previously will be brought together as the strata of film discourse analyis.

i i

i

i

i

Film Discourse Interpretation 179

4.2.1

Intersemiotic Meaning Construction in Filmic Discourse

The main distinction which has been made between the modalities within the logical forms during the analysis lies so far on the level of the sensory channel. Other differentiations are given within the detailed description, as for example done in “[a:] volume: increasing (t)” (see table 3.18 in section 3.2.2). A more fine-grained distinction of the modes, sub-modes and features given by Stockl ¨ (2004) (see section 2.2.1) will not be pursued within the presented framework. The basis here is the general semiotic principle that “the different modes have technically become the same at some level of representation [. . . ]” (Kress and van Leeuwen 2001: 2). As has already been pointed out, it is mainly the context and actual use of the respective mode which gives information about how it is involved in the meaning-making process as a whole. For this reason, it is actually the logical formulation of the eventuality that points out the resources which play an important role within the inference process. Many eventualities can for example be inferred because of the visual depiction of the action processes. Sound and audio track then act supportingly with regard to the formulation of the eventuality. In the short example in table 4.1, the notation explicitly outlines the cooperation between the visual and the auditory level in that it enlists the two referents which directly allow the inference of listen, refering to resources on different sensory levels. eπ13 = listen [v] camera panning (c) [v] Wiesler at attic storey (d) [v] colouring: grey/green (e) [a] piano playing (f) d, f |∼ listen (eπ13 ) Table 4.1: Exemplary logical form with intersemiotic references, taken from chapter 3 The eventuality listen is inferred because the audio track of the piano playing (f) is identified as the device to which the participant on the visual level (d) is listening. The formulation thus describes how the resources integrate to produce the semantic content of the shot. Furthermore, it allows an explicit definition of the process type which can be inferred from the narration. In this example, the referent, the protagonist Wiesler (d), functions as a Senser within the mental process of listening to the Phenomenon, the piano playing (f). A first description of these process types has already been given in section 2.2.1. On the basis of a taxonomy by Halliday (2004), Kress and van Leeuwen summarise narrative processes in language and visual communi-

i i

i

180 Multimodal Narrative Construction in Filmic Text

cation as a “useful background for analysing representation in multimodal texts” (Kress and van Leeuwen 1996: 78). Similar to Tseng (2013), who lists the narrative processes in language and visual communication for the description of cross-modal filmic action chains, these processes support a further category of examination which takes the interpretation of the logical form as a basis and which allows a functional consideration of resource use within the logical forms. The referents which have been formulated in the last line of the logical form can then additionally be described according to the distinction made between the named participants of the narrative processes (Actor vs. Goal / Senser vs. Phenomenon). Their definition takes into account the various realisations of the referents in different modes and therefore indicates whether the processes are established mono- or cross-modally. In table 4.2, the logical forms found in the extract from Am´elie are repeated similarly in order to outline their narrative processes and the intersemiotic meaning construction in greater detail. The events which are described by the logical forms can be analysed as shown in table 4.3. It becomes visible that, in most of the cases, the processes are realised cross-modally and that the discourse referents depicted in the formulation of the eventuality are identified as the respective participants of the process types. The analysis clearly resembles already existing examinations of process types in film, for example as done by van Leeuwen (1996) and Tseng (2013), and therefore will not be further extended here. Nevertheless, the basis for the analysis taken from the analytical results of this book, namely the formulation of the eventuality within the logical forms, offers a more detailed description of the participants within the processes which have been interpreted due to inferential reasoning. By this means, they correspond to some extent to the concept of ‘interest’ van Leeuwen mentions within his analysis: “The option we then choose will be determined by our ‘interest’, to use the term Kress (1993) employs in this connection— what we are interested in focusing on, the communicative aims we are interested in achieving, and so on. This interest may be very much our own, arising from our own particular biography, but it may also be an interest we share with some group to which we belong and to which we have to conform to a greater or lesser degree, be it voluntarily or out of economic necessity.” (van Leeuwen 1996: 104) ‘Interest’ here also points to the recipient’s expectations and presuppositions, which are established within a sequence of processes and their participants and which have to be satisfied within the further unfolding of the discourse. The description of how the recipient monitors the roles and relations between the participants in the story-world can then depict

i i

i

i

i

Film Discourse Interpretation 181

eπ1 = (dialogue) eπ1a = say [v] entrance hall (m) [v] male actor: postman (n) [a] “How’s it going?” (o) n, o |∼ say (eπ1a )

eπ1b = say [v] entrance hall (p) [v] female actor: Mrs. Wells (q) [a] “It’s always better. . . ” (r) q, r |∼ say (eπ1b )

eπ2 = (handover) eπ2a = say [v] entrance hall (s) [v] male actor: postman (t) [a] “This is for you.” (u) t, u |∼ say (eπ2a ) eπ2c = hand over [v] entrance hall (y) [v] male actor: postman (z) [v] female actor: Mrs. Wells (a) [v] letter (b) z, a, b |∼ hand over (eπ2c )

eπ2b = say [v] entrance hall (v) [v] female actor: Mrs. Wells (w) [a] “Me?” (x) w, x |∼ say (eπ2b )

eπ3 = open [v] entrance hall (c) [v] female actor: Mrs. Wells (d) [v] letter (e) d, e |∼ open (eπ3 ) eπ4 = read [v] slowly moving camera (f) [v] entrance hall (g) [v] female actor: Mrs. Wells (h) [v] letter (i) [a] gentle whispering (j) h, i, j |∼ read (eπ4 ) eπ9 = weep [v] living room (m) [v] female actor: Mrs. Wells (n) [v] letter (o) [a] sound of weeping (p) n, p |∼ weep (eπ9 )

Table 4.2: Logical forms taken from the analysis of the extract from Am´elie (see section 3.3.2)

i i

i

182 Multimodal Narrative Construction in Filmic Text

eπ 1

n, o |∼ say q, r |∼ say

eπ 2

t, u |∼ say

z, a, b, |∼ handover

eπ 3

d, e |∼ open

eπ 4

h, i, j |∼ read

eπ 9

n, p |∼ weep

verbal processes, cross-modally realised: sayers (n), (q); utterances (o), (r) verbal processes, cross-modally realised: sayers (t), (w); utterances (u), (v) transactional process, mono-modally realised: actor (z); goal (a), (b) transactional process, mono-modally realised: actor (d); goal (e) transactional/verbal process, cross-modally realised: actor/sayer (h); goal (i); utterance (j) mental/verbal process, cross-modally realised: sayer (n); utterance (p)

Table 4.3: Narrative processes in the extract from Am´elie exactly this process of interpreting the presuppositions and inferring the requested coherence between the different events. The dialogue in the extract from Am´elie, for example, is maintained by a sequence of verbal processes which are realised cross-modally. These processes include the two sayers who can also be inferred as actor and goal in the transactional process following. Their voice track points to the letter which is handed over to the second character as the dominant participant in the following events/processes. This repetition of the actor has been identified as a so-called cohesive tie within an action chain (cf. Tseng 2013), which, in the analysis provided in this book, can furthermore be formalised as a logical participant of the inference process building the basis for the interpretation of the discourse structure. The analysis of the different resources within the logical form thus not only gives a concrete view of how the resources operate together, but also offers first evidence for the “principled, nonrandom connections between textual clues and participant roles in stories” (Herman 2002: 136). Herman furthermore points out that this functional description of narrative processes and their participants turns out to be a starting point for comparing and contrasting various story types and therefore supports the framing of questions of genre:

i i

i

i

i

Film Discourse Interpretation 183

“[. . . ] process types can themselves be characterized as coding strategies in which different sorts of participant roles and relations are canonical or preferred. How a given story fits into the larger system of narrative kinds thus entails stronger or weaker preference rankings for role assignments vis-`a-vis storyworld participants.” (Herman 2002: 143) Tseng emphasises that this applies as well for filmic process types and that the analysis of the textual structure and its basic entities in different film extracts enables the description of recurring patterns and thematic configurations which may give reason for assuming certain genre structures (cf. Tseng 2013). The fine-grained analysis of the discourse referents in the logical forms carried out within this book is thus a further contribution to the wide range of research in multimodal film analysis which aims at collaborating with already existing approaches by adding a more formal depiction and identification of how the resources work together to effectively construct the meaning verbalised by eventualities. In contrast to accounts which relate the different resources by concrete semantic relations (cf., for example, Martinec and Salway 2005; Matthiessen 2007; Liu and O’Halloran 2009), the formalisms provided in this book account for a more comprehensive process of logically based reasoning about the best interpretation of the text. In combination with the functional description, it organises the filmic devices according to their task within the meaning-making process and the influence they have on the inference process. These tasks will be elucidated in more detail and from a second perspective in the following. 4.2.2

Metafunctional Diversification

Based on the assumption that semiotic systems have the potential to fulfil different metafunctions (cf. section 4.1), the intersemiosis of the resources within the logical forms of the discourse will be considered from this metafunctional perspective in the following. Whereas many filmic devices operate ideationally and construct the represented world within the film, there are as well resources which undertake the task of creating interpersonal situations or explicitly structuring the text. A detailed description of these tasks with the existing meaning categories therefore allows a stronger identification of the kinds of information within the logical forms and outlines similarities and differences that will help to get a better understanding of how meaning in film is created. Ideational Content Resources operating ideationally include those filmic devices which construct the concrete representation of the content: the events and actions of the story-line, things and characters which are

i i

i

184 Multimodal Narrative Construction in Filmic Text

part of these events and their temporal and spatial specificities. Van Leeuwen (1996) points out that in particular the dynamicity of the unfolding filmic discourse includes certain specifics of the moving image which are different to verbal meanings, namely the aspect of motion (of the camera and the characters and things in the image) and the potential of being edited into sequences or events, taken as the minimal unit in this book (cf. van Leeuwen 1996: 82). Movement, as has been pointed out above, is one of the main factors to create narrative processes and therefore often builds the basis for interpreting eventualities. The dynamicity of the camera may thus play an important role for the meaning-making, as shown in the example in table 4.4. The construction of the setting, the main character of the film and eπ4 = panic [v] protagonist at Times Square (m) [v] camera: 360-degree tracking shot (n) [v] camera: zooming out (o) [v] signs (p) [v] CD, magazine covers (q) [v] photographs (r) [a] sitar, rhythmic drums (s) [a] volume: increasing (t) [a] rhythm: getting faster (u) m, n, t, u |∼ panic (eπ4 ) Table 4.4: Example logical form taken from the analysis of Vanilla Sky (see table 3.18 in section 3.2.2) furthermore the atmosphere around him is summarised in this example as panic. The camera supports the inference of this predicate in that it maintains a certain dynamic movement in the 360-degree tracking shot and the zooming. The requirement that coherence be achieved also naturally provides potential interpretations of technical devices, such as the tracking shot. The top-down and the bottom-up are thus ideally combined. Certainly, this logical form is not only an example for an ideationally constructed representation of the content, but also includes interpersonal as well as textual aspects of meaning. Their use will be depicted in some further examples in the following. Interpersonal Content The interpersonal metafunction refers to the social aspects of semiotic resources, in particular to the participation in dialogue situations and the exchange of attitudes and feelings.

i i

i

i

i

Film Discourse Interpretation 185

In filmic text, there is no direct exchange or participation of the spectator. However, concrete references to the recipients and their social states may be established via the use of devices which construct interpersonal relationships and which influence the distance to characters in the text. This can for example be achieved by a direct identification of the characters within the story-line, as given in the logical form in table 4.5. In this example, not only the social status of the two characters within the story, namely their family relationship, is constructed by the intersemiosis of the visual and the auditory track, but there is also a possibility for the recipient to establish a relationship to, or at least an attitude towards, the characters as well. eπ6 = talk [v] actor: male person (m) [v] actor: female person (n) [v] represented participant: TV (o) [a] “Harold, not again the TV!” (p) [a] “Aw, Ma, come on, Ma.” (q) m, n, p, q |∼ talk (eπ6 ) Table 4.5: Example logical form taken from the analysis of Requiem for a Dream (see table 3.36 in section 3.4.2) On the one hand, the discourse referents, which have been described in the last line of the eventuality, stand mainly for the sayers and utterances of the verbal processes in the event and therefore construct the ideational content of the discourse. On the other hand, they function interpersonally in that they outline the social relations between the two characters. Van Leeuwen furthermore points out that these relations are often expressed by distance and angle, which can create a certain expression of attitude for example (cf. van Leeuwen 1996: 90). By this means, the camera again plays an important role. In the extract from Am´elie, for example, it slowly zooms in on the protagonist of the scene while she is sitting on her sofa and reading the letter. In the end, she is weeping and the camera clearly focuses on this emotional aspect. Textual Content Resources which operate on the textual level directly influence the textual composition of the discourse. Normally, they refer to both the ideational and the interpersonal meanings which can only be realised in combination with the textual components. One example for this combination is given in table 4.6, taken from the analysis of The Lives of Others (see section 3.1.2). The camera panning as well as the piano playing in both logical forms are continuous devices here which create a dynamic transition between

i i

i

186 Multimodal Narrative Construction in Filmic Text

eπ12 = play [v] hands on piano (w) [v] colour: brown/beige (x) [a] piano playing (y) [v] camera panning (z) [v] D.’s stature (a) [v] side view (b) w, y |∼ play (eπ12 )

eπ13 = listen [v] camera panning (c) [v] Wiesler at attic storey (d) [v] colouring: grey/green (e) [a] piano playing (f)

d, f |∼ listen (eπ13 )

Table 4.6: Example logical forms taken from the analysis of The Lives of Others (see section 3.1.2) the two events and therefore relate the two different settings and the characters to each other. Although the two events differ from each other in their semantic content, they are textually conjoined by a temporal sequentiality which is maintained by both the sound track and the camera movement. The two referents thus not only construct the ideational content of playing and listening, but influence as well the film’s texture and overall coherence. It has now been shown that the functional organisation of the semiotic resources depicts a more precise way of describing their tasks within the logical forms of filmic discourse and the meaning-making process to build these forms. It is thus not only possible to outline that the resources and relations have diverse functions in various contexts, but also how these functions can be determined more accurately. This is a first step towards a more systematic analysis of how meaning is constructed in filmic text. 4.2.3

Knowledge Sources for Building Logical Forms

As has generally been described in section 2.2.1, the construction of the logical forms of discourse already depicts an extensive process of inferring the discourse’s representation. Based on the general assumption within approaches to discourse semantics, this inference process exhibits “a highly complex interaction between many information sources” (Asher and Lascarides 2003: 94). The analyses in the preceding chapter illustrate this interaction in that they outline how different knowledge sources are involved in reasoning about the best description of the logical form.

i i

i

i

i

Film Discourse Interpretation 187

In the example in table 4.6 above, it was the knowledge about the film’s general story-line and the setting depicted in the image which explained that the protagonist is listening to a sound from another setting. This information cannot be transferred by the shot itself, but is presupposed by the preceding content. The process of encoding the semantic representation of the discourse is thus based on knowledge sources, such as world knowledge and the knowledge of watching films as well as the knowledge presupposed by the preceding discourse context. A more precise view of these sources will now be given in order to be able to describe the different layers of the strata of film discourse interpretation in more detail. General World Knowledge General world knowledge is presumably the biggest category within the various knowledge sources. It provides information for the basic and constitutional reading process of events and actions depicted in the shots. This could for example be the knowledge that the images shown at the beginning of Requiem for a Dream resemble a TV show or the fact that the protagonist in Vanilla Sky is running around Times Square in New York, identified due to well-known images. In general, all elementary information about objects in images, about sounds with or without any source visible in the image, or about specific musical elements can be classified according to categories of general world knowledge. By means of this knowledge source, it is possible to distinguish different settings and to identify characters. It is therefore taken as the information which can normally be inferred by the recipient while reading and interpreting the text. This category refers to those kinds of interpretation codes which have been distinguished by Bateman and Schmidt as ‘reality codes’, i.e., “schemes of interpretation derived from our understanding of the real, social and physical world” (Bateman and Schmidt 2011: 142). Bateman and Schmidt introduce these codes in order to make allowance for meanings that are not directly anchored within the film’s diegesis, and thus have to be interpreted on the basis of the recipient’s perception and comprehension (cf. Bateman and Schmidt 2011: 142f.). Parallel to the work of Bateman and Schmidt, this book also takes these knowledge sources as assumable information the recipient has for the interpretation of a film. Domain Knowledge / Specific Expert Knowledge A definitely more specific and thus not entirely assumable knowledge source is that of domain knowledge or specific expert knowledge, which may play a role in different scenes or entire films. The information provided by this knowledge source may support the understanding of individual coherencies within a story, for example in a film adaptation which presupposes the knowledge of the characters, the setting or the general story-line of the novel.

i i

i

188 Multimodal Narrative Construction in Filmic Text

It may as well be specific technical knowledge or the capability to identify specific details which may not be supposed to be known by all recipients. The recognition of details in the drug-taking scene in Requiem for a Dream may, for example, be a matter of specific domain knowledge. The same applies to the sounds and noises in the extract from Am´elie which may not be recognised by all recipients. Film Knowledge / Narrative Knowledge One more specific area of domain knowledge is that of film knowledge. This plays an important role for the meaning construction in filmic discourse, providing information about the general composition of films as the interplay of visual and auditory resources as well as about filmic specificities such as, for example, different montage techniques (cross-cutting or split screens), camera movements and effects, etc. Information about directors and actors as well as genre knowledge belong to this category just as does knowledge about other films allowing the comprehension of intertextual references. Following the distinction by Bateman and Schmidt (2011), this category refers to the ‘representation codes’, i.e., “schemes of interpretation that derive from the nature of the unfolding audiovisual material as discourse” (Bateman and Schmidt 2011: 142). The analysis of The Lives of Others, for example, contains the description of the cross-cutting scene which is recognised as a filmic specificity and typical design element used in a variety of films. The identification of the female voices in Vanilla Sky, in contrast, may be a more specific process of recognition which is based on the fact that the two actors and their voices are presumably well-known and paratextual information has informed the recipient about their participation in the film. Furthermore, Pen´elope Cruz as the female actress may be known as the one who played the female protagonist in the first version of the film, Abre los ojos. In combination with film knowledge, knowledge about how films tell a story, narrative knowledge, plays an important role for film interpretation. As Bordwell (1985) points out, this knowledge is the crucial qualification for comprehending a film: “To make sense of a narrative film, however, the viewer must do more than perceive movement, construe images and sounds as presenting a three-dimensional world, and understand oral or written language. The viewer must take as a central cognitive goal the construction of a more or less intelligible story.” (Bordwell 1985: 33) In Bordwell’s opinion, this knowledge is a deep-seated quality of human comprehension and lets the recipient be prepared for different story construction patterns when watching a film.

i i

i

i

i

Film Discourse Interpretation 189

Discourse Context Knowledge One of the most important knowledge sources within film discourse interpretation is that of discourse context knowledge. The information provided within this knowledge source enables for example the (re-)identification of characters within a film. The example taken from The Lives of Others (described above) is based on the fact that one of the film’s main topics is the monitoring routine in the German Democratic Republic. The interpretation of the cross-cutting scene in the extract is thus predicated on the fact that the recipient watched the beginning of the film and therefore knows this fact, as well as the characters and their roles within this situation. This knowledge then can be presupposed for the interpretation. The basis for the assumption of this knowledge source is given on the level of dynamic semantics which takes the dynamically unfolding discourse as the starting point. The interpretation of an utterance or sentence always depends on the discourse context in that it is “a relation between a set of ‘input’ contexts which represents the content of the discourse prior to the sentence being processed, and a set of ‘output’ contexts which represents the content of the discourse including that sentence” (Asher and Lascarides 2003: 41; see section 2.2.2). On the level of constructing the logical forms of the discourse segments, this relational meaning-making already comes to the fore. The semantic content of some discourse segments can sometimes only be interpreted on the basis of information available within the discourse context of the segment. The same applies to information which has to be revealed retrospectively due to new information which gives another reason for inferring the semantic content. These various knowledge sources are assumed to operate during the process of discourse interpretation and the building of the logical forms. Although it is not possible to make explicit every source of information as the respective knowledge base for the constitution of eventualities, it is absolutely necessary to take into account a general view of these sources in the analysis. It is then possible to shed light on conditions for inferential reasoning which are basically discursive, i.e., for which the text itself asks: “in other words, discourse semantics needs to be done prior to referring to background knowledge in order that access to background knowledge be appropriately guided” (Bateman and Schmidt 2011: 283). Nevertheless, these conditions have to be specified pragmatically, with regard to the actual context and the respective realisation of the semiotic resources. With reference to this pragmatic information, the description of how the film form guides the recipient’s activity in constructing a narrative story out of the textual cues offered by a film can be made more explicit. How the framework of film discourse interpretation includes all these levels within the analysis can now be explained by a more systematic combination of the various strata that have been described so far.

i i

i

190 Multimodal Narrative Construction in Filmic Text

4.2.4 Summary: A Stratificational View of Filmic Logical Forms The various organisational levels which have been pointed out above in order to find more systematic ways for describing and classifying the semiotic resources of the logical forms and their meanings will now finally be manifested within a comprehensive overview of the strata of film discourse interpretation. This is depicted graphically in figure 4.3.

Figure 4.3: The strata of film discourse analysis The filmic data constitute the basic level within this hierarchy. They represent the elementary substance which is arranged in diverse ways as the semantic content of the logical forms. This arrangement can furthermore be considered according to its individual metafunctional use as described in section 4.2.2. The various knowledge sources pointed out in section 4.2.3 operate as a further layer of the discourse analysis, which influences the processes within the logic of information content—at least representing parts of the context that is assumed for the interpretation. Further defining principles are the glue language and discourse update (see sections 2.2.5 and 2.2.6), whose significance for building film discourse structure is also a matter of this stratum. The three layers are finally enlarged by a fourth stratum, that of the social activity. It is the overall environment in which social communication takes place and which delivers regularities for the

i i

i

i

i

Film Discourse Interpretation 191

use of the semiotic resources and their analysis (see section 4.1). The classification thus displays a process of prestructuring the filmic devices which play a role for the overall interpretation of the discourse. It enables us to systematise the various meaning categories that are involved in the construction of the logical forms of filmic discourse. Constructing a narrative filmic text via inferences is a mostly data-driven operation on the basis of the viewer’s perceptual input during the reception of the film. The perceived filmic data is processed within the different strata and levels of film discourse interpretation in order to systematically build meaning structures out of the variety of modalities and their intersemiotic composition. Perception and comprehension thereby build the twofold basis for the viewer’s bottom-up activity: “In comprehending a film, we construe the outputs of filmic perception as representing a hierarchical pattern of actions, a conception, or simply a train of sensuous elements (as in an abstract film). The viewer applies a wide range of knowledge to make sense of film, segment by segment or as a whole, and to give it some literal meaning. Narrative comprehension is the clearest instance.” (Bordwell 2008: 47) However, since bottom-up and top-down processes constantly interact, meaning construction invariably is enhanced by concept-driven processes from above which take into account the social functions and uses of the textual structures. Bordwell calls this evaluation of the context and the production circumstances ‘appropriation’ (cf. Bordwell 2008). Within the framework developed here, this appropriation mostly contributes to defining the constraints of the viewer’s interpretation and therefore builds the contextual and analytical basis for the logic of film discourse interpretation. Tseng further underlines that “the analytical task is not to recount each device perceived as the film unfolds; nor is the theoretical task to describe each possible meaning-making film device” (Tseng 2009: 60). Instead, the dynamic interaction between the operations allows a comprehensive overview of film discourse analysis from both the systematic view of constructing filmic narration in the text and the recipient’s interpretation within the social context. The depiction in figure 4.3 is therefore designed as a representation of the diversity and plurality of this two-way interpretation. In contrast to figure 2.4 in section 2.3, the hierarchisation of the different strata is always accompanied by the notion of metaredundancy (see section 4.1) which underlines the cross-stratal, reciprocal relations between the different levels and which does not only focus on the specific direction of processing the inference operations. As for example became clear during the analysis of the short film WORDS in section 2.1, some situations depicted in the images of this film can only be interpreted and associated

i i

i

192 Multimodal Narrative Construction in Filmic Text

to the respective predicate with regard to general social processes. The scene in which a man and a woman are sitting on a bed (see table 2.2), for example, can only be associated to the preceding scenes if the situation is interpreted as breaking off a relationship. The construction of the logical form and the eventuality break thus depends on the stratum of the social activity and context as an information source which influences the recipient’s general world knowledge and therefore the interpretation of the propositional content. The depiction of the various strata of film discourse interpretation allows the description of all these parts of film interpretation from an integrative perspective and facilitates the understanding of how meaning is constructed within and on the basis of the various levels. Furthermore, it supports the search for connections between the filmic data and technical details on the lowest stratum and the number of sources for the interpretation on higher strata. With the organisation and prestructuring of the filmic devices, these connections can now be made more explicit. This will also help to describe in more detail the processes of the narrative logic of filmic discourse and its influence on the construction of the film’s overall narrative, which will be pursued in the section following.

4.3

The Narrative Logic of Film Interpretation

On the basis of the organisation of the different levels of interpretation in section 4.2, this chapter will give some concluding remarks on the overall principles behind these levels which are generally summarised as the textual logic of constructing narrative in filmic text. Within this book, this logic is understood as an explicitly narrative logic, which “depends upon story events and causal relations and parallelisms among them” (Bordwell et al. 1985: 6) and which interacts with the system of time and space in film. With the help of the framework provided and the results of the analysis, it is possible to set out in detail how this logic works and prompts the recipients to make sense of the filmic text by guiding his/her way through the text. “Narratives flow linearily in time, yet they mediate between worlds that are largely nonlinear: both the world of action, with its manifold possibilities, and our mental models of it are like complex, multidimensional maps, representing not just objects but also relations, in webs of immense connectivity. Narratives by contrast, are like specific paths taken through these worlds—partial, linear views of nonlinear environments.” (Doxiadis 2010: 81) It is these specific paths which can be reproduced for film by outlining the unfolding of its discourse structure with the logic of film discourse

i i

i

i

i

Film Discourse Interpretation 193

interpretation. The capacity to connect the events and actions of the nonlinear world to a linear, coherent representation is exactly what has been described as the logic of building filmic discourse structure. However, it has become clear in the analyses that this narrative logic is not exclusively based on a spatiotemporal consequence as the basic assumption of filmic narrative, but also includes a more abstract configuration of the discourse’s consecutiveness which refers to causal, associative and elucidatory links maintained by the set of discourse relations. Interpreting a film and constructing its narrative is therefore a combination of various operations on a filmic story, as for example formulated by Bordwell in the following: “People tacitly assume that a story is composed of discriminable events performed by certain agents and linked by particular principles. People also share a sense of what is secondary to the story’s point and what is essential to it. Third, and most significant from a Constructivist standpoint, people perform operations on a story. When information is missing, perceivers infer it or make guesses about it. When events are arranged out of temporal order, perceivers try to put those events in sequence. And people seek causal connections among events, both in anticipation and in retrospect.” (Bordwell 1985: 33f.) The quotation points out which basic principles are necessary for the construction of the narrative: time and space as well as causal relationships between the events of the story. These principles play an important role for the meaning construction and the analysis of the discourse structure in that they influence the determination of the default predictions that must hold for the inference of the relations. In the framework provided, these principles are thus mainly incorporated into the definitions of the discourse relations, their meaning postulates and default axioms. As pointed out in section 2.2.4, the meaning postulates include abstracted information which describes the essential operations of narrative comprehension described above, namely (spatio-)temporal consequences for and constraints on the semantic properties required to maintain a meaningful and coherent story. All relations are therefore defined on the basis of the temporal, and in some cases as well spatial, order they construct within a specified topic. This goes back to the assumption that narrative genres provide a default relation of temporal succession or progression between events (cf., e.g., Ankersmit 1983; Wildgen 1994: 149f.; Herman 2002: chapter 6). Accordingly, Asher and Lascarides point out that Narration is the essential discourse relation holding between entities in a verbal, narrative discourse (cf. Lascarides and Asher 2007). It includes the temporal and spatial overlap of two eventualities which is given by the meaning postulates for

i i

i

194 Multimodal Narrative Construction in Filmic Text

the relation. In this regard, Narration has originally been considered to be the default discourse relation of any (verbal) narrative or expository text. When there is no other rhetorical role available, it is assumed that the events are described chronologically. However, it has become clear within the analyses here that this assumption cannot be established in the same way for film. The detailed depictions of the film discourse structures demonstrate that Narration can indeed be found in every structure depicted, but nevertheless, in many cases, other relations have been inferred due to the tighter meaning postulates and default axioms available within the Segmented (Film) Discourse Representation Theory (S(F)DRT) framework. The concrete definitions of the conditions which have to be fulfilled within the process of inferring the relation give a much more restrictive desciption of Narration and the other relations and their role within the discourse. This helps to avoid overgeneralisations or misinterpretations of the narrative content (see for example the remarks on Narration in Asher and Lascarides 2003: 200 and the definitions in section 2.2.4). Furthermore, it has become clear that spatiotemporal consequence as the basic assumption of a narrative was in our examples rarely constructed by the discourse structure, but instead is already inferred on another level of description, namely within the logic of information content. As has been described in detail in chapter 2, the first logic of film discourse interpretation not only delivers the vocabulary for describing the logical forms, but also contains the meaning postulates for the film discourse relations. The meaning postulate for Narration, for example, includes the Spatiotemporal Consequence. Similar conditions are however valid for other relations, since the meaning postulates contained in most cases include a temporal specification, such as the Temporal Consequence of Elaboration or the Temporal Condition for Result (see section 2.2.4). Along with the logical forms of the discourse segments, the systems of cinematic time and space can thus in most cases already be constructed within the first process of interpretation. At this point, the stratificational organisation provided in section 4.2 comes to the fore again and allows a description of these processes within the various strata of film discourse interpretation. As depicted in figure 4.3, the construction of the principles of time and space within the logic of information content can be ascribed to the lower strata of film discourse analysis. The second logic, the logic of constructing the logical forms via discourse update, then operates on the higher strata which are realised by the lower ones. The second logic thus results in an abstract narrative configuration of the discourse’s consecutiveness which is based upon causal, associative and elucidatory links maintained by the remaining set of film discourse relations. The reasoning about these relations involves world knowledge and

i i

i

i

i

Film Discourse Interpretation 195

other information resources. These involve the following list of conditions extracted from the default axioms in section 2.2.4: causeD specif icationD circumstantial inf ormation semantic similarity semantic dissimilarity As the definitions of these conditons and their actual application within the analysis have outlined, the specificity of these descriptions decreases according to their sequential arrangement in this list. causeD , for example, has a relatively stronger granularity of description than specif icationD , which in fact demands that there is some more detailed information in the discourse but nevertheless does not give a concrete definition of this detail information. The conditions semantic similarity and semantic dissimilarity are even less concrete. By this means, each relation describes another level of detail of the discourse which, consequently, has different effects for the interpretation. Within the strata of film discourse analysis, the set of relations can be described within the third layer which includes the glue language and discourse update. As pointed out above, their default axioms can be organised due to their specificity of description. Furthermore, the results of the analysis indicate that the actual context indeed allows inferring a spatial and temporal sequence of events, but that the discursive function of the relationships is often not that of occasion but rather elaboration or expansion. It is thus the relation’s functional dependency of and within the context which turns out to be decisive for the update process. The additional content which influences the inference here is thus not exclusively temporal in nature. As Asher and Lascarides point out for verbal discourse, it is “much more plausible [. . . ] that we infer a causal link in virtue of the presence of a rhetorical link between the propositions” (Asher and Lascarides 2003: 7). The analysis in the preceding chapter has confirmed this assumption for filmic discourse. The default axioms and meaning postulates given for the other relations outline the possible causal and text-structuring connections and demonstrate to some extent which further function the relations might fulfil and how they influence the inference process. This functional dependency of the relations can in turn be related to the general metafunctional diversification which has already been described for the use of the various semiotic resources. In the following, this metafunctional organisation will also be applied to the set of discourse relations in order to motivate a paradigmatic organisation of these relations.

i i

i

196 Multimodal Narrative Construction in Filmic Text

4.4

Towards a Paradigmatic Organisation of Film Discourse Relations

As has already been pointed out above, the process of building discourse structures from the set of relations conforms to the fundamental distinction of the two different axes of description in linguistics, the paradigmatic and syntagmatic, which was first mentioned in section 1.2. Analysing the discourse structure of filmic text is effectively an examination from the syntagmatic point of view, which, on the other hand, is first and foremost based on the paradigmatic choice of discourse relations. These choices directly affect the structural development on the syntagmatic axis and therefore play as well an important role for and within montage. In this spirit, the paradigmatically available set of film discourse relations is taken as the primary mode of organisation, as argued by Halliday (2004) for language. Its task is the systematic description of the paradigmatic organisation among the relations which are available for filmic text and which then, as a consequence, determine the respective discourse structure. Both axes are interdependent and their interaction demonstrates how the analysis of coherence in filmic text sheds light on narrative meaning construction. It thereby contributes to the emphasis Bateman and Schmidt (2011) place on the combined examination of the two axes for a comprehensive semiotic analysis in order to outline the structural consequences of the proposed set of film discourse relations (see section 1.2). The structural results of the analysis in this book have been successfully shown in each depiction of the unfolding discourse structure in the preceding chapter. These structures argue that the list of film discourse relations provided as paradigmatically available choices is sufficient for the analysis of filmic text. It is, however, common usage to organise a set of discourse relations within an integrative network or taxonomy of these relations (cf. van Leeuwen 1991; Bateman 2007). In the following, this organisation will be done with the help of a metafunctional classification which has been successfully applied to a paradigmatic organisation of verbal discourse relations previously (cf. Maier and Hovy 1993; Bateman and Rondhuis 1997). On the basis of the analyses in this book, it is now possible to find at least some generalisations of the relations’ use in an actual context and to describe their various functions more systematically. For instance, the graphical representation of the Segmented Discourse Representation Structure (SDRS) of the extract from The Lives of Others (given in figure 3.4) contains only a few Narration-relations and is mostly constructed by text structuring relations such as Parallel. The discourse structure of the extract from Vanilla Sky, in contrast, is constructed exclusively by Narration-relations which outline the spatiotemporal consequence of the events. Whereas the Parallel-relations in the first example

i i

i

i

i

Film Discourse Interpretation 197

serve to organise the textual structure of the extract, the relations in the latter example serve to generate the film’s ideational content. The interpersonal metafunction, in contrast, may be fulfilled by relations that help to establish social relations between characters in a film or that create a certain emphasis on emotions, for example. Here, we can refer back to the example given in section 4.2.2 above, where the camera zoom in the extract from Am´elie focuses on the emotional reaction of the character. This zoom constructs a series of events that are connected by Narrationrelations. In this case, this type of relation fulfils both an ideational as well as an interpersonal function in order to construct the film’s content. On the basis of these and further examples in the analyses, the following principles for a metafunctional classification can be identified. Ideational Film Discourse Relations Relations which fulfil an ideational function in filmic discourse serve to construct the film’s representational content. The relations therefore express semantic correlations between discourse segments that refer to, for example, spatial and temporal specificities and enrich the recipient’s general knowledge about the film. The ideational function may be fulfilled (and can also be proven by the analyses) by the relations Narration, Elaboration, Background, Explanation and Result. Whereas Narration mainly constructs a spatiotemporal consequence between two different events, Elaboration and Background generally expand the representational content by further information. Explanation and Result, in contrast, serve as cause-effect relationships which explain causal conditions between the segments. Interpersonal Film Discourse Relations Interpersonal relations construct links between events in the film that create social relationships between filmic characters, for example by a causal relationship that explains these connections. Furthermore, they construct a certain distance or closeness to the recipient, for example by focusing on the emotional reaction or facial expression of a character via Elaboration. The function may thus be fulfilled (and can similarly be proven by the analyses) by Narration-, Elaboration- and Explanation-relations. It is also conceivable that cause-effect relationships such as Explanation and Result focus on or give a cause for emotions, etc. It becomes clear, therefore, that these relations can all fulfil both the ideational as well as interpersonal function at the same time. Textual Film Discourse Relations The two text-structuring relations Parallel and Contrast were already demonstrated in detail in the analyses and were also explicitly defined as textual relations in section 2.2.4. They

i i

i

198 Multimodal Narrative Construction in Filmic Text

serve to organise the discourse itself. It could be shown, for example, that the use of these relations gives evidence for the discursive systematic of creating specific cinematic devices such as the cross-cutting montage in The Lives of Others. Nevertheless, these relations are also able to fulfil an ideational or interpersonal function, if they serve, for example, to construct a social relationship between characters or contrast different emotional depictions. A further organisational principle which has been applied to the set of discourse relations for verbal discourse by Asher and Lascarides (2003) and Asher and Vieu (2005) is the distinction between coordinating and subordinating relations, which dominates the discourse’s hierarchical structure. This distinction could have been found equally in filmic discourse, as shown in the analysis of the extract from Vanilla Sky, for instance. A further example for a subordinated discourse structure can be seen in the graphical representation of the discourse structure in Am´elie given in figure 4.4 as a repetition of figure 3.8. In this example, the subordination is

π3 Narration π0

π4

π5 “Dear Mrs. Wells”

“we recently recovered”

Figure 4.4: Graphical representation of the first voice track in the extract from Am´elie governed by a Narration-relation which constructs the Discourse Representation Structure (DRS) labelled π 0 that in turn governs two sub-structures unfolding in parallel: the ongoing story-line with the main character of this scene and a voice track reading the letter (see section 3.3.2). Here,

i i

i

i

i

Film Discourse Interpretation 199

the Narration-relation not only operates as an ideational relation that constructs the representational content, but also as a textual relation that separates the two sub-structures from each other. For verbal discourse, Asher and Lascarides determine that Explanation, Elaboration and Result are subordinating relations in principle (cf. Asher and Lascarides 2003: 146). However, as becomes visible in the different analyses and in the example above, the relations in filmic discourse cannot be distinguished similarly. In figure 4.4, it is a Narration-relation which governs the subordination of the structures unfolding in parallel. In contrast to the definitions by Asher and Lascarides, the hierarchical structure of filmic discourse has to be defined according to the metafunctional diversification of the relations depending on the actual context that has been described above. Due to this process of hierarchisation, the different unfolding discourse structures in the preceding analysis present those structures which have been taken as the preferred ones because of their small number of nodes, the large number of relations and the low number of specifications which could have been found between and for the discourse segments (see section 4.3 above). Their choice has been governed by one of the main principles within the narrative logic, the principle of Maximise Discourse Coherence (MDC), which was outlined in detail in section 2.2.6. According to this principle, it is possible to demonstrate a partial order of the relations on the various discourse structures which then is described as τ : R(α, β) ≺ R(α, β). In the extract from Am´elie depicted above, the Narration-relation is inferred because of the default information for this relation, occasion. It is thus not overwritten by any other more specific relation and the structure presented is therefore the result of the preferred update process. The example, however, is only one of the various examples in the preceding chapter which give evidence for the processes operating during interpretation. With the help of the provided framework, it is possible to retrace these processes which, as has become clear in this chapter and in the preceding analyses, depend in most instances on the actual context of the film and therefore can at least partly be systematised in terms of the functions and uses worked out above. Thus, this systematisation on the paradigmatic axis of description stands in direct reference to the syntagmatic articulation of the relations in the discourse structure. The construction of this structure in turn operates within various layers of the strata of film discourse interpretation described in section 4.2. For a definite taxonomisation of the relations, however, more empirical examination of their use is definitely necessary, but goes beyond the scope of this book and what is currently known.

i i

i

200 Multimodal Narrative Construction in Filmic Text

4.5

Conclusion: Common Sense in Multimodal Film Analysis

The analyses in the preceding chapter have successfully demonstrated that a filmic discourse can be proven coherent when every discourse segment can be related to another segment by the set of film discourse relations. The explicit meaning postulates and default axioms give clear definitions of the relations and contain generally valid truth conditional effects that refer to the preceding semantic interpretation of the text. Therefore, they are able to depict certain circumstances which have to exist within the filmic text and its context to enable the inference of the relations. These circumstances operate as higher layers of the strata of film discourse analysis and govern the bottom-up process of constructing logical forms of the discourse from top-down. By this means, it is possible to give distinct instructions guiding the recipient’s activity in terms of the textual cues immanent to the film. This then constitutes the narrative logic of understanding and constructing filmic discourse (cf. Bordwell 1985; Bordwell et al. 1985; Bateman and Schmidt 2011). As has been pointed out in section 1.3, narrative is generally seen as a basic phenomenon of discourse; it is “essentially textual: it provides an integrating level of organisation that unifies the deployment of filmic resources within shots, across shots and over entire films in the service of broader socio-functional communicative (i.e., textual) goals” (Bateman and Schmidt 2011: 291). The organisation of the various strata in section 4.2 above is a first summary of how these broader socio-functional communicative goals can be included in a formal analysis. Moreover, this perspective allows exactly the constraint of the processes of discourse interpretation which is necessary to obtain reliable information about the construction of meaning in filmic text. With the stratificational view of the different levels of the inference process on the one hand and the description of distinct logics for the interpretation on the other, we obtain a framework which works according to the following principles: (Con-)Textual Interpretation The logic of film discourse interpretation enables the analysis of coherence in filmic text with regard to the multimodal and intersemiotic construction of meaning. The discourse’s content, which is often not explicitly mentioned, can be interpreted with the help of analytical tools from the area of discourse semantics, but without any notion of syntax or grammar. Constraint Interpretation The semantic content of the filmic text can be interpreted due to relatively determinable knowledge sources and clearly definable meaning postulates and default axioms. They identify the set of film discourse relations which in their combination produce the overall structure of the discourse.

i i

i

i

i

Film Discourse Interpretation 201

Narrative Interpretation The method of building the logical form of filmic discourse and displaying its narrative structures enables a systematic analysis of what is more frequently described as the poetics of cinema. It has now been suggested how recipients construct the film’s narrative content on the basis of inferential hypothesis-testing. According to Bateman and Schmidt (2011: 3), it is these principles that “make explicit just what is [...] ‘in’ the filmic material under investigation” and what is “[...] the basis for any further consideration that a viewer (or analyst) can reasonably undertake.” With the help of the framework provided, it is thus not only possible to set out the intuitively obvious of filmic discourse, but also to make obvious the various meanings that enable us to understand film as asked for in the quotation from Bordwell (2011) above (see page 169).

i i

i

5 Conclusions and Outlook “Texts are the human way to reduce the world to a manageable format, open to an intersubjective interpretive discourse.” (Eco 1990: 21)

The aim of this book has been to provide a method for the textual analysis of filmic meaning in terms of a logical framework which combines common functional approaches to the examination of multimodal texts with more formal accounts of discourse interpretation. On the basis of the theoretical foundation in linguistics and in particular discourse semantics, which has been described in section 2.2.2, the logic of film discourse interpretation was methodologically formulated in chapter 2 and successfully applied to several film extracts in chapter 3. It was possible to build logical forms of the filmic discourses that give evidence for the overall coherence and structure of the films. Finally, in chapter 4, these logical forms were considered with regard to their functional role within the context in order to outline the interpretation process more completely. In the following, the presented theory as well as the analytical results and their metafunctional diversification will be summarised with regard to the overall research motivation and the contributions this book makes to film analysis. The quotation cited above thereby serves as a compendium for the book’s main objective: the examination of film’s textual complexity and interpretive power, which can now be analysed in more detail. In section 5.1, the method for building logical forms of filmic discourse will again be presented and shortly summarised. The limits and boundaries of this framework which were shown during the analysis will be described in section 5.2 in order to open up new perspectives for further research. In section 5.3, this basis will be taken as a first glance towards interpretive explorations which should now be possible with the help of the results provided in this book. Finally, section 5.4 will give concluding remarks on how the book contributes to a comprehensive discipline of film analysis.

5.1

A Method for Constructing Logical Forms of Filmic Discourse

The framework of the logic of film discourse interpretation provides a method for the comprehensive analysis of filmic text in terms of its intersemiotic meaning construction on the one hand and its structural unfolding in time and space on the other. It addresses two general problems of filmic meaning-making which have also been explored by other multimodal

202 i i

i

i

i

Film Discourse Interpretation 203

researchers: (1) the question of how meaning is made cross-modally, i.e., through intersemiosis, and (2) the question of narrative comprehension in film (cf., e.g., O’Halloran 2004; Tseng and Bateman 2012; Tseng 2013; Wildgen 2013). This book contributes to this research in that it provides a tool for the analysis of how the various modalities in film work together. With the method of building logical forms of discourse segments discussed in detail in the previous chapters, it is possible to outline each particular contribution the resources make to the interpretation of the film. Their functional consideration then produces generalised patterns of how meaning construction in film takes place. Patterns operating cross-modally, in particular, represent specifics of film and give evidence for its particular organisation in comparison to other multimodal texts. Furthermore, the set of film discourse relations provided within the framework enables both the extensive description of the unfolding discourse as well as a film’s narrative configurations by means of common techniques of continuity editing. It is thus possible to outline how coherence as the dominating textual quality influences the recipient’s interpretation process and at the same time produces narrative discourse structures which depict spatiotemporal as well as causal circumstances of the film’s story. The most important research motivation, however, which has indeed accompanied this book from its beginning, is the question of how this multimodal meaning-making in filmic text can be made more reliable and to a lesser extent attributed simply to the work of the recipient. It is today commonly accepted that the interplay of the various semiotic resources in film does not make its propositional content fully explicit and that abductive and defeasible reasoning are needed to infer a film’s subject matter. However, although the notion of inference is a widely considered issue in film analysis, there are few approaches that are able to describe exactly how the recipient constructs meaning out of the textual clues and with the help of abductive reasoning. The aim of this book therefore was to provide a framework which makes it possible to depict this interpretation process from a formal perspective and with regard to a concrete logical foundation. To do this, the book presents logical formulae which give clear specifications of the conditions that must hold for film discourse relations to be found and which make it possible to constrain the inference process in order to find the best and preferred interpretation within, and for, any actual filmic context. It is thus possible to constrain the immense number of potential interpretations in order to elucidate those elements or segments which play a significant role within any particular filmic situation. Whereas previous research has often concentrated on the functional analysis of how meaning is constructed due to social and cultural processes, this book now makes available patterns for a more constrained description

i i

i

204 Conclusions and Outlook

of narrative structures which, as a consequence, offers valuable clues to further analysis of filmic texts and their comprehension. Thus, in short, the combination of multimodal and formal analysis makes explicit the textual clues and hints Bordwell and his colleagues have discussed, but have not succeeded in relating formally to required inference processes. For this reason, the analysis described in this book may be taken as a basis for further empirical investigations providing hermeneutic and sophisticated examination of the various possibilities to create meaning in film and the coexistent power of its textuality. Thereby, the book not only offers a theoretical formulation of the framework and first practical applications in order to afford more empirical work, but also strengthens further analytical and interpretative examinations which are still required in contemporary film analysis (cf. Bateman and Schmidt 2011: 3).

5.2

Towards New Perspectives of Analysis

The example analyses in chapter 3 have shown how the framework proposed in this book makes possible the examination of textual qualities of filmic discourse and their influence on the interpretation of film. Taking the analytical tools provided for verbal discourse as a basis, it is now possible to outline filmic characteristics similar to those of language and therefore to demonstrate the striking closeness between film and text. However, the analyses also showed distinctive features of filmic discourse which cannot be compared to phenomena in verbal texts. In addition to modifications of the description of the logical forms of filmic discourse and the definitions for inferring rhetorical relations between them, the framework had to include a more complex view of the resources which take part in the meaning construction of film. Although verbal text is also always characterised by the additional interplay of different semiotic resources such as volume and intensity of the voice in spoken discourse or punctuation and layout of written language, film as a multimodal text is exceptionally affected by intersemiotic meaning-making strategies that all have to be taken into considertation during the interpretation process. This has considerable consequences for the detailed description of this process, for example with regard to the systematic specification of those discourse segments that play a central role for the interpretation. It was not possible, for instance, to point out how the various images and signs as impressions of the protagonist in the extract from Vanilla Sky influence meaning construction. Neither could it be described which role exactly the sounds and noises in the extract from Am´elie play for the interpretation of the scene. All these phenomena count as new information to the preceding discourse and therefore are taking part in the meaning construction somehow. But it was not possible to demonstrate how these filmic de-

i i

i

i

i

Film Discourse Interpretation 205

vices could be analysed as attachment sites within the unfolding discourse structure. In discourse semantics, attachment sites are generally defined as those “discourse referents in the discourse context to which new information can attach with a rhetorical relation” (Asher and Lascarides 2003: 148). These are the referents which, in verbal discourse, are constrained by rules such as the right frontier constraint (e.g., Polanyi 1985; Webber 1991; Asher 2008), which has also been described in detail by Asher and Lascarides (2003: 10ff.). This constraint indicates that “only constituents at accessible nodes enjoy the formal status of currently being constructed” (Polanyi 1985: § 4.1). These accessible nodes are on the right frontier of the discourse structure, as for example visualised in the typical tree-structure a discourse has according to Polanyi and Webber in figure 5.1; the accessible nodes on the right frontier of the example discourse structure are shaded. It becomes visible in this graphic representation that the other nodes of this structure are not open for new information to be attached to the discourse.

Figure 5.1: The Right Frontier Constraint (adapted from Webber 1991: 10) For verbal discourse, Asher and Lascarides have pointed out that this new attached information depends as well on the relation it holds to the preceding context. This was also applied to filmic text in the previous chapters. The discourse structures which were presented in the analyses in chapter 3 are those structures that are seen as maximally coherent because of the best and preferred interpretation they give. During the process of updateSDRT , it was possible to determine the basic units of the film that contribute to meaning construction by describing them within the logical forms of the discourse segments. As a second step, these logical forms could be combined with the new information attached to the discourse. However, this new information was again formulated in terms of logical forms and therefore as an abstract summary and a specific choice from all

i i

i

206 Conclusions and Outlook

possible attachment points of the discourse. As indicated in conjunction with the analysis of the sound track in section 3.1.2 (see footnote 4 on page 92), the interpretation process of choosing and summarising this choice could not be described very systematically, but only with regard to the overall interpretation. The new constituent which is attached to the preceding one is thus only a result of further reasoning which, in the present examples, could not be further elucidated. This opens up new questions of analysis which should be addressed in further research on filmic discourse. It would be interesting, for example, to ask how the available attachment sites within a multimodal discourse interact and whether those sites which are not chosen for the updated Segmented Film Discourse Representation Structures (SFDRS) nevertheless influence the interpretation. Since the topic of attaching new information to discourse has already been discussed in detail for verbal discourse (cf. Polanyi 1985; Webber 1991; Asher and Vieu 2005; Asher 2008; Prevot and Vieu 2008; Afantenos, Denis, Muller and Danlos 2010), it would be useful and logical to consider an application of this research to the analysis of film. It could then be examined in detail whether the right frontier constraint holds similarly for filmic discourse or how the ‘Rule of Relevance’ described by Asher (1993: 270) may account for the comprehension of multimodal narrative text as well. The reasoning process concerning the best attachment site might then be described more accurately. This shows great promise for further research in the realm of film discourse analysis and also with regard to common-sense film theory. A further point of interest which has come to the fore during the analyses involves open questions concerning the description of the narrative structure of verbal and filmic discourse. As already pointed out in the analysis of the extract from Requiem for a Dream (see section 3.4.2), film often features multiple diegetic levels that operate simultaneously. The SFDRS in the analyses show this interaction of various levels in terms of their complex unfolding. Therefore, the traditional description of these structures had to be extended by further sub-structures that unfold in parallel. The results, similarly given in the analysis of Am´elie, for example, differ from depictions of verbal structures in their complexity. They make it possible to describe specific patterns in the narrative structure of filmic text such as the use of music for structuring and manipulating narration or montage techniques like the cross-cutting in The Lives of Others. The textual analysis provided in this book may then serve as a basis for further questions concerning the interaction of narration strategies in verbal and filmic text (cf. Bordwell 1985; Bordwell 2006; Kuhn 2011) or the description of new forms of filmic narration (cf. Kuhn 2013). At first, this needs considerably more empirical research on the basis of the framework provided here in order to be able to compare and further systemise the various possibilities of filmic structural unfolding.

i i

i

i

i

Film Discourse Interpretation 207

With regard to this empirical research, it would be interesting to draw parallels between the analysis of verbal text and the examination of filmic discourse, for example in terms of corpus-analytical examinations of a larger number of film extracts. In discourse semantics, corpora of verbal texts are for instance annotated with discourse relations in order to get a better overview of the various structures that evolve in those discourses (cf. Stede 2004; Ho-Dac, Fabre, Pery-Woodley and Rebeyrolle 2009). It is conceivable to pursue a similar analysis for filmic discourse. In a further step, the comparative consideration of both the analytical results of this book and those of further research could then serve as a stronger basis for interpretive work. Some possibilities for this interpretive work will be exemplarily suggested in the section following.

5.3

Towards Bridging the Gaps in Film Theory

As pointed out by the quotation by Eco (1990; see page 202) and as has become evident throughout the discussions in the previous chapters, filmic texts provide a multiplicity of interpretations which can be found and inferred during the reception of the dynamically unfolding discourse. This book has explained how these interpretations and the meaning potential are normally inferred by the recipient due to explicit textual information that guides and constrains the recipient’s activity. Furthermore, it cannot only be asked how this interpretation takes place, but also which interpretive possibilities are available from the inherent, significant patterns of the text. These patterns may shed light on interpretations which go beyond the pure semantic meaning of the text and rather contribute to questions of approaches in literary analysis, cultural studies as well as cognitive film theory, for example. For a short outlook on this kind of examination, the extract from The Lives of Others, which already served as an introductory example in chapter 1, will be taken into consideration one final time. Whereas the analysis in section 3.1.2 mainly focused on the structural unfolding of the discourse and the filmic specificity of the cross-cutting montage, the description in section 1.1 already mentioned first suggestions concerning the overall function of this scene as an important turning point in the story-line. This can now be related to the analytical results of this book more broadly. The scene from The Lives of Others has in fact often been described as one of the key scenes in the film, for the first time displaying the protagonist’s emotions (cf., e.g., Falck 2006; Watkins 2007). For the unfolding of the film’s story, this reaction has important consequences: “So thoroughly swept away is Wiesler that this exemplary Stasi officer starts fabricating his reports in order to protect those who have unknowingly brought him to life” (Watkins 2007). The protagonist transforms himself into a more sympathetic and compassionate person, suddenly interested in

i i

i

208 Conclusions and Outlook

the fate of the persons he is observing. He is thus becoming a ‘good man’, as indicated by the title of the piano score. Watkins describes this transformation as clearly indicated by the filmic text: “In the scene after Wiesler listens to Dreyman playing the Sonata for a Good Man, he eavesdrops on a conversation between Georg and one of his friends about music: ‘Do you know what Lenin said about Beethoven’s Appassionata? “If I keep on listening to this I won’t be able to finish the revolution.” Can anyone who heard this music — who really heard it — be a bad man?’ Wiesler, of course, has really heard music for the first time—he’s heard the emotion that it encapsulates rather than a string of notes. This line reinforces for us the transformation that is taking place in Wiesler’s character.” (Watkins 2007) The analysis in section 3.1.2 verifies this textual reference in that it outlines the causal relationship between the events of this scene. There are two clear emphases on apparently important aspects in the visual depiction, which are put into effect simply by the camera work: the score sheet of the Sonata of a Good Man and the protagonist’s emotional reaction. This textual composition can be explained by cause-effect relationships which hold between the entities (see for example the graphical representation of the structure in figure 3.4). In contrast to the preceding parallelisms between the two protagonists, mediated by the cross-cutting montage, the event of crying is highlighted by another discourse relation, which interrupts the sequence of Parallel-relations and the alternation between the two settings and zoom-ins on the protagonist’s facial expression. The focus remains on the second setting and the recipient’s attention is clearly directed to the small detail of the tear in this scene, which can be interpreted as a causal reaction on the basis of the preceding context and the so far given characterisation of the protagonist. In addition to the simple semantic potential, which is conveyed by the filmic discourse, the analysis includes as well pragmatic information needed for the successful inference of the Result-relation. The interpretation can only be effective if the piano playing and the fact that the Stasi captain is listening to this playing are interpreted as a cause for the reaction following. Thus, in this scene, we not only have results concerning the multimodal and intersemiotic construction of meaning, but we are also able to detect explicitly used patterns of guiding the recipient to a potentially adequate interpretation of the scene. Furthermore, the textual focus on the emotional reaction is a pattern ¨ of identification, which has for example been pointed out by Suckfull (1997) in the realm of cognitive film theory. For a psychological-cognitive ¨ examination of the film’s impact on the recipient, as pursued by Suckfull,

i i

i

i

i

Film Discourse Interpretation 209

this identification may be of further interest. It refers to the assumption that there is a close interrelation between film style and cognitive effects, which is a central issue in the area of film psychology (cf. Wuss 1993). The focus on the emotional reaction is exactly such a pattern of identification, which can indeed be analysed within the discursive context because of ¨ highlights the textual structure. In her work on film experience, Suckfull that a reduction to isolated stimuli within a piece of art is at the same time an objective and a challenge for empirical research on film. Therefore, an application to clearly defined patterns such as these in the example described above must be aimed for in order to be able to combine the analysis of effects with the overall context of the diegesis. This aim can now be supported by the textual analysis provided in this book in that textual clues are made explicit and, in a further step, can be related to interpretations beyond the semantic level of description, such as, for example, the appraisal of the film in terms of its often attributed ethical and moral ambitions. With the potential to find a textual basis for this research, the framework of this book thus makes another significant step towards bridging the gap still existing between semantic analysis and higher-order interpretations of filmic text. The provided analytical tool supports a systematically motivated analysis of the film’s semantic potential, which, in a further step, opens up new perspectives on text interpretation in general and provides further possibilities for empirical research. These perspectives may for example be oriented towards social and cultural aspects, as in approaches to narratology, poststructuralism and postcolonialism or to feminist and transcultural analysis, as well as patterns of cognitive and psychological research, which are generally summarised as accounts in cognitive film theory (cf. Bordwell 2009).1 These approaches take the notion of film as text as a basis, but do not ask for the specific textual qualities of the filmic text. We assume that their “very general, abstract, socially and individually revealing interpretations of literary (which now includes filmic) works” strongly need the analysis of technical details of the film’s textual composition so as to become even more valuable and relevant (Bateman and Schmidt 2011: 23; cf. Bordwell 2004). Similarly, this book can be seen as addressing the problem of the socalled semantic gap in the interpretation of filmic and other texts in Artificial Intelligence research (cf. Dorai and Venkatesh 2003). In these fields, researchers also explicitly underline the need for robust media semantics analysis “in the broadest sense” (Dorai and Venkatesh 2003: 15), in particular with regard to computational instantiations of analytical schemes. In order to pursue, for example, automatic analyses of the properties 1 A comprehensive overview of the various approaches to study in film in the last 50 years, their mutual interference and a more detailed differentiation of these accounts is for example given by Bateman and Schmidt (2011: 23).

i i

i

210 Conclusions and Outlook

of constituency and structurality, this book provides at least the needed formalisms as a basis for more empirical examinations in the realm of computational media aesthetics, as described by Dorai and Venkatesh. This approach is intended to “build innovative, high-level, semantics-based content description tools for reliable media location, access, and navigation services” (Dorai and Venkatesh 2003: 15). A textual analysis of filmic semantic representations by means of their functional embedding in social contexts delivers a first fully formalised attempt at bridging the gap between descriptions of the technical devices on a lower level of analysis and broader interpretations of societal patterns of the film on a higher level. Further research in automatic analysis on the basis of linguistic tools will help to broaden film analysis in these fields and to follow up the interest in corpus-based results that help to generalise different processes of text generation and reception.

5.4

Moving on to a Comprehensive Film Theory

In the realm of film analysis from a contemporary perspective taken by this book and the ranks of similar approaches, for example as described in section 1.3, one of the general aims is the already mentioned transgression of still-existing disciplinary boundaries in order to work together for a better understanding of film. This book contributes to this aim in that it provides a framework for the combined examination of intersemiotic meaning-making in multimodal filmic text and the reconstruction of the recipient’s inference process of interpreting this meaning. By means of the set of concrete rules for inferring film discourse relations and constructing the structure of any filmic extract, the recipient’s mental activity of executing story-constructing activities according to logical patterns of understanding can be reflected. The logic of film discourse interpretation, generally understood as a narrative logic, outlines the textual principles of film according to the textual principles of verbal discourses. It will thus be possible to empirically examine what is still required within film analysis: i.e., the description of which filmic resources generate which textual structures and how they guide the recipient’s understanding (cf. Bateman 2013). This book therefore addresses one of the main questions that has been asked within film theory over the course of time: How do we understand films? (cf. Bateman and Schmidt 2011: 1; Bordwell 2011) This question refers to the metalevel and—as has been shown in this book—at the same time basic level of interpretation and explanation that David Bordwell pursues as commonsense film theory (see the beginning of chapter 4 and section 4.5). It is this understanding that

i i

i

i

i

Film Discourse Interpretation 211

“may in turn oblige us to accept empirical experiment, evolutionary thinking, and neurological research—all of which most literary humanists find worrisome.” (Bordwell 2011) The present book accordingly offers a framework within which we can correlate the various perspectives on film analysis and interpretation in order to pursue comprehensive approaches to the theoretical and empirical issues involved. How this book contributes to this basic idea of analysing filmic meaning, as well as the basic assumptions and research results it provides for more empirical analysis, have now been pointed out in detail in the preceding sections. The textual patterns that could be worked out within the analyses of example filmic extracts are those patterns that guide our understanding of how films mean, and it is these that make it possible to ask further interpretive questions such as those addressed by Bordwell in his recent essay. These patterns are seen as the basis for every analytical and interpretive examination, be it of cognitive, neurological nature, social and cultural interest or literary research. To conclude, this book provides an integrative level of description and organisation of the essentially textual devices that help to shed light on comprehension and interpretation of film. In detail, it makes it possible to describe the different levels of meaning-making across different modes and strata of the filmic text by elucidating the fundamental categories of coherence and structure involved. It enables the analysis of filmic narrative construction by taking the dynamic activities of the recipient into consideration and giving precise constraints on the inference processes made explicit with the logical framework. It is thus a new and powerful formal tool for a fine-grained and systematic examination of how films mean.

i i

i

Appendix A Transcriptions

This appendix contains detailed transcriptions of the film extracts which have been analysed in chapter 3. Parts of these transcriptions have already been given throughout the preceding chapters but, for a comprehensive overview of the extracts, the scenes are described in detail in the following. The transcriptions contain the shots (and sometimes also some particular frames within one shot), a description of the shot as well as descriptions of the audio track, the spoken language and the music, if relevant.

212 i i

i

i /

sound of pressing the button

sound of batting staves

sound of children at play

title “WORDS” on black background

close-up of recorder buttons, “play” is pressed

a theatre stage, two persons playing in the front

two girls in a playground outside

2.

3.

4.

1.

shot

audio track

#

shot description

WORDS: Short film directed by Daniel Mercadante and Will Hoffman. Available online: http://vimeo.com/13768695.

A.1 WORDS (Everynone, 2010)

/

/

/

/

spoken language

calm piano and guitar music

calm piano and guitar music

calm music starts immediately after the play-button is pressed

/

music

i

213

i

i

i /

/

/

trumpet play

sound of blowing heavily

a football team planning a new play

close-up of a board in the meeting situation

a man with helmet and face mask

a man sounding the trumpet

close-up of the man’s mouth blowing into the trumpet

5.

6.

7.

8.

9.

/

/

“Play ball!”

“Here, the end is pinching. . . ”

“Here, the back is coming off. . . ”

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

214

i

i

i sound of blowing out

sound of ventilator and the flying paper

noises from outside (cars)

sound of heavy wind

sound of blowing one’s nose

a man blowing out candles on a cake

a ventilator blowing away some papers

a bank note on the street

trees blown by the wind

a man blowing his nose

10.

11.

12.

13.

14.

/

/

/

/

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

215

i

i

i /

sound of bubbling water

clicking

the sound of a hair blower

clicking

a blow fish

a bulb with a blowout

a woman drying her hair until the blower stops

close-up of the hair blower

16.

17.

18.

19.

/

/

/

/

sound of a blow against the headgear

2 men boxing

15.

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

216

i

i

i /

“Go, break!”

in the background: shouting

shouting of the team members

sound of squeaking brakes

an x-ray photograph showing a broken bone

a Lacrosse team

a foot applying the brakes

21.

22.

23.

24.

/

/

sound of breaking shelves

a young karate destroying wooden shelves

/

sound of breaking porcelain

a plate falling down, breaking into pieces

20.

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

217

i

i

i sound of billiard balls clashing together

/

/

sound of batting

sound of splitting the tree trunk

a pool table and a man doing the opening stroke

a man shouting into his cellular phone

a man and a woman sitting on a bed, heads down

a man hitting a TV with a bat

a man choping wood

25.

26.

27.

28.

29.

/

/

/

“You’re breaking up, man, I can’t hear you!”

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

218

i

i

i sound of sliding on the ground

/

smacking

/

sound of splitting

a young girl doing the splits

a banana split ice cream on a table

2 men eating the ice cream with spoons

2 bowling pins (split leave)

splitting trousers

30.

31.

32.

33.

34.

/

/

/

/

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

219

i

i

i sound of shopping cart

steps on the ground

/

/

sound of dropping water

a run in a pair of tights (shopping cart in the background)

a man running (close-up of legs)

close-up of running nose

a drop of paint running down the wall

water coming out of the water-tap

35.

36.

37.

38.

39.

/

/

/

/

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

220

i

i

i “I’m serious, I’m gonna run away!”

(some commands from the cockpit)

/

/

voices / noises outside

cockpit sound and sound of flying airplane

sound of bird in the sky

humming and noises from outside

a boy talking, then running away

a runway (top view)

an airplane in the sky, later birds flying in the sky

a fly on the ground

41.

42.

43.

44.

“I’m gonna run away!”

sound of car

a car running a red light

40.

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

221

i

i

i laughter

laughter

strange piano sound

sound of wind

sound of porcelain

a zip fly (close-up)

board in school class with a drawing of the middle finger, a teacher coming in a piano flying down

a sky diver in the sky

a vase falling down

45.

46.

47.

48.

49.

/

/

/

“That is not good to fly!”

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

222

i

i

i “I’m falling and I. . . ”

/

/

sound of waterfall outside

/

a TV in a sleeping room showing an old woman lying on the floor, two persons on a bed

a waterfall

a girl falling asleep in the arms of a boy

52.

53.

54.

/

/

sound of thunking

a man falling down the stairs

51.

/

/

a boy diving off high board

50.

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

i

223

i

i

i /

/

/

a feather in the air

two men at weight lifting

shining sun

58.

59.

sound of leaves

a falling tree

56.

57.

sound of birds in the background

autumn coloured leaves (top view)

55.

calm piano and guitar music

“Aaah. . . ” — “Light as a feather. . . light. . . ”

calm piano and guitar music

calm piano and guitar music

“Light as a feather, come on!”

/

calm piano and guitar music

calm piano and guitar music

/

/

i

224

i

i

i /

/

buzzing of the lights

a priest in a church

a small night light in a socket

stadium lights, later turned off

63.

64.

sound of the lighter

a man lighting a cigarette

61.

62.

laughing

a girl playing with a flashlight

60.

/

calm piano and guitar music

calm piano and guitar music

calm piano and guitar music

“Let there be light!”

/

calm piano and guitar music

calm piano and guitar music

/

/

i

225

i

i

i sound of thunderbolt

hollow noise

sound of keyboard

/

lightning illuminating a tree

stars / space, later a falling star

close-up of keyboard, space bar is pressed

blackscreen

65.

66.

67.

68.

/

/

/

/

silence

silence

silence

calm piano and guitar music, then the music stops

i

226

i

i

i audio track

telephone ringing

telephone ringing

telephone ringing, clicking

telephone ringing

shot description Dreyman’s flat: a telephone on a dresser attic storey: monitoring centre: the Stasi captain with headset attic storey: switch of the monitoring function

attic storey: Stasi captain takes the receiver

1.

2.

3.

4.

#

shot

Transcription from the midst of the film: 00:49:29–00:52:06.

A.2 The Lives of Others (Henckel von Donnersmarck, 2006)

/

/

/

/

spoken language

orchestral music fading out

orchestral music fading out

orchestral music in the background

orchestral music in the background

music

i

227

i

i

i telephone ringing in the beginning

/

/

/

/

Dreyman’s flat: Dreyman appears, answers the telephone attic storey: Stasi captain listens to the telephone conversation Dreyman’s flat: Dreyman listens to the caller

attic storey: Stasi captain listens to the conversation

Dreyman’s flat: Dreyman hangs up slowly

5.

6.

7.

8.

9.

/

“I’m going to hang up, okay?”

“Georg?”

“What’s up?” — “Georg, it’s about Jerska. He hanged himself last night.”

“Yes?” — “Georg, Wallner here.”

/

/

/

/

orchestral music stopped

i

228

i

i

i 13.

/

/

/

/

/

/

11.

12.

/

close-up of the score sheet “Sonata for a Good Man”

10.

medium closeup of Dreyman’s hands, playing the piano, camera panning to a close-up of Dreyman attic storey: Stasi captain listens to the piano playing via headset (shot from behind)

creaking floorboards

Dreyman sits down at the piano, his girlfriend appears in the background, he grasps for score sheets

piano playing (offsound)

piano playing (onsound)

piano playing (offsound)

piano playing (offsound)

i

229

i

i

i /

/

Stasi captain listening, a teardrop runs over his face (front view)

Dreyman’s flat: Dreyman completes the piano playing, his girlfriend behind him

14.

15.

piano playing (offsound)

piano playing (onsound) — silence

/

“You know what Lenin said about Beethoven’s Appassionata: — “If I keep listening to it, I won’t finish the revolution.” — Can anyone who has heard this music, I mean truly heard it, really be a bad person?

i

230

i

i

i music

/

/

blowing wind — noise of the street (cars)

blowing wind — noise of the street (cars)

black screen

shot from above of streets in a city

1.

2.

shot

audio track

#

shot description

Transcription of the beginning sequence: 00:00:00–00:03:57. The opening sequence of the movie contains a montage of alternating shots which succeed rapidly and which are nearly not identifiable. Therefore, single frames have been taken for the detailed transcription. They are put together in the figures at the end of this section to outline every single bit of information given on the visual level. The montage is accompanied by rhythmic music produced by drums and a sitar. During the alternation of shots, the rhythm gets faster and the sound intensity grows synchronously to the increasing number of images. A 360-degree tracking shot from a low camera angle closes this scene. The following three shots are then transcribed regularly. Within the transcription, there are three stills taken from a single long shot. Images 16–18 display what is happening in this shot.

A.3 Vanilla Sky (Crowe, 2001)

i

231

i

i

i blowing wind

blowing wind

high angle shot of a park in a city

black screen

6.

7.

/

/

/

blowing wind — background noises from the streets

black screen

5.

/

blowing wind

shot from above of buildings and cars

4.

/

blowing wind

black screen

3.

i

232

i

i

i /

/

blowing wind — chirping

blowing wind — indistinct echo

indistinct echo — noise of the streets

medium long shot of trees and a lake

black screen

medium long shot of a building on a street

10.

11.

12.

/

/

blowing wind

black screen

9.

/

blowing wind — peal of bells

medium long shot of the park

8.

i

233

i

i

i dominant music, sound track

dominant music, sound track: Everything Is in Its Right Place by Radiohead

female voice, Spanish accent: “Open your eyes!” “Open your eyes! Open your eyes!’ Open your —” (stopped)

a television in a room showing a black-andwhite film (Sabrina (Wilder, 1954)) pan to a man lying in a bed, turning off the alarm

16.

17.

music begins

female voice, Spanish accent: “Open your eyes!”

black screen

15.

/

female voice: “Abre los ojos!”

medium shot of the same building, zooming in

14.

/

indistinct echo

black screen

13.

i

234

i

i

i background noises synchronous to the events

background noises

background noises

background noises

background noises

the man gets up and turns off the TV, walks through the room man in bathroom looks into the mirror

reflection of the man in the mirror

close-up of pair of tweezers in a box

close-up of the man’s reflection in the mirror, pulling out a grey hair

18.

19.

20.

21.

22.

“Everything is. . . ”

“. . . everything. . . ”

“. . . everything. . . ”

“Everything. . . ”

sound track: “Everything, everything, everything. . . ”

i

235

i

i

i /

background noises

/

/

reflection of the man in another mirror in the living room close-up of papers and watch on a chair

man next to a window, putting on his shirt medium long shot from behind on the man next to his bed

23.

24.

25.

26.

“. . . in its right place.”

“. . . place. . . ”

“. . . place, in its right. . . ”

“. . . in its right place, in its right. . . ”

i

236

i

i

i roar of car engine

roar of car engine

zooming in on the driver: the man in the car

POV-shot of the street (to the right)

30.

music: roar of car engine

a black car coming out of the garage

28.

29.

background noises

man in the living room packing things together

27.

/

/

/

sound track in the background: “Yesterday I woke up sucking a lemon. . . ”, fading out

i

237

i

i

i roar of car engine

roar of car engine

roar of car engine

roar of car engine

roar of car engine

POV-shot of the street (to the left)

close-up of man in the car

the car on the street

close-up of man in the car, driving

POV-shot of the empty street and the traffic lights

31.

32.

33.

34.

35.

/

/

/

/

/

i

238

i

i

i roar of car engine

roar of car engine — ticking

roar of car engine

roar of car engine

music: roar of car engine

medium close-up of the man from the co-driver’s seat perspective

close-up, POV-shot of the watch

close-up of man in the car, irritated look

POV-shot of the empty street (to the right)

close-up of man, looking to his right

36.

37.

38.

39.

40.

/

/

/

/

/

i

239

i

i

i roar of car engine

/

/

/

/

POV-shot of the street

close-up of the man, looking to his left

POV-shot of the empty street (to the left)

POV-shot, close-up of the traffic lights

close-up of the man, looking above

41.

42.

43.

44.

45.

gentle sitar playing

gentle sitar playing

gentle sitar playing

gentle sitar playing

/

i

240

i

i

i /

/

/

medium close-up: the man gets off the car and looks around long shot from above: the man starts running

medium close-up: the man runs

47.

48.

49.

The stills following are single frames:

background noises

long shot of the street where the car is approaching

46.

sitar playing — rhythmic music (drums)

sitar playing — rhythmic music (drums)

sitar playing

sitar playing (louder)

i

241

i

i

i

242 i i

i

i

i

243 i i

i breathing — female voice, no accent: “Open your eyes!” female voice: “Open your eyes! David. . . open. . . ”

close-up of the man from above, lying on the bed where he wakes up medium long shot of the man, sitting on the bed

50.

51.

The scene ends with the two shots following:

/

/

i

244

i

i

i rustling — steps

corridor, woman reads the letter while slowly moving

3.

“How’s it going, Mrs. Wells?” — “It’s always better when it’s not raining.”

background noise of the block, some voices

[1st voice-over:] Dear Mrs. Wells, we recently recovered a mailbag that was lost in a plane crash on October 12th 1969 on Mont Blanc.

“This is for you.” — “Me?”

spoken language

audio track

background noise of the block

shot description corridor in a block of flats, a woman dusts the wall while a postman delivers letters

2.

shot

corridor, the postman hands over a letter to the woman

1.

#

Transcription of a sequence in the second third of the film: 01:22:50–01:24:17. Since the whole sequence is composed of only three shots, the transcription gives a few more stills to make visible the events happening in the first and third shot and the zoom which is given in the second shot. The lines within the table mark the cut between the three shots. The different voice-overs on the auditory level are marked by different font types.

A.4 Am´elie (Jeunet, 2001)

i

245

i

i

i woman sits on a sofa in a living room and reads the letter (zoomed in)

6.

7.

5.

trumpet sound (fanfare) — drumming — rush of water — bellringing chirping birds — drumming — chirping of birds — baby crying — rush of water — chirping — orchestra

mumbling

corridor, woman reads the letter while slowly entering her flat

4.

woman sits on a sofa in a living room and reads the letter

rustling — steps

corridor, woman reads the letter while entering her flat

I turned down that woman’s money. If all goes well, I’ll soon be able to afford a house.

[2nd voice-over:] Darling Mado, I’m in exile. I can’t sleep, can’t eat. I think of you endlessly. I know I’ve made the biggest mistake of my life.

Jacques Grosjean, Customer Services

We are forwarding the enclosed letter to your address. Please accept our apologies for the uncustomary delay.

i

246

i

i

i music: piano playing (theme)

music: piano playing (theme)

woman polishes the photo

11.

/

/

. . . one orange-coloured day. Your ever-loving Adrien.

chime of a bell — chirping — ticking

woman kisses a framed photo of a man

I dream of better times ahead when you’ll forgive me and join me here. . .

drumming — bellringing — chirping

10.

9.

8.

woman sits on a sofa in a living room and reads the letter (closeup) woman sits on a sofa in a living room and reads the letter (closeup)

i

247

i

i

i “Ooooaaaaahhh. . . ”

“Juice, juice, juice. . . Juice by you, juice by you.”

applause, background noise of the TV show TV replay with inserts

3.

applause, background noise of the TV show

TV replay with inserts

spoken language “Juice for Tappy, juice for Tappy, juice for Tappy. Ooaah! Juice for Tappy!”

2.

audio track

shot description

applause, background noise of the TV show

shot

credits

1.

#

/

/

/

music

Transcription of the beginning: 00:00:12–00:04:55. The first twelve seconds show animated logos of the film’s production companies Artisan Entertainment and Thousand Words, gentle music is in the background. The logos are not relevant for the unfolding story which immediately starts with the first shot shown in the transcription following. The corresponding sound fades in slowly while the background music stops.

A.5 Requiem for a Dream (Aronofsky, 2000)

i

248

i

i

i /

/

/

“We got a winner.” — “We got a winner.” — “I said we got a winner.” “We got a winner.” — “We got a winner.” — “We got a winner.” “Our first winner is a flight attendant from Washington, D.C. Will you please. . . ” “. . . welcome Mary Kellington!”

applause, background noise of the TV show

applause, background noise of the TV show

applause, background noise of the TV show

applause, background noise of the TV show

TV replay, the show master

TV replay with inserts

credits

TV replay, the show master

5.

6.

7.

8.

/

/

“Thank you, thank you, thank you.”

applause, background noise of the TV show

credits

4.

i

249

i

i

i applause, background noise of the TV show

applause, background noise of the TV show

sound of pulling out the connector

steps on the ground, sound of closing the door

TV replay, a candidate of the show, subtitle with her name TV replay, the candidate and the show master, subtitle with her name

a connector pulled out of a socket an older female and a younger male person in a living room, talking about the TV

9.

10.

11.

12.

“Harold please, not again the TV.” — “Aw Ma. Come on, Ma.”

“Oooaaaaahh. . . ” (stopped abruptly)

“Juice by Mary, juice by Mary! Ooaahh. . . ”

“Juice by Mary, juice by Mary!”

/

/

/

i

250

i

i

i chattering

steps on the ground, at the end: ripping noise

/

/

split screen (caused by a wipe): mother in front of the door, son in living room in front of the door split screen: POV shot through keyhole, son acting with the TV in the living room split screen: mother watching her son, a close-up of a chain split screen: mother watching her son, son talking to her

13.

14.

15.

16.

very gentle music in the background

very gentle music in the background

“What are you trying to do? You trying to get me to break my own mother’s set?” “Or break the radiator?’

very gentle music in the background

very gentle music in the background (long, monotonic strokes of the violin bow)

/

“Why do you have to make such a big deal out of this, Ma? You know you’ll get the set back in a couple of hours. Why ya gotta make me feel so guilty?”

i

251

i

i

i /

/

/

/

split screen: closeup of mother taking the key off her neck, son talking to her in front of the door split screen: POV shot through keyhole, close-up of key on the ground split screen: closeup of mother, son talking to her, taking the key from the ground split screen: closeup of mother, door handle

17.

18.

19.

20.

very gentle music in the background

very gentle music in the background

“Harold, I wouldn’t do that. The chain isn’t for you. It’s for the robbers.”

very gentle music in the background

very gentle music in the background

“Why do you always gotta play games with my head?”

“ls that what you’re trying to do? Your own son?”

“And maybe blow up the whole house, Ma. Your own son, your own flesh and blood, Ma?”

i

252

i

i

i /

/

/

/

split screen: closeup of mother, son talking to her split screen: POV shot through keyhole, son talking to her split screen: closeup of mother; son shaking the door, then walking through the living room split screen: POV shot through keyhole, extreme close-up of son

21.

22.

23.

24.

/

“See what I mean? See how you always gotta upset me, Ma?”

“. . . come out?”

“Then why won’t you. . . ”

gentle music in the background getting louder

very gentle music in the background

very gentle music in the background

very gentle music in the background

i

253

i

i

i knocking

broaching

/

/

split screen: POV shot through keyhole, medium shot of son split screen: closeup of mother, son acting in living room split screen: closeup of mother, son acting in living room

close-up of mother watching her son through keyhole

25.

26.

27.

28.

/

gentle music in the background (violins), short break (silence) — then music getting louder

“This isn’t happening. And if it should be happening it would be alright. So don’t worry, Seymour. It’ll all work out. You’ll see already. In the end it’s all nice.”

gentle music in the background, then accompanied by piano

gentle music in the background getting louder

/

“Ma. . . Ma. . . C’mon out?”

/

i

254

i

i

i /

/

the third protagonist standing in the entrance hall, talking

32.

30.

the son coming out of the flat with the TV

/

a third protagonist talking on the corridor

29.

31.

/

title of the film on black screen falling down the screen, letters dissolving

violin strokes in the background, accompanied by piano

“Hey, baby, ah don’t care just as long as we get our braid.”—“Just give me a hand.”

“Shit, man! That mother fucker’s starting to look a little seedy Jim.”

violin strokes in the background

violin strokes in the background

/

“What, are you particular all of a sudden?”

music gets dominant: strong strokes of the violin bow, accompanied by wind section

i

255

i

i

i murmur of the women in the background, noise of the table

sound of moving the TV table

sound of moving the TV table

sound of moving the TV table

sound of moving the TV table

long shot outside: women sitting in front of the house the son and the third protagonist moving the TV along the street

medium shot of women son and third protagonist walking along the street, talking to the women medium shot of women

33.

34.

35.

36.

37.

“Yeah. . . hello.”

“Hello.”

/

/

(murmur, several dialogues)

piano in the background, very gentle

piano in the background, very gentle

piano in the background, very gentle

piano in the background, very gentle

piano in the background, very gentle

i

256

i

i

i sound of moving the TV table

sound of moving the TV table

sound of moving the TV table

/

/

son and third protagonist walking along the street

medium shot of women

long shot of women sitting in front of the house

the men walking through the streets

credits, letters dissolving

38.

39.

40.

41.

42.

/

/

/

/

/

soundtrack

soundtrack: rhythmic, violins, mostly dominant; violins getting faster

piano in the background, very gentle, then getting louder

piano in the background, very gentle

piano in the background, very gentle

i

257

i

i

i /

/

/

/

/

medium shot of the protagonists walking through the streets

credits, letters dissolving

long shot of a street

credits, letters dissolving

long shot of a street

43.

44.

45.

46.

47.

/

/

/

/

/

soundtrack

soundtrack

soundtrack

soundtrack

soundtrack

i

258

i

i

i /

/

/

/

/

credits, letters dissolving

long shot of the two men walking with the TV

medium shot of the two protagonists with the TV

credits, letters dissolving another scenery with the protagonists walking, a roller coaster in the background

48.

49.

50.

51.

52.

/

/

/

/

/

soundtrack

soundtrack

soundtrack

soundtrack

soundtrack

i

259

i

i

i /

/

/

/

background noise (market, street. . . )

credits, letters dissolving

medium long shot of the roller coaster

the two protagonists in front of the roller coaster insert ”SUMMER” on black screen, falling down the screen

an older man sitting in a garage

53.

54.

55.

56.

57.

“Whoopie fuck, the table too.”

/

/

/

/

/

silence when letters arrived at centre of the screen

soundtrack

soundtrack

soundtrack

i

260

i

i

i /

/

/

/

/

long shot of the garage, the two protagonists in front of it

close-up of older man

the two protagonists standing in front of the man, talking to him

older man talking

close-up of a bank note

58.

59.

60.

61.

62.

/

/

“Such a son. A goniff.”

“Your mother needs you. . . ”

/

/

/

“I ain’t my lepers schlepper. Shit.”

“You got a friend.”

“Hey, do you want me to schlep it on my back?”

i

261

i

i

i /

/

moaning

/

crackling noise

long shot of garage

close-up of a mouth

a white bubble on black background

close-up of a lighter

extreme close-up of bubbles

63.

64.

65.

66.

67.

/

/

/

/

“. . . like a moose needs a hat rack.” — (laughing)

/

/

/

/

/

i

262

i

i

i /

shortly increasing noise

/

brewing

/

close-up of a fluid in an injection

close-up of a pupil, dilating

close-up of a second mouth (darker skin)

close-up of a bottle’s cap

close-up of a steaming fluid in the cap

68.

69.

70.

71.

72.

/

/

/

/

/

/

/

/

/

/

i

263

i

i

i /

moaning

/

/

close-up of a fluid in an injection

abstract image of the infiltration of the fluid into the blood a close-up of a pupil, dilating (darker skin) long shot from above: the two men listening to music and dancing

73.

74.

75.

76.

“Sheeit. . . that’s some boss scag, baby.”

/

/

/

music (record player) — short break — music

/

/

/

i

264

i

i

i

i

Filmography Abre los ojos (1997). Alejandro Amen´abar. Canal Espana, Spain / France / Italy. Am´elie (2001). Jean-Pierre Jeunet. Claudie Ossard Productions, France / Germany. Original title: Le fabuleux destin d’Am´elie Poulain. Industrial Britain (1931). Arthur Elton and Robert J. Flaherty and Basil Wright. Empire Marketing Board Film Unit, UK. Requiem for a Dream (2000). Darren Aronofsky. Artisan Entertainment, USA. The Lives of Others (2006). Florian Henckel von Donnersmarck. Arte, Bayerischer Rundfunk, Creado Film, Germany. Original title: Das Leben der Anderen. Vanilla Sky (2001). Cameron Crowe. Paramount Pictures, USA / Spain. WORDS (2010). Daniel Mercadante and Will Hoffman. Everynone, USA.

265 i i

i

Bibliography Afantenos, S., Denis, P., Muller, P. and Danlos, L. (2010), Learning Recursive Segments for Discourse Parsing, in N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner and D. Tapias, eds., ‘Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10)’, European Language Resources Association (ELRA), Valletta, Malta. Allen, J. F. (1984), ‘Towards a General Theory of Action and Time’, Artificial Intelligence, 23, 123–154. Andrew, J. D. (1976), The Major Film Theories. An Introduction, Oxford University Press. Ankersmit, F. R. (1983), Narrative Logic. A Semantic Analysis of the Historian’s Language, Martinus Nijhoff. Arnheim, R. (1960), Art and Visual Perception: A Psychology of Creative Eye, University of California Press. Arnheim, R. (1969), Visual Thinking, University of California Press. Asher, N. (1993), Reference to Abstract Objects in Discourse, Kluwer Academic. Asher, N. (2008), Troubles on the Right Frontier, in P. Kuhnlein ¨ and A. Benz, eds., ‘Constraints on Discourse (CID 2005). Pragmatics & Beyond New Series’, John Benjamins, pp. 29–52. Asher, N. and Lascarides, A. (2003), Logics of Conversation, Cambridge University Press. Asher, N. and Vieu, L. (2005), ‘Subordinating and Coordinating Discourse Relations’, Lingua, 115, 591–610. Baggett, P. (1979), ‘Structurally Equivalent Stories in Movie and Text and the Effect of the Medium on Recall’, Journal of Verbal Learning & Verbal Behavior, 18(3), 333–356. Baldry, A. P. and Thibault, P. (2006), Multimodal Transcription and Text Analysis. A Multimedia Toolkit and Coursebook, Equinox. Bamberg, B. (1983), ‘What Makes a Text Coherent?’ College Composition and Communication, 34(4), 417–429. Bateman, J. A. (2007), ‘Towards a Grande Paradigmatique of Film: Christian Metz Reloaded’, Semiotica, 167(1/4), 13–64. Bateman, J. A. (2013), Filmische Textualit¨at jenseits der narrativen Instanz, in J. Bateman, M. Kepser and M. Kuhn, eds., ‘Film, Text, Kultur. Beitr¨age zur ¨ Textualit¨at des Films’, Schuren, pp. 88–138. Bateman, J. A. and Rondhuis, K. (1997), ‘Coherence Relations: Towards a General Specification’, Discourse Processes, 24, 3–49. Bateman, J. A. and Schmidt, K.-H. (2011), Multimodal Film Analysis: How Films Mean, Routledge. Bazin, A. (1967), What Is Cinema. Essays selected and translated by Hugh Gray, University of California Press. Bordwell, D. (1985), Narration in the Fiction Film, Routledge. Bordwell, D. (1989), Making Meaning. Inference and Rhetoric in the Interpretation of Cinema, Harvard University Press.

266 i i

i

i

i

Bibliography 267 Bordwell, D. (2004), Neo-Structuralist Narratology and the Functions of Filmic Storytelling, in M.-L. Ryan, ed., ‘Narrative Across Media. The Languages of Storytelling’, University of Nebraska Press, pp. 203–219. Bordwell, D. (2006), The Way Hollywood Tells It. Story and Style in Modern Movies, University of California Press. Bordwell, D. (2008), Poetics of Cinema, Taylor & Francis Group. Bordwell, D. (2009), Cognitive Theory, in P. Livingston and C. Plantinga, eds., ‘The Routledge Companion to Philosophy and Film’, Routledge, pp. 356–367. Bordwell, D. (2011), ‘Common Sense + Film Theory = Common-Sense Film Theory?’ David Bordwell’s website on cinema: Essays. http://www.davidbordwell.net/essays/commonsense.php

Bordwell, D., Staiger, J. and Thompson, K. (1985), The Classical Hollywood Cinema. Film Style & Mode of Production to 1960, Routledge & Kegan Paul. Bordwell, D. and Thompson, K. (2001), Film Art: An Introduction, McGraw Hill. Bouquet, S. and Sanders, C. (2006), Ferdinand de Saussure. Writings in General Linguistics, Oxford University Press. Branigan, E. (1975), ‘Formal Permutations of the Point-of-View Shot’, Screen, 16(3), 54–64. Branigan, E. (1992), Narrative Comprehension and Film, Routledge. Colburn, T. R. (1991), ‘Defeasible Reasoning and Logic Programming’, Minds and Machines, 1(4), 417–436. Colin, M. (1995), The Grande Syntagmatique Revisited, in W. Buckland, ed., ‘The Film Spectator: From Sign to Mind’, Amsterdam University Press, pp. 45–86. Dorai, C. and Venkatesh, S. (2003), ‘Bridging the Semantic Gap with Computational Media Aesthetics’, IEEE MultiMedia, 10(2), 15–17. Doxiadis, A. (2010), ‘Narrative, Rhetoric, and the Origins of Logic’, Story Worlds: A Journal of Narrative Studies, 2, 77–99. Eckkrammer, E.-M. (2004), Drawing on Theories of Inter-Semiotic Layering to Analyse Multimodality in Medical Self-Counselling Texts and Hypertexts, in E. Ventola, C. Charles and M. Kaltenbacher, eds., ‘Perspectives on Multimodality’, Benjamins, pp. 211–226. Eco, U. (1990), The Limits of Interpretation, Indiana University Press. Eickmann, B. (2003), Incrementality in Discourse Update, Master’s thesis, School of Informatics, University of Edinburgh. http://www.inf.ed.ac.uk/publications/thesis/online/IM030069.pdf

Eisenstein, S. (1949), Film Form. Essays in Film Theory. Edited and translated by Jay Leyda. Harvest Book. Falck, M. (2006), Das Leben der Anderen, in Bundeszentrale fur ¨ politische Bildung/bpb Fachbereich Multimedia & IT, ed., ‘Filmheft Das Leben der Anderen. Florian Henckel von Donnersmarck. Deutschland 2005’, dmv druckmedienverlag, pp. 3–22. http://www.bpb.de/files/NSUEAK.pdf

Firth, J. R. (1957), A Synopsis of Linguistic Theory, 1930–1955, in T. P. Society, ed., ‘Studies in Linguistic Analysis’, Blackwell, pp. 1–32. Fledelius, K. (1979), ‘Syntagmatic Film Analysis’, Papiere des Munsteraner ¨ Arbeitskreises fur ¨ Semiotik e.V. (papmaks), 8, 31–68.

i i

i

268

Bibliography

Forceville, C. J. (2007), ‘Book Review: Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Coursebook: Anthony Baldry, Paul J. Thibault, Equinox, London/Oakville, 2006’, Journal of Pragmatics, 39(6), 1235–1238. Frege, G. (1980), Translations from the Philosophical Writings of Gottlob Frege. Edited by Peter Geach and Max Black. Blackwell. Gallie, W. B. (1966), Peirce and Pragmatism, Dover. Genette, G. (1980), Narrative Discourse: An Essay in Method. Translated by Jane Lewin. Cornell University Press. Graesser, A. C., McNamara, D. S. and Louwerse, M. M. (2003), What Do Readers Need to Learn in Order to Process Coherence Relations in Narrative and Expository Text? in A. P. Sweet and C. Snow, eds., ‘Rethinking Reading Comprehension’, Guilford, pp. 82–98. Grice, H. (1975), Logic and Conversation, in P. Cole and J. Morgan, eds., ‘Syntax and Semantics’, Vol. 3, Academic Press, pp. 41–58. Grimes, J. (1975), The Thread of Discourse, Mouton. Groenendijk, J. and Stokhof, M. (1991), ‘Dynamic Predicate Logic’, Linguistics and Philosophy, 14, 39–100. Halliday, M. (1978), Language as Social Semiotic: The Social Interpretation of Language and Meaning, Edward Arnold. Halliday, M. (2004), An Introduction to Functional Grammar. Third Edition, Edward Arnold. Halliday, M. and Hasan, R. (1976), Cohesion in English, Longman. Harris, Z. (1952), ‘Discourse Analysis’, Language, 28, 1–30. Hartung, M. (2006), Die Ausnutzung von Diskurswissen zum Zwecke der Informationsextraktion. Zur Gewinnung impliziter Information aus Texten. Master’s thesis, Ruprecht-Karls-Universit¨at Heidelberg. Heim, I. (1983), On the Projection Problem for Presupposition, in M. Barlow, D. P. Flickinger and M. Westcoat, eds., ‘Proceedings of the West Coast Conference on Formal Linguistics’, Vol. 2, pp. 397–405. Herman, D. (2002), Story Logic. Problems and Possibilities of Narrative, University of Nebraska Press. Hjelmslev, L. (1961), Prolegomena to a Theory of Language. Translated by F.J. Whitfield. Originally published 1943. University of Wisconsin Press. Ho-Dac, L.-M., Fabre, C., Pery-Woodley, M.-P. and Rebeyrolle, J. (2009), ‘Corpus Annotation of Macro Discourse Structures’. http://www.um.es/lacell/aelinco/contenido/pdf/60.pdf

Hobbs, J. R. (1979), ‘Coherence and Coreference’, Cognitive Science, 3(1), 67–82. Hobbs, J. R. (1990), Literature and Cognition, Lecture Notes. Hobbs, J. R. (2003), Discourse and Inference, Information Sciences Institute. http://www.isi.edu/∼hobbs/disinf-tc.html

Hobbs, J. R. and Redeker, G. (1997), A Note on Coherence Relations. Unpublished. http://www.isi.edu/∼hobbs/redeker.pdf

Hobbs, J. R., Stickel, M. E., Appelt, D. E. and Martin, P. H. (1993), ‘Interpretation as Abduction’, Artificial Intelligence, 63, 69–142. Jasinskaja, K. (2010), ‘Knowledge Base: Quest, the Paper Scene’. http://www.ims.uni-stuttgart.de/∼jasinsk/Event sequences/Other documents/ wissensbasis.pdf

i i

i

i

i

Bibliography 269 Jasinskaja, K. and Rossdeutscher, A. (2009), ‘Through Narrative Planning Towards the Preverbal Message: A DRT-based Approach’. http://www.ims.uni-stuttgart.de/∼jasinsk/Event sequences/Papers/jasinskaja rossdeutscher 2009.pdf

Jaworski, A. and Thurlow, C. (2009), Gesture and Movement in Tourist Spaces, in C. Jewitt, ed., ‘The Routledge Handbook of Multimodal Analysis’, Routledge, pp. 253–262. Jewitt, C. (2009), The Routledge Handbook of Multimodality, Routledge. Jost, F. (1989), L’oeil-cam´era. Entre film et roman. 2eme e´ dition revue et augment´ee, Presses Universitaires de Lyon. Kamp, H. (1981), A Theory of Truth and Semantic Representation, in J. Groenendijk, T. M. V. Janssen and M. Stokhof, eds., ‘Truth, Interpretation and Information. Selected Papers from the Third Amsterdam Colloquium’, Mathematics Center, pp. 1–42. Kamp, H. (2007), ‘Script Semantics II: Syllabus Semantics II’. http://www.ims.uni-stuttgart.de/∼hans/Teaching/SemII.Scr.PrtI.\-1.21.12.07.pdf

Kamp, H. and Reyle, U. (1993), From Discourse to Logic. Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory, Kluwer Academic. Kehler, A. (2002), Coherence, Reference, and the Theory of Grammar, CSLI. Kintsch, W. (1988), ‘The Role of Knowledge in Discourse Comprehension: A Construction-Integration Model’, Psychological Review, 95, 163–182. Kracauer, S. (2004), From Caligari to Hitler. A Psychological History of the German Film. Revised and Expanded Edition. Edited and Introduced by Leonardo Quaresima. Princeton University Press. Kress, G. (1993), ‘Against Arbitrariness: The Social Production of the Sign as a Foundational Issue in Critical Discourse Analysis’, Discourse and Society, 4(2), 169–93. Kress, G. (2010), Multimodality. A Social Semiotic Approach to Contemporary Communication, Routledge. Kress, G. and van Leeuwen, T. (1996), Reading Images: The Grammar of Visual Design, Routledge. Kress, G. and van Leeuwen, T. (2001), Multimodal Discourse. The Modes and Media of Contemporary Communication, Arnold. Kuhn, M. (2011), Filmnarratologie. Ein erz¨ahltheoretisches Analysemodell, de Gruyter. Kuhn, M. (2013), Narrativit¨at Transmedial. Von der sprachbasierten zur audio¨ visuellen Narratologie. Uberlegungen zur medialen Reichweite der Narrativit¨at und den Grundlagen einer audiovisuellen Filmnarratologie, in J. Bateman, M. Kepser and M. Kuhn, eds., ‘Film, Text, Kultur. Beitr¨age zur Textualit¨at des ¨ Films’, Schuren, pp. 58–87. Lascarides, A. and Asher, N. (1991), Temporal Interpretation, Discourse Relations and Common Sense Entailments, HCRC/RP -16 Edinburgh. Human Communication Research Centre. Lascarides, A. and Asher, N. (2007), Segmented Discourse Representation Theory: Dynamic Semantics with Discourse Structure, in H. Bunt and R. Muskens, eds., ‘Computing Meaning: Volume 3’, Kluwer Academic Publishers, pp. 87–124. Lemke, J. (1985), Ideology, Intertextuality and the Notion of Register, in J. D. Benson and W. S. Greaves, eds., ‘Systemic Perspectives on Discourse. Selected Theoretical Papers from the 9th International Systemic Workshop’, Ablex, pp. 275–294.

i i

i

270

Bibliography

Lemke, J. (1998), Multiplying Meaning. Visual and Verbal Semiotics in Scientific Text, in J. R. Martin and R. Veel, eds., ‘Reading Science’, Routledge, pp. 87–113. Liu, Y. and O’Halloran, K. L. (2009), ‘Intersemiotic Texture: Analyzing Cohesive Devices between Language and Images’, Social Semiotics, 19(4), 367–388. Longacre, R. E. (1996), The Grammar of Discourse. Second Edition, Plenum Press. Lundquist, L. (1989), Coherence in Scientific Texts, in W. Heydrich, F. Neubauer, J. S. Petofi ¨ and E. Sozer, ¨ eds., ‘Connexity and Coherence. Analysis of Text and Discourse’, de Gruyter, pp. 122–149. Magliano, J. P., Miller, J. and Zwaan, R. A. (2001), ‘Indexing Space and Time in Film Understanding’, Applied Cognitive Psychology, 15, 533–545. Maier, E. and Hovy, E. (1993), Organizing Discourse Structure Relations Using Metafunctions, in H. Horacek and M. Zock, eds., ‘New Concepts in Natural Language Generation’, Pinter, pp. 69–86. Mancini, C. (2000), ‘From Cinematographic to Hypertext Narrative’, Hypertext 2000, pp. 236–237. Mancini, C. (2005), Cinematic Hypertext. Investigating a New Paradigm, IOS Press. Mann, W. C. and Thompson, S. A. (1987), ‘Rhetorical Structure Theory: A Theory of Text Organization’, Text, 8(3), 243–281. Martin, J. (1992), English Text — System and Structure, Benjamins. Martin, J. and Rose, D. (2003), Working with Discourse: Meaning beyond the Clause, Continuum. Martinec, R. and Salway, A. (2005), ‘A System for Image-Text Relations in New (and Old) Media’, Visual Communication, 4(3), 337–371. Matthiessen, C. M. (2007), The Multimodal Page: A Systematic Functional Exploration, in T. Royce and W. Bowcher, eds., ‘New Directions in the Analysis of Multimodal Discourse’, Lawrence Erlbaum Associates, pp. 1–62. Meng, A. P. K. (2004), Making History in From Colony to Nation: A Multimodal Analysis of a Museum Exhibition in Singapore, in K. O’Halloran, ed., ‘Multimodal Discourse Analysis: Systemic-Functional Linguistics’, Continuum, pp. 28–54. Metz, C. (1974), Film Language: A Semiotics of the Cinema, University of Chicago Press. Metz, C. (1995), The Impersonal Enunciation, or the Site of Film, in W. Buckland, ed., ‘The Film Spectator: From Sign to Mind’, Amsterdam University Press, pp. 140–163. Mitchell, E. (2011, November 2), ‘Am´elie (2001). Film Review; Little Miss Sunshine as Urban Sprite’, New York Times. Moeller-Nass, K.-D. (1986), Filmsprache — Eine kritische Theoriegeschichte, MAkS Publikatkionen. Montague, R. (1974), Formal Philosophy: Selected Papers of Richard Montague. Edited and with an Introduction by Richmond H. Thomason. Yale University Press. Muller, P. and Reymonet, A. (2005), Using Inference for Evaluating Models of Temporal Discourse, in ‘Proceedings of the 12th International Symposium on Temporal Representation and Reasoning (TIME’05)’. Noad, B. and Unsworth, L. (2007), ‘Semiosis in the Film Soundtrack: Aural Perspective and Social Distance in The Queen Filmtrailer’, Literacy Learning: The Middle Years, 15(2), 8–19. O’Halloran, K. (1999), ‘Interdependence, Interaction and Metaphor in MultiSemiotic Texts’, Social Semiotics, 9(3), 317–354.

i i

i

i

i

Bibliography 271 O’Halloran, K. (2004), Visual Semiosis in Film, in K. O’Halloran, ed., ‘Multimodal Discourse Analysis: Systemic Functional Perspectives’, Continuum, pp. 109– 130. Peirce, C. S., Hartshorne, C. and Weiss, P. (1979), Collected Papers of Charles Sanders Peirce, Belknap Press of Harvard University Press. Polanyi, L. (1985), A Theory of Discourse Structure and Discourse Coherence, in W. H. Elifort, P. D. Kroeber and K. L. Peterson, eds., ‘Papers from the General Session of the 21st Regional Meeting of the Chicago Linguistics Society’, University of Chicago, pp. 306–322. Pollock, J. L. (1974), Knowledge and Identification, Princeton University Press. Portner, P. and Partee, B. (2002), Formal Semantics. The Essential Readings, Blackwell. Prevot, L. and Vieu, L. (2008), The Moving Right Frontier, in A. Benz and ¨ P. Kuhnlein, eds., ‘Constraints in Discourse’, John Benjamins, pp. 53–66. Pudovkin, V. (1926), Film Technique and Film Acting, Bonanza Books. Republished by Grove, New York, 1960. Reichman, R. (1985), Getting Computers to Talk Like You and Me: Discourse Context, Focus, and Semantics, MIT Press. Rumelhart, D. E. (1975), Notes on a Schema for Stories, in D. G. Bobrow and A. Collins, eds., ‘Representation and Understanding: Studies in Cognitive Science’, Academic Press, pp. 185–210. Saeed, J. I. (2003), Semantics. Second Edition, Blackwell. Sandys, J. E. and Cope, E. M. (1970), The Rhetoric of Aristoteles, Olms. Sperber, D. and Wilson, D. (1986), Relevance. Communication and Cognition, Basil Blackwell. Stanovich, K. and West, R. (1981), ‘The Effect of Sentence Context on Ongoing Word Recognition: Tests of a Two-Process Theory’, Journal of Experimental Psychology: Human Perception and Performance, 7, 658–672. Stede, M. (2004), The Potsdam Commentary Corpus, in B. Webber and D. K. Byron, eds, ‘Association for Computational Linguistics (ACL) 2004 Workshop on Discourse Annotation’, pp. 96–102. ¨ Stockl, H. (2004), In Between Modes. Language and Image in Printed Media, in E. Ventola, C. Charles and M. Kaltenbacher, eds., ‘Perspectives on Multimodality’, Benjamins, pp. 9–30. ¨ M. (1997), Film Erleben. Narrative Strukturen und physiologische Prozesse — Suckfull, “Das Piano” von Jane Campion, Sigma. Thompson, K. (1988), Breaking the Glass Armor. Neoformalist Film Analysis, Princeton University Press. Tibus, M. (2008), Do Films Make You Think? — Inference Processes in Expository ¨ Film Comprehension, PhD thesis, Eberhard-Karls-Universit¨at Tubingen. Tseng, C. (2009), Cohesion in Film and the Construction of Filmic Thematic Configuration: A Functional Perspective. PhD thesis, University of Bremen. Tseng, C. (2013), Cohesion in Film: Tracking Film Elements, Palgrave Macmillan. Tseng, C. and Bateman, J. A. (2010), Chain and Choice in Film Narrative: An Analysis of Multimodal Narrative Construction in The Fountain, in C. Hoffmann, ed., ‘Narrative Revisited: Telling a Story in the Age of New, Multimodal Media’, John Benjamins, pp. 213–244.

i i

i

272

Bibliography

Tseng, C. and Bateman, J. A. (2012), ‘Multimodal Narrative Construction in Christopher Nolan’s Memento. A Description of Analytic Method’, Journal of Visual Communication, 11(1), 91–119. van Dijk, T. (1980), Macrostructures: An Interdisciplinary Study of Global Structures in Discourse, Interaction, and Cognition, Erlbaum Associates. van Dijk, T. (1985), Discourse and Communication: New Approaches to the Analysis of Mass Media Discourse and Communication, de Gruyter. van Dijk, T. (1990), Issues in Functional Discourse Analysis, in H. Pinkster, ed., ‘Liber Amicorum for Simon Dik’, Foris, pp. 27–46. van Dijk, T. A. and Kintsch, W. (1978), Cognitive Psychology and Discourse: Retelling and Summarizing Stories, in W. U. Dressler, ed., ‘Current Trends in Text Linguistics’, de Gruyter, pp. 61–80. van Dijk, T. A. and Kintsch, W. (1983), Strategies of Discourse Comprehension, Academic Press. van Leeuwen, T. (1991), ‘Conjunctive Structure in Documentary Film and Television’, Continuum, 5(1), 76–114. van Leeuwen, T. (1996), Moving English: The Visual Language of Film, in S. Goodman and D. Graddol, eds., ‘Redesigning English—New Texts, New Identities’, Routledge, pp. 81–105. van Leeuwen, T. (1999), Speech, Music, Sound, Macmillan Palgrave. van Leeuwen, T. (2005), Introducing Social Semiotics, Routledge. von Stutterheim, C., Halm, U. and Carroll, M. (2012), Macrostructural Principles and the Development of Narrative Competence in L1-German: The Role of Grammar (8–14 year olds), in S. Benazzo, M. Hickmann and M. Watorek, eds., ‘Comparative Perspectives to Language Acquisition: A Tribute to Clive Perdue’, Multilingual Matters, pp. 59–585. Watkins, T. (2007), ‘He Who Has Ears To Hear’. Exploring the Message Behind the Media. http://www.damaris.org/content/culturewatcharticles/535

Webber, B. L. (1991), ‘Structure and Ostension in the Interpretation of Discourse’, Language and Cognitive Processes, 6(2), 107–135. Weber, H. J. (1984), Elements of Text-Based and Image-Based Connectedness in Comic Stories, and Some Analogies to Cinema and Written Text, in M.-E. Conte, ¨ and E. Sozer, ¨ J. S. Petofi eds., ‘Text and Discourse Connectedness. Proceedings of the Conference on Connexity and Coherence Urbino, July 16–21, 1984’, John Benjamins, pp. 337–360. Wildfeuer, J. (2012), ‘More than WORDS. Semantic Continuity in Moving Images’, Image and Narrative. Online Magazine of the Visual Narrative, 13(4), 181–203. Wildfeuer, J. (2013), Der Film als Text. Ein Definitionsversuch aus linguistischer Sicht, in J. Bateman, M. Kepser and M. Kuhn, eds., ‘Film, Text, Kultur. Beitr¨age ¨ zur Textualit¨at des Films’, Schuren, pp. 32–57. Wildgen, W. (1994), Process, Image, and Meaning, John Benjamins. Wildgen, W. (2013), Erz¨ahlung und Action im James Bond-Film: Ein Quantum Trost, in J. Bateman, M. Kepser and M. Kuhn, eds., ‘Film, Text, Kultur. Beitr¨age ¨ zur Textualit¨at des Films’, Schuren, pp. 320–344. Wirth, U. (1995), ‘Abduktion und ihre Anwendungen’, Zeitschrift fur ¨ Semiotik, 17, 405–424.

i i

i

i

i

Bibliography 273 Wirth, U. (2000), Die Welt als Zeichen und Hypothese. Perspektiven des semiotischen Pragmatismus von Charles Sanders Peirce, Suhrkamp. Wirth, U. (2005), ‘Abductive Reasoning in Peirce’s and Davidson’s Account of Interpretation’, Semiotica, 153(1), 199–208. Wuss, P. (1993), Filmanalyse und Psychologie. Strukturen des Films im Wahrnehmungsprozess, Sigma. Zacks, J. M. and Magliano, J. P. (2011), Film, Narrative, and Cognitive Neuroscience, in D. P. Melcher and F. Bacci, eds., ‘Art and the Senses’, Oxford University Press, pp. 435–454. Zacks, J. M., Speer, N. K. and Reynolds, J. R. (2009), ‘Segmentation in Reading and Film Comprehension’, American Psychological Association, 138(2), 307–327. Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S. and Reynolds, J. R. (2007), ‘Event Perception: A Mind-Brain Perspective’, Psychological Bulletin, 133(2), 273– 293. Zhao, S. (2009), Intersemiotic Relations as Logogenetic Patterns: Towards the Restoration of the Time Dimension in Hypertext Description, in J. R. Martin and M. Bednarek, eds., ‘New Discourse on Language: Functional Perspective on Multimodality, Identity and Affiliation’, Continuum, pp. 195–218.

i i

i

Index Abre los ojos (1997), 110, 265 Afantenos et al. (2010), 206, 266 Allen (1984), 65, 66, 70, 266 Am´elie (2001), 87, 265 Andrew (1976), 33, 266 Ankersmit (1983), 193, 266 Arnheim (1960), 35, 266 Arnheim (1969), 35, 266 Asher and Lascarides (2003), 2, 15, 16, 21, 22, 31, 37, 40–53, 59–65, 67, 69–71, 73, 74, 76–83, 92, 120, 123, 128, 152, 164, 186, 189, 194, 195, 198, 205, 266 Asher and Vieu (2005), 198, 206, 266 Asher (1993), 206, 266 Asher (2008), 205, 206, 266

Eckkrammer (2004), 178, 267 Eco (1990), 202, 267 Eickmann (2003), 44–46, 79, 267 Eisenstein (1949), 7, 267 Falck (2006), 4, 207, 267 Firth (1957), 172, 267 Fledelius (1979), 33, 267 Forceville (2007), 36, 268 Frege (1980), 39, 268 Gallie (1966), 14, 268 Genette (1980), 12, 268 Graesser et al. (2003), 41, 268 Grice (1975), 42, 268 Grimes (1975), 38, 268 Groenendijk and Stokhof (1991), 39, 268

Baggett (1979), 58, 266 Baldry and Thibault (2006), 32, 33, 36, 266 Bamberg (1983), 10, 266 Bateman and Rondhuis (1997), 84, 196, 266 Bateman and Schmidt (2011), 4, 5, 7–9, 12, 13, 17, 18, 58, 66, 83, 96, 129, 187– 189, 196, 200, 201, 204, 209, 210, 266 Bateman (2007), 8, 9, 13, 33, 122, 123, 129, 196, 266 Bateman (2013), 2, 10, 210, 266 Bazin (1967), 7, 266 Bordwell and Thompson (2001), 7, 64, 87, 267 Bordwell et al. (1985), 12, 147, 192, 200, 267 Bordwell (1985), 12, 87, 164, 167, 176, 177, 188, 189, 193, 200, 206, 266 Bordwell (1989), 5, 11, 13, 15, 167, 176, 177, 266 Bordwell (2004), 209, 267 Bordwell (2006), 12, 206, 267 Bordwell (2008), 2, 12, 13, 177, 191, 267 Bordwell (2009), 209, 267 Bordwell (2011), 169, 179, 201, 210, 211, 267 Bouquet and Sanders (2006), 7, 9, 267 Branigan (1975), 155, 267 Branigan (1992), 12, 267

Halliday and Hasan (1976), 38, 170, 176, 268 Halliday (1978), 170, 172–175, 268 Halliday (2004), 84, 170, 172, 179, 196, 268 Harris (1952), 37, 268 Hartung (2006), 42, 268 Heim (1983), 39, 268 Herman (2002), 17, 182, 183, 193, 268 Hjelmslev (1961), 171, 268 Ho-Dac et al. (2009), 207, 268 Hobbs and Redeker (1997), 41, 268 Hobbs et al. (1993), 15, 16, 268 Hobbs (1979), 11, 268 Hobbs (1990), 40, 72, 268 Hobbs (2003), 15, 268 Industrial Britain (1931), 54, 265 Jasinskaja and Rossdeutscher (2009), 52, 165, 268 Jasinskaja (2010), 52, 165, 269 Jaworski and Thurlow (2009), 175, 269 Jewitt (2009), 2, 176, 269 Jost (1989), 90, 269 Kamp and Reyle (1993), 10, 15, 39, 269 Kamp (1981), 39, 269 Kamp (2007), 36, 269 Kehler (2002), 72, 269 Kintsch (1988), 59, 269 Kracauer (2004), 33, 269 Kress and van Leeuwen (1996), 2, 30, 34, 35, 54, 73, 141, 155, 173, 175, 180, 269

Colburn (1991), 44, 267 Colin (1995), 8, 267 Dorai and Venkatesh (2003), 209, 210, 267 Doxiadis (2010), 17, 192, 267

274 i i

i

i

i

Index 275 Kress and van Leeuwen (2001), 2, 32, 173, 179, 269 Kress (1993), 180, 269 Kress (2010), 10, 269 Kuhn (2011), 206, 269 Kuhn (2013), 206, 269 Lascarides and Asher (1991), 46, 269 Lascarides and Asher (2007), 31, 39, 40, 46, 65, 66, 68, 193, 269 Lemke (1985), 171, 269 Lemke (1998), 176, 270 Liu and O’Halloran (2009), 176, 178, 183, 270 Longacre (1996), 38, 270 Lundquist (1989), 11, 270 Magliano et al. (2001), 54, 55, 270 Maier and Hovy (1993), 60, 196, 270 Mancini (2000), 52, 270 Mancini (2005), 52, 270 Mann and Thompson (1987), 60, 270 Martin and Rose (2003), 9, 170–172, 175, 270 Martin (1992), 170, 171, 176, 270 Martinec and Salway (2005), 176, 178, 183, 270 Matthiessen (2007), 183, 270 Meng (2004), 175, 270 Metz (1974), 7, 8, 270 Metz (1995), 18, 270 Mitchell (2011, NOvember 2), 138, 270 Moeller-Nass (1986), 8, 270 Montague (1974), 38, 270 Muller and Reymonet (2005), 15, 270 Noad and Unsworth (2007), 35, 270 O’Halloran (1999), 176, 270 O’Halloran (2004), 2, 13, 174, 175, 203, 271 Peirce et al. (1979), 1, 14–16, 81, 271 Polanyi (1985), 205, 206, 271 Pollock (1974), 44, 271 Portner and Partee (2002), 38, 39, 271 Prevot and Vieu (2008), 82, 206, 271 Pudovkin (1926), 7, 271 Reichman (1985), 38, 271 Requiem for a Dream (2000), 87, 265 Rumelhart (1975), 38, 271

Saeed (2003), 39, 271 Sandys and Cope (1970), 10, 271 Sperber and Wilson (1986), 16, 17, 271 Stanovich and West (1981), 58, 271 Stede (2004), 207, 271 ¨ Stockl (2004), 34, 179, 271 ¨ (1997), 208, 271 Suckfull Thompson (1988), 12, 271 Tibus (2008), 58, 271 The Lives of Others (2006), 87, 265 Tseng and Bateman (2010), 16, 271 Tseng and Bateman (2012), 13, 168, 203, 272 Tseng (2009), 7, 13, 14, 21, 84, 168, 171, 176–178, 191, 271 Tseng (2013), 7, 13, 35, 176, 177, 180, 182, 183, 203, 271 van Dijk (1980), 38, 272 van Dijk (1985), 36, 37, 272 van Dijk (1990), 172, 272 van Dijk and Kintsch (1978), 57, 272 van Dijk and Kintsch (1983), 57–59, 272 Vanilla Sky (2001), 87, 265 van Leeuwen (1991), 13, 29, 54, 176, 196, 272 van Leeuwen (1996), 180, 184, 185, 272 van Leeuwen (1999), 35, 36, 272 van Leeuwen (2005), 173, 174, 272 von Stutterheim et al. (2012), 52, 165, 272 Watkins (2007), 207, 208, 272 Webber (1991), 205, 206, 272 Weber (1984), 33, 272 Wildfeuer (2012), 22, 272 Wildfeuer (2013), 10, 89, 272 Wildgen (1994), 193, 272 Wildgen (2013), 203, 272 Wirth (1995), 15, 272 Wirth (2000), 14, 273 Wirth (2005), 14, 273 WORDS (2010), 22, 265 Wuss (1993), 209, 273 Zacks and Magliano (2011), 55, 57, 59, 96, 104, 273 Zacks et al. (2007), 54, 273 Zacks et al. (2009), 54–56, 273 Zhao (2009), 178, 273

i i