--- 7o;-r--

" .;:

MULTIMODAL
THE MODES AND MEDIA OF CONTEMPORARY COMMUNICATION

DISCOURSE
GUNTHER KRESS & THEO VAN LEEUWEN
 First published in Creat Britain in 2001 by
Hodder Education, part of Hachette L-K,
338 Euslon Road, London l\W1 3BH

www.hoddcrcducation.co.uk

(g)2001 Cunther Kress and Thco \-an Lee-..:"'·en

All righls rescrved. No parl of Ihis publication m.ay =-e :e?roduced or

transmitled in any form or by any means, ele<"troruc.a:t:" o: mechanically,
including photocopying, rccording or any inforrr..ahon S!Orageo: remeval
system, without either prior permission in writing fro:::¡ L;e publisher or a
licence permitting reslrictcd copying. In Ihe United KI:-.géornsuch licences
are issued by the Copyright Licensing Agency: Saifnr: H...~.....se, 6-10 Kirby
Street, London EClN 8TS

The advice and information in this book arE.'~-eC ::;:> :.e =-.;.e a..-td
accurate allhe date of going lo press, bul neither the.a::::::xcs-:x:r~ r":::.:s..~
can accept an)' legal responsibilily or liabilily fo: a..~. ~ cr ~

Britisll Library Catalogllif/g if/ PucllG- ..•.

-: ::::=.
A catalogue record for this book is available fro=1 :::e 3..~'l.. T-~.

Librnry 01 Congress Catalogil!g·in·P:JH:.:=L~ :::::::

A cata lag record for this book is available from :.~ :"--:r-;- ::.i Ca:tgtess

ISBN 978 O 340 60877 i

8 9 10

Typcsel in 10/13pt Utopia by Phocnix Photoseo::::-~ e-o" -:oa::n. J<enl

Prinled and bound in India by Replika ~?T-_ :..::d

What do you think aboul Ihis book? Or an)" othe r~ Education tille?
PIcase visil our website: www.hodde:rd~_uk
mullimodal Discourse
The modes and media of
conlemporary communication

GUNTHER KRESS
Culture, Communication and Societies,
Institute of Education, University of London

THEO VAN LEEUWEN

Centre for Language and Communication Research,
Cardiff University

l, ~D~~A~I~~
PART OF HACHETIE UK
 Contents

Preface vii
lntroduction 1
2 Discourse 24
3 Design 45
4 Production 66
5 Distribution 86
6 lssues for the multimodal agenda 111
References 134
Index 139
Multimodality

For some time now, there has been, in Western culture, a distinct preference for
monomodality. The most highly valued genres of IVriting (Iiterary novels, academic
treatises, official documcnts and reports, etc.) came entirely ,,~thout illustration, and
had graphically uniform, dense pages of print. Paintings nearly all used the same
support (canvas) and the same medium (oils), IVhatever their style or subject. In
concert performances all musicians dressed identically and only conductor and
soloists were allolVed a modicum of bodily expression. The specialised theoretical
and critical disciplines IVhich developed to speak of these arts became equally
monomodal: one language to speak about language (Iingllistics), another to speak
about art (art history), yet another to speak about music (musicology), and so on,
each with its OlVnmethods, its OIvn assumptions, its own technical vocabulary, its
OIVnstrengths and its olVn blind spots.
More recently this dominan ce of monomodality has begun to reverse. Not only
the mass media, the pages of magazines and comic strips for example, but also the
docllments produced by corporations, lIniversities, government departments etc.,
have acquired colour ilIustrations and sophislÍcated layout and typography. And not
only the cinema and the semiotically exuberant performances and videos of popular
music, but al so the avant-gardes of the 'high culture' arts have begun ro use an
increasing variety of materials and ro cross the boundaries betIVeen the various art,
design and performance disciplines, tOIVards multimodal Gesamtkunstwerke, multi-
media events, and so on.
The desire for crossing boundaries inspired twentieth-century semiotics. The
main schools of semiotics all sought ro develop a theoretical framework applicable to
all semiotic modes, from folk costume ro poelry, from traffic signs ro c1assical music,
from fashion ro lhe theatre. Vet lhere IVas also a paradox. In our oIVn work on visual
semiotics (Kress and Van LeeuIVen, 1996), IVe,too, IVere in a sense 'specialisls' of lhe
image, still slanding Wilh one foot in lhe \Vorld of monomodal disciplines. But al lhe
sam~ lime \Ve aimed al a common terminology for all semiotic modes, and slressed
that, \Vilhin a given social-cultural domain, the 'same' meanings can often be
expressed in different semiotic modes.
In lhis book IVemake lhis move our primary aim; and so \Veexplore lhe common
2 Mu/timorla/ rliscourse

principies behind multimodal communicarion. We move away from rhe idea rhar
rhe different modes in multimodal rexrs have srricrly bounded and framed specialisr
rasks, as in a film where images mal' provide rhe acrion, sync sounds a sense of
realism, music a layer of emolion, and so on, IVilh Ihe ediling process supplying Ihe
'imegralion code', Ihe mean s for synchronising Ihe elemems Ihrough a common
rhylhm (Van Leeuwen, 1985). Inslead \Ve move lOIVards a vieIV of multimodalily in
\Vhich common semiolic principies operale in and across differem modes, and in
wJiich il is lherefore quile possible for music lO encode aClion, or images lO ellcode
emolion. This move comes, on our pan, nOl beca use IVe lhink \Ve had ir all IVrong
before and have no\V suddenll' seen lhe ligh!. Ir is because \Ve IVam to creare a
lheory of semiolics appropriale to comemporary semiolic praclice. In rhe pas!, and
in manl' contexrs srill todal', mulrimodal rexls (such as films or neIVspapers) lUere
organised as hierarchies of specialisl modes integraled by an ediling process.
Moreover, Ihey IVere produced in rhis IVay, IVirh different, hierarchically organised
specialisrs in charge of lhe differem modes, and an ediring process bringing rheir
IVork logerher.
Today, hOIVever, in lhe age of digilisalion, Ihe differenr modes have rechnically
beco me rhe same ar some level of represenlalion, and lhey can be operaled by one
multi-skilled person, using one interface, one mode ofphysical manipularion, so rhar
he or she can as k, ar every point: 'Shall I express lhis \Virh sound or music?', 'Shalll sal'
lhis visually or verballyr, and so on. Our approach lakes ilS poinl of depanure from
rhis new developmem, and seeks to provide lhe elemenr rhar has so far been missing
from rhe equalion: rhe semiotic ralher Ihan Ihe rechnical element, Ihe queslion of
ho\V lhis rechnical possibilily can be made ro \Vork semiolically, of holV IVe mighr
have, nor only a unified and unifying lechnology, bUl also a unified and unifying
semiotics.
Lel us give one specific example. In Reading lmages (1996) IVediscussed 'framing'
as specific 10 visual communicalion. By 'framing' \Ve meant, in that context, the way
elemems of a visual composilion mal' be disconnecled, marked off from each orher,
for insrance by framelines, piclorial framing devices (boundaries formed by rhe edge
of a building, airee, elc.), empry space bet\Veen elemems, disconlinuiries of colour,
and so on. The concepr also included lhe \Vays in IVhich elemenls of a composirion
may be connecled to each orher, rhrough lhe absence of disconneclion devices,
lhrough vecrors, and rhrough continuilies and similarilies of colour, visual shape and
so on. The significance is rhal disconnecred elements IVillbe read as, in some sen se,
separa le and independent, perhaps even as conlrasling unils of meaning, whereas
connecled elements \ViIIbe read as belonging logerher in so me sen se, as conrinuous
or complemenlary. Arnheim's discussion ofTitian's Noli Me Tangere (1982: 112)
provides an example: 'IChrisr'sl sraff acrs as a visual boundary berIVeen rhe figures',
he comments, and 'Magdalen breaks lhe visual separarion ... by rhe aggressive acr of
her righl arm' (see Fig. 1.1).
 Introductioll 3

Figure I.l Noli Me TI/llgere

Bur c1early framing is a muhimodal principIe. There can be framing, not only
between the elements uf a visual composition, but also berlVeen the bits oflVriting in
a newspaper or magazine layout (Kress and Van Leeuwen, 1998), betll'een the people
in an office, the seats in a train or restaurant (e.g. private comparrments versus
sharing tables), the dlVeJlings in a suburb, etc., and such insrances offraming lViJlalso
be realised by 'framelines', empty space, discontinuities of aJl kinds, and so on. In
time-based modes, moreover, 'framing' beco mes 'phrasing' and is realised by the
shon pauses and discontinuities of various kinds (rhythmic, dynamic, etc.) which
separa te the phrases of speech, of music and of actors' movements. We have here a
common semiotic principIe, though differently realised in different semiotic modes.
The search far such common principIes can be undenaken in differenr lVays. It is
possible 10 work out detailed grammars for each and every semiotic mode, detailed
accounts of what can be 'said' with thm mode and holl', using for each of the
grammars as much as possible (as much as the materiality of the mode makes that
plausible) the same approach and the same terminology. At the end of this process it
would then become possible 10 overlay these different grammars and lO see where
they overlap and where they do not, which areas are common to which of rhe modes,
and in which respects the modes are specialised. There have by now been a number
of attempts al devising such grammars, all based 10 a greater or les ser degree on the
semiotic theories ofHaJliday (HaJliday 1978,1985) and Hodge and Kress (1988), and
hence sharing a common approach - for instan ce the semiotics of action of Maninec
(1996, 1998), the semiotics of images of OToole (1994) and Kress and Van Leeuwen
(1996), the semiotics of sound of Van LeeulVen (1999), the semiotics of thearre of
Manin (1997) and Mclnnes (1998), and so on.
 4 JHultimodal discollrse

We are and will cominue to be part of Ihis enlerprise ourselves. BUI in Ihis book
\Ve wam to pause, as il lVere, to lake stock of \Vhal general piclure is emerging. We
\Vam to skelch a multimodal Iheory of communicalion based, nor on ideas which
naluralise Ihe characlerislics of semiolic modes by equaling sensory channels and
semiolic modes, bUI on an analysis oflhe specificities and common trailS of semioric
modes IVhich lakes account of rheir social, cultural and historical produclion, of
IVhen and holV Ihe modes ofproduclion are specialised or multi-skilled, hierarchical
. or leam-based, ofwhen and how lechnologies are specialised or multi-purpose, and
so on.

The issue of rneaning in a rnultirnodal theory of cornrnunication

We indicaled in Ihe preface Ihal il was our focus on praclices and our use of Ihe
nOlion of reSOllrces, ralher Ihan a focus on fixed, slable entities, IVhich allolVed us to
make progress wilh a multimodal approach lO representalion and communcialion.
In relalion 10 one specific queslion Ihis has been particularly crucial, namely Ihe
queslion of meaning. The Iradirionallinguislic account is one in which meaning is
lI1ade once, so to speak. By conlrasl, we see Ihe multimodal resources which are
available in a culture used 10 make meanings in any and every sign, al every level, and
in any mode_ Where Iradilional Iinguislics had defined language as a syslem Ihal
worked through dOllble artiCl/laliol1, \Vhere a message lVas an articulalion as a form
and as a meaning, we see multimodallexls as making meaning in mulriple articula-
lions. Here \Veskelch Ihe four domains ofpraclice in which meanings are dominantly
made. We call rhese SIra/a to sho\V a relarion ro Hallidayan funcrional Iinguislics, for
reasons of Ihe porenlial compalibiliry of descriprion of differenr modes. We do nor
hOlVever see strala as being hierarchically ordered, as one aboue Ihe olher for
inslance, or some such imerprelalion. Qur four strala are diseourse, design, produc-
rion and disrribulion.

Discollrse

Discourses are socially construcled kno\Vledges of (some aspecr 00 realiry_ By

'socially construcled' lVe mean Ihal rhey have been developed in specific social
cOntexlS, and in lVays \Vhich are appropriare to Ihe inleresrs of social actors in rhese
contexlS, wherher Ihese are very broad contexrs ('Wesrern Europe') or nol (a
particular family), explicilly inslilurionalised comexls (newspapers) or nol (dinner-
lable conversalions), and so on. For inslance, Ihe 'elhnic conflicl' discourse of\Var
can be drawn on by WeSlern journalisls when reporting civillVars in Africa or former
Yugoslavia, bUI ir is also an available resource in certain kinds of conversation, in
airport Ihrillers or in movies sel in Africa, and so on. War discourses involve borh a
 11llroducti01l 5

certain version of what actually happens in wars, of who is involved, what they do,
and where and when, and a set of interpretations, evaluative judgements, critical or
justifying arguments and so on, related to \Vars or aspects of them. The 'ethnic
conflict' discourses of war in newspapers, for instan ce, serve the interests of the
countries in which the newspapers are produced, as perceived by the projected
readership of the papers. Hence they usually leave out mention of the influence of
colonisation and de-colonisation and defend non-intervention by constructing
conflicts as going back hundreds of years or more, to mention just t\Voaspects. There
are other discourses of war, for instance discourses in which 'economics' or
'ideology' feature as explanatory categories. These will inelude and exelude other
participants and events, link their versions of \Vhat actually goes on in wars with other
interpretations, judgements, arguments etc., and serve other interests. And while
sorne discourses inelude a great deal of emphasis on the actual events and provide
few interpretations or arguments, others form a storehouse of abstract interpretation
and argument but make do with only a broad and general version of what warring
parties actually do.
Any discourse may be realised in different ways. The 'ethnic conflict' discourse of
war, for instance, may be realised as (part 00 a dinner-table conversation, a television
documentary, a newspaper feature, an airport thriller, and so on. In other lVords,
discourse is relatively independent of geme, of mode and (somelVhat less) of designo
Yet discourses can only be realised in semiotic modes which have developed the
means for realising them. In the 1920s, following the Russian Revolution, film had not
developed the means for realising Marxist discourses. Hence a film-maker like
Eisenstein, for instance, who dreamt offilming Marx's Capila/, set about developing
his method of 'dialectical montage' (Eisenstein, 1949), and in the process extended
the semiotic reach of the medium.

Design

Design stands midlVay between content and expression. It is the conceptual side of
expression, and the expression side of conception. Designs are (uses 00 semiotic
resources, in all semiotic modes and combinations of semiotic modes. Designs are
means to realise discourses in the context of a given communication situation. But
designs also add something nelV: they realise the communication situation which
changes socially constructed knolVledge into social (inter-) action. Consider writers
IVho write thrillers in a setting of 'ethnic conflict', for instance: at the same time as
they realise the 'ethnic conflict' discourse of lVar, they realise a particular mode of
interaction in which it is their purpose to entertain an audience of a particular kind.
In doing so, designs may either follolV lVell-trodden paths ofhabit, convention, tradi-
tion, or prescription, or be innovative and ground-breaking, just as discourses may
either express common sense, or be innovative and perhaps even subversive,
6 M/llrimodnl discollrse

But design is still separa te from the actual material production of the semiotic
product or the actual material articulation of the semiotic evenl. The resources on
which design draws, the semiOlic modes, are still abstract, capable of being realised
in different materialities. Language, for instance, is a semiotic mode because it can be
realised either as speech or as IVriting, and IVriting is a semiotic mode too, because it
can be realised as engraving in stone, as calligraphy on certificates, as print on glossy
paper, and all these media add a further layer of signification. The IVriter of the
. 'ethnic connict' thriller, apart from using language, also uses the resources of the
mode of narrative in designing the thriller. And this mode is separate from
the ",edil/", of the printed book in which it \Viiibe produced. The same design mal'
be realised in different media. The same story mal' become a Illainstrcalll movie or an
airport thriller, given a shared communicative purpose and conception of \Vho the
audience is. Quite different skills are of course required for actually IVriting tlle book
or producing the movie.
This vielVof design also applies to semiotic practices IVhich do not so clearly have
a 'subject maller'. An architect, for instance, designs (but does not build) a house or
a block of apartlllents. The discourse provides a certain view of how houses are lived
in, of how many and which kinds of people live in houses, of IVhal they do in their
houses, coupled lVith interpretations of IVhythey live the way Ihey do, and argulllents
which critique or defend Ihese lVays of life. The design of the house then conceptu-
alises holV 10 give shape 10 Ihis discourse in the form of a house or a type of apart-
menl. According to architect Chris Timmerman (1998: 11-12), there are architectural
projects 'which are never builr, but remain on paper, in the mind, on the hard disk',
and they often are ground-breaking archilecture because 'one can allow oneself the
luxury and freedom of concelllraring on Ihe spalial experiential aspects of architec-
ture as opposed to the economic and structural reality of building'. He quotes Virilio
(1997: 26) to support the idea that architecture can be realised in several different
materialities, not only in the form of buildings, but also, for instance, as interactive
compUler programmes: 'While the topical City lVas once constructed around the gate
and the port, the telelOpical metaciry is now reconstructed around the windolV and
the teleport, that is 10 sal', around the screen and the time slot'.

Prodllcrio//

'Production' refers 10 the organisation of the expression, 10 the actual material

articulation of the semiotic evenl or the actual material production of the semiotic
artefacl. A IVhole other set of skills is involved here: lechnical skilis, skills of the hand
and the eye, ski lis related nOI 10 semiOlic modes, but 10 semiOlic media. We use Ihe
term 'medium' here in the sense of 'medium of execution' (the material substance
draIVn into culture and worked over cultural time), the sen se in IVhich artists use it
when they speak of the medium of 'oil', or 'tempera on paper', or 'bronze moullled
 /nrroductioll 7

on marble base'; it applies of course also lO media \Vhich do nOl produce Iraces Ihat
last beyond Ihe momem of aniculalion, such as speech or music.
Somelimes design and produclion, mode and medium, are hard lO separa le.
Improvising musicians, for inslance, bOlh design and perform lheir music. They
rehearse, perhaps, bUI even in rehearsals il may be difficuh lO know where 'design'
ends and 'performance' begins. In olher comexls there is a gap belween Ihe 111'0,and
Ihey separale OUI in differem roles: composers design the music and performers
exetule it. In Ihal case Ihe work of perfol'lners \ViII oflen be seen as adding Iinle
meaning, as 'merel)' realising and making audible Ihe imemions of the composer as
fairhfully as possible, and as adding, al besl, Ihe 'expressiveness' \Vhich black dors on
paper do nol have. Linguisrs have Ihe same view of language: the expression plane
do es nO! add meaning and 'merely' realises whar can also be writren down, \Virhour
1055of essentialmeaning. Teachers, for instance, may eirher design Iheir own lessons
or merely 'execute' a detailed syllabus designed byexpen educalors. In O!her words,
when design and produclion separale, design beco mes a means for ['ontrolling the
actions of olhers, Ihe pOlel11ial for a unity berween discourse, design and produclion
diminishes, and rhere is no longer room for Ihe 'producers' lO make rhe design 'their
oll'n', lO add rheir own accent. In all rhis, wriring and ils abiliry lO provide delailed
'scripls' and 'prescriplions' ('pro-grammes') for acrion has undoubledly played a
pivotal role.

Disrribwioll

As already memioned, Ihe stratum of expression needs lO be srratified funher.

Musical performers may need Ihe technicians who record the music on lape and disc
for preservalion and dislribution; designers of a producI may need the crafls people
who produce Ihe prolO!ype of Ihe producI, and Ihe olher crafls people ",ho produce
Ihe mould for mass production.
Distribulion, too, tends to be seen as not semiolic, as not adding any meaning, as
mereIy facilitating rhe pragmatic functions of preservation and distribution. Just as ir
is rhe performer's job lO be failhfullO Ihe imentions of the composer, so il is rhe
recording and sound-mixing engineers' job lO achieve 'high fidelity': '1 wan! lO make
records which \Viiisound in the public's home exaclly Iike wha! Ihey would hear in
Ihe best seat in an acoustically perfecI hall', said EMI producer Waher Legge (quoted
in Chanan, 1995: 133). But the public's home is nO! a coneen hall, and acoustically
perfecr halls do nOI exist. Imroducing orcheslral music into Ihe home and being able
lO hear Ihe same performance over and over already fundamentally challges lhe
mealling of music, for example Ihrough Ihe 105501''aura' of which Waher Benjamin
\Vrole (1977). As time moves on, dislribulion media may, in pan or in whole, tum into
production media. The contribulion oflhe sound engineer may beco me equallO rhal
of Ihe musician, lI'ilh paramelers Iike reverb used, nOllO (re)creale 'Ihe acouslically
8 MIl/timoda/ discollrse

perfect hall', but to act as independent signifiers, able, for instan ce, to make sounds
either 'interior' and subjective or 'exterior' and objective, as in many contemporary
dance music mixes (Van Leeuwen, 1999). \Vhere the drum and bass are so 'clase up'
that they do not seem to be played in an aClllal space at all, but inside the head or
body, in a space where all sound is absorbed instantly.

Articulation and interpretation

The terms \Vehave used ('design', 'production', 'distribution') might suggest that \Ve
are looking at multimodal communication only from the point of view of the
producers. But this is not so. Our model applies equally to interpretation. Indeed,
we define communication as only having taken place when there has been both
articulation and interpretation. (In fact IVe might go one step further and say that
communication depends on sorne 'interpretive community' having decided that
sorne aspect of the world has been articulated in order to be interpreted.)
Interpreters need to supply semiotie knolVledge at all four of the levels IVe have
distinguished. At the level of distribution, they need to know, for instance, whether
they are dealing \Vith a reproduetion or an original, even in cases where the
boundaries are deliberately blurred, as in sorne of Andy Warhol's work. They also
need lO understand the respective values of 'design' and 'production'. Adorno
(1976,1978). forlVhom 'structurallistening' IVasthe highest form ofmusic Iistening,
condemned jazz because of the simplicity of what IVe call here its 'design' (the
simple chord schemas of Broad\Vay songs). For this he IVas taken to task by
Middleton (1990). who argued that he did not know how lO appreciate the semiotic
richness of IVhat IVe call here the 'production' of jazz singers and musicians. The
same phenomenon sometimes occurs in comparisons between Iiterary novels and
their movie adaptations.
Design and discourse play their role in interpretation lOO, even though a given
interaction may be experienced differently, and a given discourse interpreted
differently, from the way it was intended. A story may be wrinen to entertain, bur an
interpreter may not be entertained because of the story's built-in ethnocentric bias
against the interpreter's ethnic group. A product may be designed to make its use
easy, but certain users may not appreciate products which do their thinking for
them. Such users operate from a different discourse, a different conception of \Vhat
is involved in that task and a different set of associated values and ideas. Which dis-
courses interpreters or users may bring to bear on a semiotic producl or event has
everything lO do, in turn, with their place in lhe social and cultural \Vorld, and also
\Vith the conten!. The degree to which intention and interpretation \ViII match
depends on contex!. For inslance, mosl of us interpret a traffic sign the same IVay
(there are differences: do you slow dOIVn when amber appears, or do you speed
 11llroduCliOll 9

up?l. unless it is panicularly badil' designed, or unless an interpreter has recently

emerged from a place IVhere there is no traffic. But when, for instance, a traffic sign
is displayed as an obje! !/'OI/vé in an an gallery, our interpretations are likely 10 differ
significantly.

StrataI configurations
__ o - • __

At the level ofthe social organisation or semiotic production different configurations

of discourse, design, production and distriburion mal' occur. Three of these mal' be
merged for instan ce, as in everyday conversational speech, where any speaker or
listener incorpora tes discourse, design and production skills and probably
experiences them subjectively as one and the same. Nevenheless, even here they do
remain distinct strata. Speakers need access to discourses, knoll'ledges II'hich are
socially structured fOrlhe purpose at hand; they need 10knoll' holl' 10formulate these
knowledges in the appropriate register and holl' to embed them in an (inter)active
event; and they need to be able 10 speak. Much as we might take lhese skills for
granled and see them as a unified II'hole, they are distinct, as 1V0uld quickly become
apparent if any one of lhem beca me impaired.
Al the other end of lhe scale from everyday conversalion lVe mighl have lhe
speech, sal', of professional voice-over specialists. Here the division of labour is
maximised. Each Slralum involves different people and difrerent skilis. Expen
sources provide lhe discourse, scriptwriters the design, voice specialists the voices,
recording engineers lhe recordings, and so on. Yellhe division of labour is nOllolal.
The expens lVillbe handpicked for their understanding oflVhar lhe media need and
lheir ability lO provide lhe kinds of discourse appropriate 10 television
documentaries. lhe scriptwriters will have ro kllow somethillg about televisioll
produclion so as nol 10 II'rile lhings IVhich cannOl be filmed or are 100 expensive 10
film, and so as lO make good use of the medium's specific 'produclion values'. The
voice-over specialists muSl understand whallhey are reading and rake accoul1l of lhe
requiremel1ls of the recording engineers, by keeping their voice al an evenlevel, nOl
rllstling the paper, and so on.ln other 1I'0rds, \VhallVe shall call 'strataluncoupling' is
never absolure.
Moreover, the tlVOtypes of semiotic prodllction exist in the same socie!)'. We live
in a 1V0rld IVhere discourse, design and production no longer form a uni!)', IVhere
teachers are lrained lO teach without any reference 10 II'hat they might be teaching,
managers 10 manage without any reference 10 what they might be managing,
intervie\Vees 10being il1lervielVed withour any reference to whar the intervie\Vs might
be abou!. Again, in many COnlexts lVe are encouraged or even obliged to reproduce
discollrses 'in our olVn lI'ords', that is, without also reproducing their designo And lVe
know that design and production are sometimes coupled, so that differenl
10 J'vlll/timodal discollrse

productions of the same design can be regarded as 'saying the same thing' (e.g.
performances of c1assical music) and sometimes uncoupled (e.g. jazz performances,
where two different versions of the same tune might be 'saying' something quite
different). This makes our semiotic landscape fundamentally different from that of
oral societies where knolVledge is indissolubly welded to its formulation, and where
the distinetion betIVeen 'what you say' and 'how you say it' IVould be difficult to
understand.
It is aboye all the invention of writing which has made this possible, which has
disrupted the direct link between discourse and production that can still be observed
for instance in the semiotic prodllcrion ofyoung children (Kress, 1997). Writing has
produced 'Ianguage', a semiotic resource no longer tied to its material realisation, no
longer just 'tonglle' (the original meaning ofthe word 'Iangllage') or 'inscription' (the
word 'graphic' originally meant 'make marks', 'scratch'), but 'S)~1tax' (a word which
originated as a military term, meaning 'organisarion', 'battle formation', and only
later came also to mean 'organise', 'write', 'compose'). As such, wriring can be used
to create order, and to govern human acrion, and make it predictable, repeatable,
whether this is internalised as a set of grammatical rules, or externalised as a script, a
written proeedure, a programme, a syllabus, etc. It is only in certain marginal or
marginalised fields, or during times \Vhen new discourses, nelV designs, and/or new
modes of produetion and distriburion are needed, that a more immediate link
betlVeen discourse and prodllction is maintained or reinstated, and that orher less
prescriptive and systematic semioric principies come to the fore.
In this book IVe\ViiidisCllSS t\Vo such principies in panicular. The first is prove-
l/al/ce, 'where signs come from'. The idea here is that IVeeonstantly 'impon' signs
from other eontexts (another era, social group, culture) into the eontext in \Vhich IVe
are now making a ne\V sign, in order to signify ideas and values \Vhich are associated
\Vith that other contex! by those who impon the signo To take a musical example, in
the 1960s the Beatles introduced the sound of the sitar into their musie to signify
values whieh, in the 'psyehedelie' youth culture of!hat time, \Vere associated \Vith the
sitar's country of origin: meditation, drugs as expansion of consciousness, and so on.
The idea of'provenanee' is c10sely related to the ideas of'my!h' and 'eonnotation' as
introduced into semiotics by Roland Barrhes (1972, 1977).
lhe second is experielllialmeal/ing potel/tial, the idea that signifiers have a mean-
ing potential deriving from \Vhat it is we do when IVeproduce them, and from our
abili!y to turn action into knowledge, to extend our practical experience metaphori-
cally, and to grasp similar extensions made by orhers. To give an example, the sound
quality of 'breathiness' derives its meaning from our knowledge of the kinds of
situation in which it may occur - when we are out of breath, for instance, and when
\Ve are unable to control our breathing due to excitemenr. Hence 'breathiness' can
become a signifier for intimacy and sensuality, for instance in singing styles or in !he
speech in television commercials for products tha! can be associated with intimacy
 IlItroductioll 11

or sensuality. The same principIe mal' once upon a time have helped create the words
we nolV use: think ofthe way the IVords 'Ianguage' and 'tongue' both require a maxi-
mllm amount of IOnglle movement from the front 10 the back of the mouth. The idea
of 'experiential meaning potemiaJ' is close to the view of 'metaphor' elaborated in
Lakoff and Johnson (1980).
Thus the social stratification of semiotic production is mirrored by the stratifica-
tion of the semiotic resources themselves. And while it can be argued that 'distribu-
tion', at this stage, has not yet been imernalised as a 'stratum' of semiotic modes, it is
at least possible that new technologies, increasingly ubiquitous, multi-purpose and
'natural' in terms oftheir interfaces, will help crea te a fourth dimension of communi-
cation in the same IVaythat writing created a third - and this time not at the cost of a
decrease in multimodality.

Stephanie's bedroom as a multimodaI text

We \Viiiuse the discourse, design and production of children's bedrooms, and oftexts
about children's bedrooms, as a first example of our approach. 'Children's bedroom
discourses' form pan of 'family life' discourses, socially constructed knowledges
about \Vho forms pan of families, what family members do (lOgether or separately),
where they do it, which outsiders mal' take pan in which family activities, and so on.
There are always likely 10 be several such discourses, associated, for instance, with
different social classes or ethnic groups (in Britain and Australia many middle-class
families set a specific space aside for 'entenaining', for instan ce), or based on deviant
practices, wrong ways of living in a family home which therefore form a danger for
other families (such as the demonic children's bedroom of the next door kid Sid in
Disney's ToySlOry).
Discourses which are still in the process of being elaborated and have not yet
become common sense and subject to \Vhat Bourdieu has called 'genesis amnesia'
are of particular interest. Early socialist 'family life' discourses are an example of
this. They were developed in Ihe early decades of Ihe twentieth cemllry in several
European cities. The Amsterdam councillor Wibaut, for instance, began to visil
\Vorking-class families at home and found 'many d\Vellings where large families 'Aoth
six, seven, eight children Iived, cooked, worked and slept in one room' (Roegholt,
1976: 13). In 1904 he inaugurated a policy of declaring such dwellings uninhabitable
and building new suburbs for their occupants. In the process he and others
developed a discourse of workers' family Iives, in which workers would see their
homes as fonresses for protecting their families against a threatening oUlside \Vorld,
and as a place to relax afler a hard day's \Vork. Archilecls then realised this discourse
in buildings which indeed looked Iike fortresses (see Fig. 1.2). There were forbidding
fa~ades, heavy doors wilh small barred windows, hidden in deep and monumental
 IlItrodllctioll 13

emrance recesses, and lVindolVs so high Ihal Ihe occupanls needed lO sland on a
5100/10 loo k through them - proteclion againsl the Ihreatening outside \Vorld and
promolion of in\Vard-Iooking family values \Vas the mOlivation (Roeghoh, 1976: 32),
Hygiene was another key Iheme, at least for the dty planners, beca use the 1V0rkers
themselves oflen longed for Iheir remembered cosy alcoves, used Ihe loilels as
slorage and Ihe sholVers as broom cupboards, and did not appredate the lVashing
and drying spaces in Ihe anics \Vhich had been inlended 10 free Ihem from the smell
of drying lVashing: 'The \Vomen did nol Iike to do Iheir lVashing communally and
preferred 10 keep Iheir under\Vear to Ihemselves' (Hoegholt, 1976: 41),
Public'housing projecls in Vienna \Vere based on a similar discourse, Eventually
some of il became enshrined in Ihe la\V, \Vhich stipulates thal Ihere has 10 be
'approximately 10 square melres for every person', 'a kilchen and suilable sanilary
facililies for every household' and 'a bedroom separale from Ihe living-room in Ihe
case of families \Vilh children', Today Ihis la\Vis used to prevent immigrant workers
from being reuniled lVilh Iheir families, and so me magislrales funher elaborale on il
in an anempllO prove thatthe 'family Iife' discourse of Ihe immigrant 1V0rkers is not
a/'tsiiblic/¡, nOI 'in accordance lVilh localtradilion' (Van Leeuwen and Wodak, 1999),
One magislrale, for inslance, rejecled an applicalion because Ihe applicant's apan-
ment did not have 'space for Ihe social and eullUral developmenl of Ihe family', The
apanmeIll of another lacked a separa le bedroom for the daughter, a siluation IVhich
Ihe magistrale judged Ilot to be 'beneficial for the educational development of the
child', The applicants themselves had a different view of family Iife IVhich did not
include closed doors, and protested Ihat 'close spatial proximity betlVeen parents
and children is imponaIll',
The pictures of children's bedrooms in Hallse Beallliflll type magazines rarely
sho\V school-age children or teenagers, Only three- or four-year-olds are depicted, or
mentioned by name in the text. An anicle aboul three-year-old Stephanie's room
(Hallse Bealltiflll, September 1996: 160-2) contains some details of \Vhat young
children actually do in their room: 'The muhicoloured sofa provides Stephanie \Vith
somelVhere to sit and read her books'; 'Handy pegs \Vere anached 10Ihe bright yellOlv
dado that runs round the room lOmake it easy for Stephanie to hang up her coats and
toys', Slephanie also has a miniature theatre in her room, '1 sing and dance \Vith mI'
friends up here', she says, 'We dress IIp and act in our o\Vn plays - it's greal fun.' She
fcalures in tIVOofthe pictures, once looking up from a picture book, once holding up
a marionene on the stage of her miniature theatre (Fig, 1.3), Other pictures provide
evidence of at least tlVOother aClivilies, drawing (a blackboard on the door of the
IVardrobe) and sleeping (a bed lVith a colourful patchlVork quih),
Three-year-old Noel is shown in his room holding a 101' car (Hallse Beallfiflll,
September 1996: 32): 'Mosl of Ihe time you IVill find him playing IVilh his model
cars', And three-year-old Will is sholVn in t\Vo pictures, building a raillVay track in
one, and playing IVith a mllhi-Ievel garage in another. JI comer of a bed and a chest
 IntrodllCliOll 15

of dralVers suggest other acti\~ties (Ideal Homes alld Lifestyle, September 1998: 100).
As a social conmuction of \Vhat kind of (three-year-old) children Iive in 'beautiful
homes' and of \Vhat these children do in their bedrooms (playing with toys; neatly
plltting toys and clothes away; sleeping, always by themselves). these discOllrses are
clearly selective. They are also gendered: although there are toys in her room,
Stephanie does not play with toys, but reads, sings, dances and dresses up. Noel and
WiII play with trains and toys. The magazines also contain pictures of the rooms of
sorne older children. These usually include a desk, typically \Vith a desk Iight and a
globe: a place for home-work. The computer interface in Figure lA sholVs the
children's bedroom as a vinual space for playas well as work, with toys stacked on
shelves on the left and labelled dralVers for '\Vork' on the righ!.
A totally different family-Iife discourse emerges from the pictures in the 1998/99
IKEAcatalogue. As the occasionallVritings on blackboards, boa k spines etc. indicate,
the pictures lVere all taken in Sweden. Here children often play together (a boy and a
girl are reading together, for instance). and they also play with their parents (a father
is served a make-believe cup of tea in his daughter's rooml. Teenagers have
computers and hi-fis in rooms "~th pictures of pop stars and spons heroes on the
walls: 'After a cenain age, children want their olVn bit of personal space, somelVhere
to keep them happy, and keep all their stuff. somewhere to tell all their friends about'
(p.67).
Discourses not only provide versions of who does IVhat, when and where, they
add evaluations, interpretations and arguments to these versions. We have already
discussed sorne of the arguments of the socialist family-life discourses of the early
decades of this century. In magazines of the HOl/se Beal/tifl/I type, the arguments are
presented as common sen se and are not explicitly farmulated. Motivations come
across most often through cenain aspects of the rooms themselves, for instance lhe
colour scheme, and lhrough value-Iaden adjeclives. Let us have anolher loo k al
Slephanie's room. There is a slrong emphasis on colour, bOlh in lhe texl and in lhe
piclures, and the colours are called, on lhe one hand, 'bright', 'bold', 'dramatic' etc.,
and on lhe olher hand 'sunny' and 'cheerfuJ'. The anicle as a whole ends as follows:
'With so much inspiralion in her new room, Slephanie is full of ideas about what she
wants to be when she grows up. She's clearly had plenty of practice al being a
mountain climber ¡lhis refers lO fixtures in the room which were too high for her
before the redecoralionJ, and nolV she can add acting and interior design lOher CV!'
This children's bedroom is clearly a pedagogic tool, a medium for communicaling
to lhe child, in the language ofinterior design, the qualities (already complex: 'bold',
yet also 'sunny' and 'cheerflll'J, lhe pleasures (,singing and dancing wilh your
friends'), the duties (orderly management of possessions and, eventually, 'work').
and lhe kind of fUlUre her parents desire far her. This destiny, moreover, is commu-
nicated to her in a language that is to be lived, lived as an individual identity-building
and identily-confirming experience in thal indi\~dual bedrool11. Such a pedagogic
--
I:
'.
f
r
4
 Itllroductiofl 17

discourse is only one of a number of possible 'children's bedroom' discourses. There

are and will be orhers. But they have not found their way into the British magazines
lVehave looked at.
lhe pedagogic 'children's bedroom' discourse can be realised in a number of
lVays, lt can be realised as an actual children's room, through the multimodal
'language of interior design' in which meanings are realised by spatial arrangements
(the 'dado' which runs right around the room and makes 'puning your things aIVay'
literally an omnipresent feature ofthe room); by choice offurniture (the sofa, a place
for reading); by colour schemes (the 'bold' and yet also 'sunny' and 'cheerfu\'
colours); and so on. AII this has to be conceptualised as 'design' before it can be
produced, regardless oflVhether the parents themselves both design and produce the
redecoration, use a professional designer, or follow an explicit pre-existing model
designed or endorsed by an expen,
lhe same discourse can be realised as a HOlIse Bealltiflll aniele, in the text and
pictures of children's books, or in IKEAcatalogues. Here the practice of communicat-
ing pedagogic messages through the design of a children's bedroom is represented in
other contexts, contexts such as the magazine, or the children's book, And these
contexts have their own communicative purposes and their olVn 'recipients'. The
children's book Mnrk al/d Mal/dy (Leete-Hodge, n,d,). for instance, is \Vrinen to be
read lO young children and deals with the transition from home 10 school. lhe t\Vo
children are apprehensive about 'the first day', but in the end school turns out 10 be
enjoyable, and the first day at school the most memorable event of their lives. lhe
children's room s are implicated: somelVhere along the way Mandy's room acquires a
new piece offurniture, 'a fine blackboard and easel, with a packet ofwhite chalks and
a yellow duster, just like schoo\'. Like Stephanie's room, Mnrk al/d Mal/dy has a
pedagogic purpose, 'getting children prepared for schoo\'. But it uses a different
method, the method of slOf)~elling. HOlIse Benlltiflll seeks to provide models for
creating the right kind of setting forrhe right kind offamily life. lhe houses it features
in the anieles are 'ideal homes', 'dream houses' 10 aspire to -Ihe homes of celebrities
and of model couples IVho have tastefully renovated their 'rustic-style cottages' and
'spacious Georgian houses', lhe 110uses featured in Ihe advertisements, on the other
hand, are a linle more dOlVnmarket.
lhe skills required for designing Hallse Beallliflll features about children's bed-
rooms differ from those required for designing children's bedrooms. lhey inelude
the skill of writing in a slyle appropriate for the purpose at hand, of producing the
right kind of photographs, designing the right kind oflayout, and so on. For one thing,
the audiences for the t\Vo differ: parents as readers in one case, children as users in
the other. The pictures, for instance, must be 'analytica\', piclUres which elearly show
how the room is made up ofits component pans (Kress and Van LeeuIVen, 1996). lhe
language similarly must foreground place, furniture, room flxtures, and sholV holV
the room and its various pans 'hang togelher'. But language does this in a different
18 MIlI/imadal discolITse

IVay from thar of image, for instance by 'thematising' the elemelHs of the room
(Halliday, 19851. putting rhem ar the head of the sen ten ces:

Handy pegs \Vere attached 10 rhe bright yellow dado that runs around the
room 10 make it easy for Stephanie 10 hang up her coars and roys.

Writers of children's books \Vould design rhe same content yet again differently.
They ",ould most probably 'thematise' character and action, add some derail about
the action perhaps, and reduce the derail of rhe description of the room and its
fixtures and furniture:

Stephanie nearly hung her coats and lOys on rhe yellow dado in her room.

Children's illustrated books ",ould in rheir tum be different. Whereas mosr of the
pictures in HOllse Beautiflll do not show people, most of the picrures in children's
books do, again 10 put the emphasis on characters and actions, the I\VOvital elements
in any slOry.
Design also involves a knowledge ofthe relationship bet\Veen ",ords and pictures.
lhe House Beautiftli article features no less rhan ten pictures on three pages, and rhey
occupy by far rhe grearesr amoulH of space on every one of rhese pages. Alrer all,
pictures are much better at conveying hall' furniture is arranged in a room, and at
'describing' exactly ",hat a sofa or a colour looks like. In spatial malters, language
comes a poor second ro image. BUI then, language is used for other things: to telllhe
slOry oflhe ",al' the house \Vas acquired and the room decorated, 10 link the layout of
the room to rhe child's activities, 10 reinforce the meanings ofthe colour scheme by
means of evaluative adjectives, and 10 bring out, however implicitly, the pedagogic
'message' of the room. The 'redecoration' story srarts the article, and the 'pedagogic
message' ends ir. In other \Vords, the I\VOsemiotic modes are given complemelHary
specialist tasks, jusr like the photographer and the writer.
lhe design of the anicle is quire similar to other features in HOllse Beautiful, and
10 fearures in other, similar magazines. Such relative standardisation is typical of
much journalistic IVork, and derives 10 quite some exrent from rhe srandardised
routines of journalistic work and rhe intricate division of labour of magazine produc-
tion. Vet, rhere is no 'recipe·. lhere is tradition, but not prescription, a formula, but
not a remplare, and it is this that makes it possible for the joumalists, the
pholOgraphers and the layout anists lOfeel thar every job presents a new challenge, a
new problem 10 be solved (Bell and Van LeeuIVen, 1994: 1741. A]though semiotic
modes have developed in this field, as can be demonstrated by linguistic analysis of
rhe 'generic structure' of journalistic ",riting and television intervie\Vs (e.g. Van
Leeuwen, 1987; Bell, 1991; ledema, 1993; Bell and Van LeeulVen, 1994), the IVriters,
pholOgraphers and designers can 'make these sryles lheir oIVn' and develop their
 Jllrroductioll 19

own 'accems'. It is not quite the textual equivalent of lVearing a uniform, but rather
the textual equivalent of lVearing a business suit, a prescribed form of dress IVhich
nevertheless lea ves the lVearer some room for a personal touch.
House Beall/¡Iul presems the story of holV Slephanie's room was produced as a
nelV invention, ralher Ihan as the parems' choice from a memal or actual catalogue
of socially available possibilities. Like Ihe socialist cily fathers of early twentieth-
century Amsterdam and Vienna, Stephanie's parems knew that rooms have to be
. 'Iight' and 'airy', but unfortunately Stephanie's room did not get much ligh!. How
could they resolve this?

They didn'l knolV IVhere to begin until a friend ca me round with a patchwork
duvel cover he'd bought as a presem for Stephanie. Boasting all the colours of
the rainbow, it was perfect for a youngster's bedroom and provided plenty of
inspiration for a new look.

This duvet cover is shown in one of the photos. It is made up of a number of squares
featuring simple, basic pictures of objects (a boal, a teapot, a car), in bright primary
colours. They are instances of a recognisable genre of comemporary pictures and
toys for very young children: 'essemial' 10comOlives, cars, planes, birds, trees, in
Mondrian-like colours. Yet this convemional colour scheme is here presemed as a
unique solution to the problem of how to make an existing space, IVhich lVas not
really designed for that purpose, fit a discourse. It is a problem which many families
face when selecting an apartmem or house which was perhaps built in a differem era
for a differem kind offamily life: how to accommodate it to conremporary family life.
Why is rhis? Is il because families should be se en ro have a unique idemity, and nor
one thar is, as it IVere, pre-designed, 'pre-fabricated' by dominam 'designs'? Or is it
to justify the magazine's presemation of this room as a 'model' room, an original
creation, a piece of art, weH IVorth imitating by lesser parems?
Finally, IVhereas House Beall/¡Iul and Mark alld Malld)' are mass-produced and
distributed to a dispersed readership, Stephanie's bedroom is of course unique: there
is only one and it can only be found in the tolVn where Stephanie lives. There is no
'distribution' stralUm in the case of architecture or imerior designo HOlVever, new
technology mal' yet change this. Virtual reality can nolV reproduce a given space in
such a way that one can walk through it and have a multi-sensory experience of i!. Ar
present, not least as a result of the encumbrances of goggles, datagloves etc., the
difference berween actual and virtual spaces seems overwhelming. Virtual reality
entails a complete loss of actual physical presence. But so did to Walter Benjamin the
difference betIVeen the work of arr and irs mechanical reproduction: a complele loss
of'aura'. How many of us still feel an essentiallack when looking al lhe reproducrion
of a work of art, or listening 10 the recording of a musical performance? The lime mal'
yet come IVhen little girls can while away countless hours in virtual rooms, and
20 M/llrimodal disco/lrse

experience a variety of identities, dulies and pleasures realised in a spatial mass

medium, a globally dislribulable language ofinterior designo

Conclusion

In Ihis chapler we have skelched Ihe oUlline of a Iheory of multimodal communica-

lion. We have defined multimodalily as Ihe use of several semiOlic modes in Ihe
design of a semiolic producI or event, logelher wilh Ihe panicular way in which Ihese
modes are combined - Ihey may for inslance reinforce each olher ('say Ihe same
Ihing in different ways'), fulfil complementa!')' roles, as in Ihe [-[alISe Beallliflll anicle
aboul Slephanie's bedroom, or be hierarchically ordered, as in aClion films, where
aClion is dominant, wilh music adding a lOuch of emolive colour and sync sound a
lOuch of realislic 'presence'. We defined communicalion as a process in which a
semiolic producI or event is bOlh aniculared or produced al/d inlerpreled or used. II
follows from Ihis definilion Ihal IVe consider Ihe produclion and use of designed
objecls and environments as a form of communicalion: we used Ihe example of a
room, bUI could al so have used a designed objecl as our example.
The main concepls IVe have introduced in the chapler are recapilulaled in Ihe
disCllssion oflerms below.

Recapitulation

Strata: The basis of slralification is Ihe distinction belween Ihe COl/lel/( and the
expressiol/ of communication, which includes that between the signifieds and the
signifiers ofthe signs used. As a result ofthe invention ofwriting, the content slratum
could be funher stratified into discourse and designo As a result of Ihe invention of
modern communication lechnologies, the expression stratum cauld be funher
stratified into production and distribution.
The slralification of semiolic resources has its counterpan in the social slralifica-
tion of semiotic produclion, cenainly in the early stages of the use of new communi-
cation lechnologies.ln later stages il may become possible forone person 10 produce
the product or event from stan 10 finish, as is beginning 10 happen IOday with
interactive multimedia.
In Ihis book we argue Ihal production and distribution produce tlleir own layers
of signification. Indeed, we have argued thal semiotic !nodes and design ideas
uSllally now out of prodllction, llsing principies of semiosis typical for prodllction,
sllch as provenance and experientialmeaning potential.

Discourse: Discourses are socially situated forms of knowledge about (aspects 00

realiry. This includes knowledge of the events constituting that reality (who is
 llltrodllctioll 21

involved, what takes place, where and when it takes place, and so on} as well as a set
of related evaluations, purposes, interpretations and legitimations.
People often have several alternative discourses available with respect to a
particular aspect of reality. rhey will then use the one that is most appropriate 10 the
interests of the communication situation in which they find themselves.

Design: Designs are conceptualisations of the fonn of semiotic products and events.
Three things are designed simultaneously: (1) a formulation of a discourse or combi-
nation of discourses, (2) a particular (inter}action, in which the discourse is
embedded, and (3) a particular wal' of combining semiotic modes.
Design is separa te from the aClualmalerial production of the semiOlic product or
event, and uses (abstract) semiotic modes as its resources. It mal' involve inter-
mediate productions (musical scores, play scripts, blueprints, etc.) but the forl11these
take is not the form in which the design is eventually to reach the public, and they
tend be produced in as abstract a 1110dalityas possible, using austere methods of
realisation Ihal do not involve any form of realistic delail, texlure, colour and so on.

Production: Produclion is the articulalion in malerial fonn of semiOlic products 01'

evenlS, whelher in Ihe fonn of a prolOlype thal is slill 10 be 'Iranscoded' inlO anolher
form for purposes of dislribulion (e.g. a 35 mm relemovie) or in ils final fonn (e.g. a
videolape packaged for commercial distribulion).
Production nol only gives perceivable fonn lO designs bllt adds meanings which
now direclly fromlhe physical process of articulalion and Ihe physical qualilies of Ihe
malerials used, for inSlance from Ihe articulalOry geslures involved in speech
produclion, or fromlhe weighl, colour and texlure of Ihe materialused by a Sculplor.

Dislribulion: Dislribulion refers 10 Ihe lechnical 're-coding' of semiolic producls

and events, for purposes of recording (e.g. lape recording, digilal recording) and/or
dislribulion (e.g. radio and lelevision Iransmission, lelephony).
Distribulionlechnologies are generally nOI intended as produclion lechnologies,
bUI as re-produclion lechnologies, and are Iherefore nOImeanl10 produce meaning
themselves. However, they soon begin to acquire a semiotic potential of their OWI1,
and even ul1\vanled 'noise' sources such as Ihe scralches and discoloralions of old
film prinls mal' beco me signifiers in Iheir own righ!. In Ihe age of digilal media,
however, Ihe funclions of production and dislribution become lechnically integrated
to a much greater extent.
Anolher key dislinction in this chapler is Ihe distinction between /liarle, which is
on Ihe 'conlenl' side, and lI1edill/1I, which is on the 'expression' side.

Mode: Modes are semiotic resources IVhich alioli' the simultaneous realisation of
discourses and types of (inter}action. Designs then use lhese resources, combining
22 Multilllodal discollrse

semiotic modes, and selecting from the options which they make available according
to the interests of a particular communication situation.
Modes can be realised in more than one production medium. Narrative is a mode
beca use it allows discourses 10 be formulated in particular ways (lVays which
'personify' and 'dramatise' discourses, among other things), because it constitutes a
particular kind of interaction, and because it can be realised in a range of different
media.
. Ir follows that media beco me modes once their principies of semiosis begin 10
be conceived of in more abstract ways (as 'grammars' of some kind). This in turn
will make it possible 10 realise them in a range of media. They lose their tie 10 a
specific form of material realisation.

Medium: Media are the material resources used in the production of semiotic
products and events, including both the 100ls and the materials used (e.g. the musical
instrument and air; the chisel and the block oflVood). They usually are specially pro-
duced for this purpose, not only in culture (ink, paint, cameras, computers), but also
in nature (our vocal apparatus).
Recording and distribution media have been developed specifically for the
recording and/or distribution of semiotic products and events which have already
been materially realised by production media, and as such are not supposed to func-
tion semiotically. But in the course of their development, they usually start function-
ing as produclion media - JUSI as production media may become design mades.

Lastly, IVediscussed the specific IVaysin which meaning is produced 'in production'.
This is not always a matter of 'realising designs', in the way that a speech may realise
\Vhat the speaker has prepared, or a building \Vhat the architect has designed, and it
certainly does not usually happen in the 'arbitrary' ways which have been fore-
grounded by linguists. In facl, signification starts on the side of production, using
semiotic principies which have not yet sedimented into conventions, traditions,
grammars, or laIVs of designo Only eventually, as the particular medium gains in
social importance, will more abstract modes ofregulation ('grammars') develop, and
the medium ,,011 become a mode. Tite opposite, modes becoming media again, is also
possible. The science of physiognomy, for instance, lost its status as a result of its
racist excesses, and nolV semiotic practices like casting are 'media' again, operating
on the basis of primary semiotic principies such as 'provenance' and 'experiential
meaning potential'.

Experiential meaning potential: This refers to the idea that material signifiers have a
meaning potential that derives from \Vhat it is IVedo when we articulate them, and
from our abiliry 10 eXlend our practical experience metaphorically and turn action
into knowledge. This happens, for instance, with the textural characterislics of sound
 Jlltroductioll 23

qualities (as when singers adopt a soft, breathy voice to signify sensuality), in the
absence of a conventionalised 'system' of sound qualities (such as the symphony
orchestra).

Provenance: This refers 10 the idea that signs may be 'imported' from one conrext
(another era, social group, culture) into another, in arder 10 signify the ideas and
values associated wirh rhar other context by rhose \Vho do the importing. This
happens, for instance, in gi,ong names 10 people, places or things (e.g. in naming a
perfume 'Paris') when there is no 'code', no sedimented set of rules for naming
perfumes.