Olga Anissimova
Maria Mäkelä
Katre Pärn
Tytti Rantanen
Maarja Saldre
Maria Seppänen
Katre Väli

Katre Pärn
Department of Semiotics
University of Tartu


Adaptation is defined as one specific, sometimes even quintessential, form of intertextual activity, where a story is transposed from one text to another, usually from one medium or signification system to another as well.

Delineated as a form of dialogic relation among texts, the problem of an adaptation is generally theorized as a text-based issue, concerned with what happens to a text when it is transferred to another medum. For example the problem of fidelity, the articulations of and departures form the source text, i.e. the measure of structural changes between the source text and adaptated text, the so-called politics of intertextuality as such and the function of adapted stories in culture.

Even the question of medium specificity – a perspective that assumes that representational practices have individual material and formal structures that distinguish and differentiate them from other practices (Corrigan 2007: 31) or to put it more shortly that every medium should develop its own unique language (Manovich 1999: 94) –, a perspective, that might, at first glace, seem rather as a question of expressive material, i.e. of language, is in adaptation studies essentially a text-based issue, being evaluated from the perspective of textual demands: what kind of aspects of a story one or the other media language is (in)capable to communicate or what kinds of narrative techniques it offers, what are the expressive possibilities or limitations that the story must be adapted to when transferred from one medium to another. For example the question of narrative strategies, where the main idea is that we have two different means for presenting narrative devices like focalization, voice, metaphor, POV etc and the critical distinction between those narrative features that can be transferred from one medium to another and those that cannot (McFarlane 2007).

When semiotics, following Roman Jakobson, tries to conceptualise adaptation as one of three practices of translation – an intersemiotic translation, where the story is transposed from one medium to another, more precisely from one sign system – or language – to another, the question of language of mediation should become one of the central issues. Yet what this kind of accounts of medium specificity reveal is the disparity between tha way adaptation studies conceptualises texts and the way it conceptualises languages. From the point of view of text we have a dynamical and dialogical culture where texts interact and influence each other in the intertextual networks, but from the point of view of language, we have a sphere of static systems with defined borders and connections between them (that have became theorized through the notion of transmediality). Languages are considered to be something given and static, subjected to narrative, existing virtually in isolation, without being influenced by anything, without evolving and developing. Yet what does it mean to say intersemioticity in next to intertextuality?

This article argues that the problem of adaptation could also be framed as a language-based issue, where medium (i.e. language) specificity functions as a more general study of expressive possibilities of a particular medium, i.e. the study of languages in semiotic sense, and transposition from one language to another offers a possibility to observe the equivalences and noncompliances between those languages – the way particular language enables or prevents certain articulations on textual level. More importantly the adaptations offer an exceptional ground for studying the dynamic interaction between languages – intermedial and intersemiotic relations on the level of languages, i.e. the way languages, when coming in contact when adapting a text, influence each other. And behind this kind of perspective must be a contceptualisation of languages as dynamic, evolving organisms that exist, come in contact with and influence each other through or sans texts, the same way that text themselves do in the sphere of culture.

The so far latent problem of semiolinguistic (for the lack of better term) influences and transpositions in adaptation was activated by recent tendency in comic book adaptation, the case extraordinaire being the adaptation of Frank Miller’s Sin City by Robert Rodriguez and Frank Miller (2005), where intersemiotic influences were not a barely noticeable hidden layer, but “spelled out” throughout the film. From semiotic point of view, thus, Sin City is not simply an attempt to adapt the story, but at the same time an attempt to adapt an aesthetics – a language as well. The outcome of this strategy could be called formalistic adaptation, where some formal parameters of the original text or medium will be transposed with (or even instead) of the story but above all, not only a story, but the language itself has been adapted to new medium. Yet this explicitness of formal dominant leads to the question whether this kind of phenomena is innovation in cinema or can there be found other examples of language adaptation on less noticeable scale.

This opens up whole new horizon of research in the field of intermedial adaptation that enables to go further then the question of media specificity and its influence on narration and in some ways reconnect the question of narrative and narration with the question of language, in consequence permitting to see the languages themselves as active participants in the process of adaptation being influenced by one another.

Thus the main aim of my paper is to seek to conceptualise the problem of adaptation from the point of view of intermedial relationships and influences on the level of language and the “essence” of film language from the point of view of these intermedial relationships it participates in and how these mechanisms might affect the cineasts or viewers competence and even performance. More broadly I try to sketch out some outcomes that these kind of processes might have on the dynamics on cultural/semiotic languages, more particularly to change and development of languages as affected by narrative transposition.

1. Formalistic adaptation and the problem of language

The adaptation of comic books is not necessarily new – we can find legendary examples of this kind of cultural exchange in 1970s and 1980s, the phenomena itself starting already in 1940s – but the extent that this particular “fad” has acquired is quite noteworthy. Comic book adaptations from classics like Batman, Superman to Spider-Man, X-Men, Hulk, Hellboy etc have become box-office hits that seem to ensure certain amount of returns.

When we take broader perspective, the inter-relationship between the language of comic and film language is rather intriguing as a systems of mediation. The birth of a comics as sequential art coincides by and large with the birth of the cinema and as cinema in the beginning, comic strips too struggled with the static and “theatrical” point of view [1]. Both consist of sequential arrangement of framed elements combined to form a narrative, thus sharing many common problems and developments, at the same time borrowing visual solutions from one another (cf. Lacassin 1972, Kunzle 1972).

Then there is the stage of découpage in cinema, containing shooting script that associates with comic art very intimately as if renewing their bond – although written screenplay is often more emphasised, in the end cinema is dominantly visual means of expression and shooting script can be as essential intermedium or transmedial text as screenplay. Thus Francis Lacassin comments on this relationship:

It is no accident that such film-makers as Federico Fellini, Alain Resnais, Chris Marker, Jacques Rivette, Jean-Luc Godard, Ado Kyrou, Claude Chabrol, Jacques Rozier, Boileau-Narcejac, Claude Lelouch, Jean-Paul Savignac, and Remo Forlani, not to mention television people, are assiduous readers of comic strips. And we know that Alfred Hitchcock and Henri-Georges Clouzot compose their entire films on paper before shooting them. (Lacassin 1972: 11).

On another level this question intercuts with the whole history of animated cartoons which are in themselves kind of hybrid language between comics and cinema that achieve a reflexive account of this hybridity in Who Framed Roger Rabbit (Robert Zemeckis, 1988) where the filmed and the drawn exist side by side in same frame.

On the other side of all these similarities there, as always, is the tendency to oppose, to develop oneself to directions unattainable by other, to find one’s identity in specificity and uniqueness. For the language of comics this might manifest itself in stretching the frame vertically or horisontally beyond the possibilities of film frame or acknowledging the “double framing” of cartoon first by sheet of paper, then by drawn frame of an individual image, resulting in endless play of compositional variations which structure tends to be much more complex that the traditional split screen in cinema. Or, realizing its potential as essentially language of “freeze frame”, creating and reconceptualising the illusion and sense of movement and forms of sound and audition. Often we even see an iconisation of verbal language by forming words out of diegetic elements.

On the other hand the film language in its mainstream form developed according to the laws of continuity where initially celebrated sound and even the movement itself needed to be transparent and unnoticeable. Cinema that conceptualized itself in the beginning as a art of montage is developed into illusion of continuous mouvement that has in its extreme forms turned into continuous chain of frames, into one global sequence shot like Alfred Hitchcock’s Rope (1948). Of course the everyday functioning of film language varies itself between slow pace of sequence shot films and fast montage of action films.

Since these two languages are closely connected, author of a comic book adaptation has always a choice whether to opt for film language in its “purest” form without any aesthetic or formal references to the original – the long history of comic book adaptation shows that it is quite easy to make one without any formal references to the original – or try to convey some aspects of the visual and formal aesthetics of the source text as well.

An adaptation of Sin City (Frank Miller, Robert Rodriguez, 2005) has became an epitome of a formalistic, i.e. language adaptation because of authors attempt to preserve the aesthetics, even the language of comics instead of using traditional film language. As Rodriguez himself explains:

And the more I looked at the book to adapt it, I realized it didn’t need adapting. It’s visual storytelling and it works so well on the page. I felt it should work exactly the same way on the screen. [..]
I didn’t want to make a movie out of ‘Sin City.’ I wanted to make movies into the comic. I wanted to turn cinema into the comic. Not take it and suddenly turn it into a regular movie[2] .

Rodriguez calls the process a “translation” instead of “adaptation”, yet in essence he does not transpose the story from one language to another but instead tries to transpose the language of comics itself - to transmute or transform it into “film language”. The outcome is not the language of comics nor film language, but the language of comics adapted to possibilities and limitations of cinema, of film language, thus being the adapted synthesis of both languages, a hybrid language.

Although Sin City is possibly one of the most complete attempt of formal adaptation, it is not the only one and we do find fragments of this kind of language adaptation also in other comic book adaptations like some attempts to adapt comic book framing into film language in The Incredible Hulk (Louis Leterrier, 2008), to “motivate” words by building them into and from diegetic objects in Wanted (Timur Bekmambetov, 2008). Even the split screen solutions in The Thomas Crown Affaire (John McTiernan, 1999) can be read as reminiscent of language of comics. Or outside comic book adaptations we have the time slice technique used in Like a Rolling Stone music video for Rolling Stones (Michel Gondry, 1995), The Matrix (Andy and Larry Wachowski, 1999) or Buffalo’66 (Vincent Gallo, 1998), playing with the stop-motion – a technique that was first developed by a photographer Tim MacMillan in 1980s in attempt to recreate photographic version of cubist art.

Thus these influences exceed the narrow problem of textual adaptation and enter into the broader field of language adaptation for stylistic purposes – the adaptation of language emerges step by step as independent creational choice next to adaptation of text itself.

2. the “essence” of language

But what is the ‘language’ – in case of film, comic book, or any other extension of the notion of language – seems in many cases to be the hardest problem to solve and often we speak instead of language about stylistic alternatives, aesthetical influences or formal choices without trying to elaborate or limit the scope of the concept, accepting the intuitive notion that we tend to have of it.

In everyday language notion has its roots, analogically with language of flowers, language of animals, language of music language of painting etc, in metaphorical use of ‘language’ as a means for communication – everything that can be used to in order to communicate can be conceiver as language. Thus on this level one can define language as a means of communication where the particular nature of these means in secondary or even unimportant and the dominant lies usually in message or meaning communicated, i.e. in narrative itself – what we need in this case is to understand what someone is trying to tell us, without being interested in the structure of particular “language”.

In film theory this initial metaphorical use developed into more particular and sometimes normative question about the expressive resources enabling this communication[3] and film theoretical writings the question of language was always a balancing act between that which is communicated and what are the means used to communicate. Thus the dominant parameter in defining language had changed from communicative function to expressive function – language is not only the “what ?” but also “how ?”. From certain perspective this definition of language was already very close to the ‘language proper’, and thus the initial metaphor of film language was developing into theoretical concept, but it still lacked consistency with regards to what counts as a substance or material of this language and any regard to the possible structure of this language.

Semiology of cinema could have been an expansion of the understanding that a study of language concerns the “how ?” both structurally and ‘substantially’, since the language, as defined by Ferdinand de Saussure is a structuring agent, “form, not a substance” (Saussure 1969: 169 ) on the level of expression as well as on the level of content. However the founder of semiology of cinema, Christian Metz, being at that time strongly influenced by the ontological conception of André Bazin, started the semiological adventure instead with the question of “what ?”, by defining the language of cinema through substance of image as “above all the literalness of a plot” (Metz 1994: 99). The problem was at the same time in preferring the “what ?” to “how ?” through the level where this language was located – in narrative, i.e. in diegetic substance, not in image, i.e. in visual substance. Thus even when early Metz asked “how ?”, this had more to do with narrative than with language.

Yet soon the importance of “how ?” was realised and Juri Lotman already started his discussions on film language by moving even further from the structuring agency of montage to structuring agency of frame itself – frame created discrete elements out of continuous profilmic reality, turns it from blind copy of life into language by creating a possibility to choose and to combine different views to the same object or simlilar views to different objects (Lotman 2004: 42).

Mihhail Lotman adds:

nobody goes to the movies to watch frames, but namely the tail of a dog. But the tail of a dog in a movie differs from a real tail of a dog by being placed in a frame, it has become an element of a frame. Hence, in a movie we are dealing not with a tail, but with a tail-in-frame. But a frame is one of the elements of film language, its structure characterizes the concrete film language. Not only what is in the frame is important, but also, as well as in the case of natural language, what is outside of it. We do not see an element, but we sense its absence. (M. Lotman 2002)

Although these kind of advancements in semiology and semiotics of cinema were more concerned with how exactly cinema creates meaning, what are its semiotic resources, the limits of film language were never quite precisely defined. The formal aspects of image, i.e. technological elements of cinema that Metz at first relegated to the secondary problem of connotation, find their way into analysis of film language, yet the question remains to which extent elements depicted inside a frame function as elements of film language.

For example Lotman defines an element of film language as any textual unit (visual, graphic or sound) that has an alternative even if only by not using itself and the use of which would not be automatic but with accompaning meaning. Its use should also be governed by some kind of regularity. (Lotman 2004: 55) Thus for example when object in diegetic world acquires some kind of generic stability, it will be part of film language.

Jan M. Peters has imported the Saussurean conception that language is a form, not a substance, into the semiotic film theory to form the basis for the theory of pictorial language most effectively. He explains: “form is not what, but how something is seen [..], picture’s form is what distinguishes a picture from other pictures of the same substance and the same object depicted” (Peters 1981: 12). Thus the essence of film language is in formal parameters of framing, angle, movement of camera, etc. You can use close-up or long shot, static or dynamic camera, but you cannot make a film (in unextended sense) without using camera, framing in a certain way etc.

Even when we define film language as a ensemble of formal (stylistic) elements and the conventions of their use, we have several dilemmas about the extent to which a diegetic world of the film should be integrated to this formal view.

The problem of limits of film language appear, for example, when trying to integrate Thomas Leitch view on possible markers of the genre of adaptation with the problem of film language. Leitch (2008: 111-113) discusses four textual markers that enable to recognize adaptations as adaptations in the traditions of Dumas père and Dumas fils. These markers or cues that encourage filmgoers to experience adaptations as adaptations, even if they know nothing of their sources are: (1) a period setting, (2) period music, (3) obsession with authors, books, and words, and (4) use of intertitles to give information about their settings, particularly if those settings are remote in time or place.

Now Metz, applying the “heterogeneous code” model of Language and Cinema (1977) would probably suggest that this is indeed a code of adaptation enabling us to interpret these text as adaptations, more precisely this would be a particular cinematographic code, i.e. a subcode of adaptation. Yet all of these markers – apart from intertitles – are nontechnical aspects that concern the image and more particularly diegesis, because even the music, although in itself used non-diegetically has its roots in diegesis (being ‘period music’). Although these markers constitute a code, this convention – as Leitch rightfully points out – can be at best textual, not “cine-linguistic”, since they do not concern aspects involved in making actual cinematic text (framing, camera angles, movements, techniques of montage etc), but only certain textual practices. Therefore the differentsiation between speech and language or textual elements and linguistic elements is essential.

Thus when trying to isolate the level of language that could participate in language adaptation, it is most functional to limit the problem of language with formal technical possibilities and more importantly formal and structural conventions of cinema – it is not a question of audiovisual possibilities of cinema as such but about the stabilised and conventionalised formal elements in it.

Analogical situation is with the language of comics where certain formal conventions can be – and are – listed as functional categories of language: the panels, the gutter, the balloon, the caption, the conventions of combining word and image, the conventional ways of framing the image and combining panels into sequence etc (Saraceni 2003, Eisner 1996).

3. adaptation of language : broadening the field

When sketching the problem of adaptation of language, the first issue is still about narrative and narration. Whether we speak about natural language, literary language, film language or of the language of comics as a formal system of discrete elements or more or less stable forms of expression, we always speak about language adapted for and by narration. This ‘primary’ adaptation is the general base that makes it possible to adapt languages or talk about possibility of adapting languages – common roots of all cultural and artistic means of expression.

Languages adapted through the process of narration have developed an ensemble of narrative techniques that makes this kind of intermedial exchange possible because these kinds of languages meet (are adapted to meet) certain requirements of narrative communication.

Thus far I have discussed the possibility of intersemiotic adaptation on the level of language through the example of comic book adaptations but it might be possible to broaden this view on all possible intermedial contacts.

An aesthetic and formal exchange between two (audio)visual languages like film language and language of comics might even seem trivial considering their common roots and development, yet these extensive exchanges that have taken place show that both languages have developed rather specific and differentiated expressive and narrative modes that, when subjected to the process of adaptation seem to produce new hybrid languages.

When it comes to explaining the mechanisms of adaptation, Leich refers to Hutcheon:

Hutcheon’s description of the distinctive mode in which adaptation is experienced as ‘a conceptual flipping back and forth between the work we know and the work we are experiencing’ suggests a more general rubric: Watching or reading an adaptation as an adaptation invites audience members to test their assumptions, not only about familiar texts but about the ideas of themselves, others, and the world those texts project against the new ideas fostered by the adaptation and the new reading strategies it encourages. On a more general level, of course, the conceptual flipping back and forth Hutcheon ascribes to readers or viewers of adaptations is the attitude prescribed by all genres, from Westerns to musicals. And on a more general level still, it is the work of all reading or viewing, since reading any book, attending any play, looking at any painting, or watching any film allows an audience to test assumptions formed by earlier experiences of books or plays or paintings or films against a new set of norms and values. The distinctiveness of adaptation as a genre is that it foregrounds this possibility and makes it more active, more exigent, more indispensable. Comparisons that are discretionary in all texts, because they are all intertexts, become foundational to the extent that any audience experiences an adaptation as an adaptation. (Leitch 2008:116-7)

And he seems to suggest that his conception of “genre markers” lessens the importance Hutcheon places on relationship between source and adaptation since – as Ken Gerder points out - Jane Campion’s film The Piano (1993) “attracted the kind of sustained analytical criticism which worked to designate it as “literary,” even though it was not actually an adaptation” (ibid. 115). But whether it is enough to reduce the problem of “literariness” to a certain ambience created by certain settings, music and intertitles or could there be some formal and aesthetic ways literary expression is adapted to film language so that the latter “feels” literary ? For example the role of description, form of dialogues, adjustments in portrayal of inner worlds – elements that can be perceived as different from non-adaptation or that can be “imitated” by non-adaptations to achieve the effect of literariness ?

The uncomfortable and “undefineable” question of language per se seems to be left out from most of the adaptation studies and even sometimes from semiotic study of intersemiotic processes in cultur.

But when we look at the history of film theory we see that film language has always struggled against the expressive potential of natural language and to prove oneself as capable and in no way lesser narrative medium it has had to develop new means of expression – but in many instances the possibilities of natural language are the examples to follow or the vision for self-enhancement for film language.

Doesn’t the demand that cineasts should be able to say “I” like novelists and poets do, voiced by Alexandre Astruc in his manifesto “The Birth of Avant-Garde: La Camera-Stylo” refer to the influence natural language has as a model-language ? What are the means cinema has developed itself according to natural language and in which way has it surpassed the possibilities of natural language ?

What about the early discussions about the (im)possibility to express through film language the linguistic categories like negation, past and future etc ? We can see the history of natural language filled with layers of vocabulary of other natural languages it has collided with in cultural space – thus we have natural language often adjusting themselves to be able to express the information particular times require. Shouldn’t we study other semiotic languages in the same manner as influencing and adapting other languages and in collision giving birth to or growing into new languages ?

Or when Edward Murray writes:

[Truman Copote] was once quoted saying: “I think most of the younger writers have learned and borrowed from the visual, structural side of movie techniques. I have”. No one can read Capote’s book without being struck by its filmic construction. (Murray 1973: 132)

What would be the laws of dynamics in this kind of cultural univers – what keeps it from desintegrating into thousands of individual languages, what kind of mechanisms are behind conventionalisation of particular expressions ?

4. instead of conclusion : semiotic polyglotism and adaptation

Juri Lotman once said that the basis of semiotics of culture lies understanding of the necessity and inevitability of semiotic polyglotism (Lotman 1990: 272) – the semiotic sphere is heterogeneous and comprised of multitude of semiotic systems (languages), all with different levels of complexity. These different systems or languages require different literacies from both film-maker and viewer, allowing at the same time immeasurable scale of innovation and creation.

Thus, one the one, hand the languages have a constant need to discover and develop their specificity and uniqueness that differentiates them from other languages of culture. On the other hand, we have a constant processes of hybridisation and creolisation, languages influencing each other. And Lotman notes that many texts are made in mixed languages but we do not notice that. Even the film languages can be viewed as a mixture of the principles and elements of the language of silent cinema and the language of sound cinema. Thus in the sphere of language, there are always these two processes – of disintegration of languages on one hand and of integration of languages on the other – that is part of cultural dynamics (Lotman 2002).

When we approach the field of adaptation and translation with extended and yet limited notion of language as semiotic resources any medium must have to be able to produce texts, adaptations emerge as a meeting point of not only stories but languages as well – sphere of their interplay, mutual influences and hybridisation. If text as transposed from one language to another adapts to new means of expression, doesn’t this text, being incompatible at first, trigger some changes or developments in recipient language as well ? What kind of literacy these intermedial and intersemiotic relations require from a creator and a reader and what kind of new competences they produce ?

The central question in this line of thought is the possibility to conceptualize the development and change of language of cinema or other languages of culture through the study of adaptation. In the same manner as we might conceptualise the impacts of translation culture and contacts between natural languages as a force of adaptation and change in ordinary language.


Corrigan, Timothy. 2007. “Literature on Screen, a history: in the gap”. – In Cartmell, Debora, Imelda Whelehan (eds.) A Cambridge Companion to Literature on Screen. pp. 29-43

Eisner, Will. 1996. Graphic Storytelling and Visual Narrative. Paramus: Poorhose Press

Kunzle, David. 1972. “Some Supplementary notes by David Kunzle”. – Film Quarterly, Vol. 26, No. 1, (Autumn, 1972), pp. 19-23

Lacassin, Francis. 1972. “The Comic Strip and Film Language“. – Film Quarterly, Vol. 26, No. 1, (Autumn, 1972), pp. 11-19

Leich, Thomas. 2008. “Adaptation, the Genre” – Adaptation Vol. 1, No. 2, pp. 106-120

Lotman, Juri. 1990. Kultuurisemiootika. (Semiotics of Culture) Tallin: Olion

                - 2004. Filmisemiootika. (Semiotics of Cinema) Tallinn: Varrak

               - 2002. “Kultuuri fenomen” (The Phenomenon of Culture) – Akadeemia 12. pp. 2644-2662

Lotman, Mihhail. 2002. “Atomistic versus holistic semiotics” – Sign System Studies, Vol. 30.2., pp. 513-527

Manovich, Lev. 1999. “Database as Symbolic Form” – Convergence 5; 80, pp. 80-99

McFarlane, Brian. 2007. “Reading film and literature” – In Cartmell, Debora, Imelda Whelehan (eds.) A Cambridge Companion to Literature on Screen. pp. 15-29

Metz, Christian. 1994. Film Language: A Smeiotics of the Cinema. The University of Chicago Press

Murray, Edward. 1973. “In Cold Blood': The Filmic Novel and the Problem of Adaptation” – Literature / Film Quaterly, Vol. 1, No. 2, pp. 132-137

Peters, Jan M. 1981. Pictorial Signs and the Language of Film. Amsterdam: Rodopi

Sabin, Roger. 2006. Comics, comix & graphic novels. London : Phaidon

Saraceni, Mario. 2003. The language of comics. London ; New York : Routledge

Saussure, Ferdinand de. 1969. Course en linguistique générale. Paris: Payot

This research was supported by the European Union through the European Regional Development Fund (Center of Excellence CECT) and has been carried out as part of the research project “Typology of
cultural autocommunication” (ETF7594)