This thesis synthezises theories from intermedia studies, semiotics, Gestalt psychology, cognitive linguistics, cognitive psychology, cognitive poetics, reader response criticism, narratology and possible worlds-theories adjusted to literary studies. The aim is to provide a transdisciplinary explanatory model of the transaction between text and reader during the reading process resulting in the reader experiencing a mental, virtual world. Departing from Mitchells statement that all media are mixed media, this thesis points to Peirce’s tricotomies of different types of signs and to the relation between representamen (sign), object and interpretant, which states that the interpretant can be developed into a more complex sign, for example from a symbolic to an iconic sign. This is explained in cognitive science by the fact that our perceptions are multimodal. We can easily connect sounds and symbolic signs to images. Our brain is highly active in finding structures and patterns, matching them with structures already stored in memory. Cognitive semantics holds that such structures and schematic mental images form the basis for our understanding of concepts. In cognitive linguistics Lakoff and Johnsons theories of conceptual metaphors show that our bodily experiences are fundamental in thought and language, and that abstract thought is concretized by a metaphorical system grounded in our bodily, spatial experiences. Cognitive science has shown that we build situation models based on what the text describes. These mental models are simultaneously influenced by the reader’s personal world knowledge and earlier experiences. Reader response-theorists emphasize the number of gaps that a text leaves to the reader to fill in, using scripts. Eye tracking research reveals that people use mental imaging both when they are re-describing a previously seen picture and when their re-description is based purely on verbal information about a picture. Mental spaces are small conceptual packets constructed as we think and talk. A story is built up by a large number of such spaces and the viewpoint and focus changes constantly. There are numerous possible combinations and relations of mental spaces. For the reader it is important to separate them as well as to connect them. Mental spaces can also be blended. In their integration network model Fauconnier and Turner describe four types of blending, where the structures of the input spaces are blended in different ways. A similar act of separation and fusion is needed dealing with different diegetic levels and focalizations, the question of who tells and who sees in the text. Ryan uses possible worlds-theories from modal logic to describe fictional worlds as both possible and parallel worlds. While fictional worlds are comparable to possible worlds if seen as mental constructions created within our actual world, they must also be treated as parallel worlds, with their own actual, reference world from which their own logic stems. As readers we must recenter ourselves into this fictional world to be able to deal with states of affairs that are logically impossible in our own actual world. The principle of minimal departure states that during our recentering, we only make the adjustments necessary due to explicit statements in the text.

L’observation de la présence des arts non verbaux au cœur des œuvres de Marguerite Duras, Lawrence Durrell, Elfriede Jelinek et Péter Nádas nous invite à considérer la musicalité et l’iconicité des écritures dramatiques et romanesques contemporaines en termes de modèle mais aussi de dispositif. Des phénomènes de dialogue, d’hybridation, de polyphonie, de dialogisme, d’intermédialité, de ce que Jacques Rancière nomme « l’impurification » au cœur d’un « régime esthétique de l’art » révèlent les rêves, désirs et pulsions du verbal pour d’autres arts, mais aussi pour des représentations à l’artisticité discutable. La fabrique d’un corps organique, sexuel, érotique voire pornographique par les écritures contemporaines nous convie à envisager le métissage entre art et non-art en termes de dispositif performatif et à proposer une lecture queer des œuvres. À l’heure du postmodernisme, le recours des écritures au non-verbal se donne à lire à la fois comme la manifestation d’une crise du logos et de la représentation et comme l’enjeu d’une rénovation esthétique et politique de la littérature. Qu’ils modélisent le verbal ou fassent brutalement irruption et déchirure en son sein, les arts non verbaux concourent au renouvellement des formes littéraires, mais aussi à leur politicité et au renouveau de la fiction. Cette étude ambitionne donc d’explorer le carrefour esthético-politique que dessinent, entre le milieu du XXe siècle et ce début de XXIe siècle, les relations plurielles entre les arts verbaux et non verbaux dans l’art verbal par excellence, la littérature. / The observation of the presence of non verbal arts within the works of Marguerite Duras, Lawrence Durrell, Elfriede Jelinek and Péter Nádas leads us to examine the musicality and the iconicity of contemporary dramatic and novelistic writings in terms of model, pattern and devices. Dialogue, hybridization, polyphony, dialogism, intermediality, and what Jacques Rancière calls “impurification” within the “Aesthetic Regime of Art”, display the dreams, desires and longings of verbal art for other arts, but also for representations whose artistic content is arguable. The fact that contemporary writings produce an organic, sexual, erotic, even pornographic body invites us to focus on the interactions between arts and non-arts with regard to their performative devices and to propose a queer reading of the works. In Postmodernism, the fact that writings draw on non verbal forms can be understood as the expression of the failure of Logos – both language and reason – and of representation. Moreover, what is also at stake is an aesthetic and political reform of literature. Whether they tend to impose new verbal models or break into them, non verbal arts contribute not only to reshape literary forms but also to emphasize their political substance and renew their fictional content. This dissertation aims to investigate the crossroads between aesthetics and politics that the various relationships between verbal and non-verbal arts display, from mid-20th century to the beginning of the 21st century, within Literature, the verbal art par excellence.

This essay aims to understand how meaning and materiality work in four picture books from the 00s and 10s; Gittan och gråvargarna by Pija Lindenbaum, I skogen by Eva Lindström, Vem ser Dim? by Maria Nilsson Thore and Vi blåste bort ibland by Viveka Sjögren. The essay uses two theoretical works about picture books: Bilderbokens pusselbitar by Maria Nikolajeva and Modernismens bilder – Den moderna bilderboken i Norden by Elina Druker. Questions to lead the analysis are: How can text and picture be understood as materiality? How can meaning be understood as a material-discursive phenomenon? How can reading and meaning be understood as performative processes? The chapter “Diffractions: Differences, Contingencies, and Entanglements That Matter” from Karen Barad’s book Meeting the Universe Halfway – Quantum Physics and the Entanglement of Matter and Meaning provides help to understand diffraction as a metaphor and methodological attitude. Eve Kosofsky Sedgwick’s understanding of knowledge as performative in the book Touching Feeling – Affect, Pedagogy, Performativity is used to understand the picture book as an act of knowledge and meaning, and reading as a performative process where human involvement cannot be overlooked. The essay problematizes the theoretical texts about picture books by taking on different parts of the picture book’s form and drawing examples from the four chosen picture books. Parts that are examined are divisions of signs in semiotics (and in the picture book theory), the picture book’s limits between for instance framework and content, fiction and reality, and book and reader, mimesis and the analytical problems with looking for truth, the impact of the picture book’s size and shape, and lastly illustration techniques and the influence that illustrator, materiality and reader have on the book. The essay shows how different involvements affect the meaning that is created in the picture book, how solid delimitations of different kinds can be problematic, and how words such as picture, text, paratext and content should be treated gently and consciously. Different ways of breaking with methods and thoughts of habit are presented, for instance by using Barad’s concept of diffraction. The essay ends by emphasizing the complexity that follows meaning, materiality and reading, the responsibility it demands of us and how picture book theory sometimes simplifies analysis. This essay stresses the importance of questioning how the methods and words we use affect what is possible to understand, but not in the sense that we should read in a paranoid and closed way, but rather open minded and with room for failures, reconsiderations and other ways forward. / Syftet med denna uppsats är att förstå hur mening och materialitet hänger ihop i fyra bilderböcker från 00- och 10-tal; Gittan och gråvargarna av Pija Lindenbaum, I skogen av Eva Lindström, Vem ser Dim? av Maria Nilsson Thore och Vi blåste bort ibland av Viveka Sjögren. Uppsatsen går i dialog med de bilderboksteoretiska verken Bilderbokens pusselbitar av Maria Nikolajeva och Modernismens bilder – Den moderna bilderboken i Norden av Elina Druker. Frågor för att leda undersökningen är som följer: På vilket sätt är text och bild materialitet? Hur kan mening förstås som materiellt-diskursivt fenomen i bilderböckerna? Hur kan läsning och mening förstås som performativa processer? Avsnittet ”Diffractions: Differences, Contingencies, and Entanglements That Matter” ur Karen Barads verk Meeting the Universe Halfway – Quantum Physics and the Entanglement of Matter and Meaning fungerar som hjälp för att förstå diffraktion som metafor och metodologisk hållning. Eve Kosofsky Sedgwicks förståelse av kunskap som performativ i verket Touching Feeling – Affect, Pedagogy, Performativity används för att förstå bilderboken som ett görande, och läsningen som en performativ process där mänsklig inblandning inte kan förbises. Uppsatsen problematiserar de bilderboksteoretiska verken genom att ta sig an olika aspekter av bilderbokens form samt exemplifiera detta med hjälp av det skönlitterära materialet. Områden som vidrörs är semiotikens (och bilderboksteoretikernas) indelning av tecken, bilderbokens gränser mellan bland annat ramar och innehåll, fiktion och verklighet samt verk och läsare, mimesis och sanningssökandets analytiska problem, bilderbokens påverkan av storlek och format, samt slutligen illustrationstekniker och den inverkan illustratör, materialitet och läsare har på verket. Uppsatsen visar på hur olika inblandningar påverkar den mening som skapas i bilderboken, hur fasta gränsdragningar av olika slag kan anses problematiska, samt hur begrepp som till exempel bild, text, paratext och innehåll bör behandlas varsamt och medvetet. Olika sätt att komma runt upptrampade metodologiska tillvägagångssätt och tankespår presenteras, bland annat med hjälp av Barads diffraktionsbegrepp. Uppsatsen avslutas med att betona den komplexitet som följer med mening, materialitet och läsning, det ansvar detta avkräver oss och hur bilderboksteorin ibland fungerar förenklande i analyser. Denna uppsats lyfter vikten av att ifrågasätta vad de metoder och ord vi använder egentligen gör med vad som är möjligt att förstå, men inte i den mening att vi bör läsa misstänksamt och stängt, utan snarare öppensinnat och med utrymme för misslyckanden, omprövningar och andra vägar vidare.