Thesis advisor: Alan Richardson / The purpose of this project is to examine how text and image function cognitively in the graphic narrative paying particular attention to its manifestation in Satrapi’s "Persepolis." The structure is such that each chapter will progressively become more specific, from medium to genre to text, drawing on specific examples from Persepolis for support. Chapter 2 will first categorize the various relationships between text and image as they function in the graphic narrative as a medium. They are described in terms of the reader’s cognitive experience of the verbal narrative line’s juxtaposition against the visual narrative line. Chapter 3 will examine how a multimodal narrative—a dual verbal/visual narrative—affects the genre of nonfiction in the graphic narrative medium. It will define not only the tensions that text and image necessarily bring to the authenticity of nonfiction, but also the benefits. Chapter 4 will focus on Persepolis as a cognitive product in its own right. Stemming from theories of autobiography which suggest that an autobiographic text is a self-construction or self-understanding of identity, one can examine Persepolis as a material product and personal construction through this lens. I offer a cognitive approach which suggests that Persepolis functions as a material anchor of a conceptual blend—cognitive theories developed by Mark Turner, Gilles Fauconnier, and Edward Hutchins which are further explained in Chapters 2 and 4. While the primary goal of this project is specific to the research goals explained above, the secondary goal is advocacy. Both cognitive literary theory and comics criticism are marginalized in current literary studies. The former—whose scientific method looms over literature—threatens to overshadow the beauty and philosophy behind prose and poetry, while the latter—as a product of mass consumption and popular culture—threatens to undermine the legitimacy of literature. However, this project will show the viability of both cognitive literary theory as a method and the graphic narrative as a subject for serious academic inquiry. / Thesis (BA) — Boston College, 2010. / Submitted to: Boston College. College of Arts and Sciences. / Discipline: College Honors Program. / Discipline: English Honors Program. / Discipline: English.

abstract: Multimodal Representation Learning is a multi-disciplinary research field which aims to integrate information from multiple communicative modalities in a meaningful manner to help solve some downstream task. These modalities can be visual, acoustic, linguistic, haptic etc. The interpretation of ’meaningful integration of information from different modalities’ remains modality and task dependent. The downstream task can range from understanding one modality in the presence of information from other modalities, to that of translating input from one modality to another. In this thesis the utility of multimodal representation learning for understanding one modality vis-à-vis Image Understanding for Visual Reasoning given corresponding information in other modalities, as well as translating from one modality to the other, specifically, Text to Image Translation was investigated. Visual Reasoning has been an active area of research in computer vision. It encompasses advanced image processing and artificial intelligence techniques to locate, characterize and recognize objects, regions and their attributes in the image in order to comprehend the image itself. One way of building a visual reasoning system is to ask the system to answer questions about the image that requires attribute identification, counting, comparison, multi-step attention, and reasoning. An intelligent system is thought to have a proper grasp of the image if it can answer said questions correctly and provide a valid reasoning for the given answers. In this work how a system can be built by learning a multimodal representation between the stated image and the questions was investigated. Also, how background knowledge, specifically scene-graph information, if available, can be incorporated into existing image understanding models was demonstrated. Multimodal learning provides an intuitive way of learning a joint representation between different modalities. Such a joint representation can be used to translate from one modality to the other. It also gives way to learning a shared representation between these varied modalities and allows to provide meaning to what this shared representation should capture. In this work, using the surrogate task of text to image translation, neural network based architectures to learn a shared representation between these two modalities was investigated. Also, the ability that such a shared representation is capable of capturing parts of different modalities that are equivalent in some sense is proposed. Specifically, given an image and a semantic description of certain objects present in the image, a shared representation between the text and the image modality capable of capturing parts of the image being mentioned in the text was demonstrated. Such a capability was showcased on a publicly available dataset. / Dissertation/Thesis / Masters Thesis Computer Engineering 2018

My work is a subjective investigation of a constant transit through the real and symbolic frontiers that create memories. By considering the different cultural and social dimensions of my life as a Latino, I am interested in the relationship with a time that is vital, personal but also shared. Through my research, a constant interaction with the materiality of the print process informs the creation of strategies of auto-recognition that are both personal and collective and introduces me to the acceptance of the affective world, thus of my body and my own temporality. The print represents fragmentary traces of an incomplete cartography, that is at the same time intermittent and selective. They are subjective approaches of a personal narrative that is at the same time collective. This visual narrative reflects on my forced displacement and cultural periphery conscience and the unconscious desire for an unmapped world.

The gokan is a medium of pre-modern Japanese literature where the story is told through a mixture of text and image on every page, with the narrative and dialogue of the story surrounding the image illustrations on all sides. The Nise Murasaki inaka Genji (A Rustic Genji by a Fake Murasaki, 1829-42), written by Ryutei Tanehiko (1783-1842) and illustrated by Utagawa Kunisada (1786-1865), was one of the most popular gokan of the Edo Period (1605-1868) and has been republished approximately twenty-four times from the end of the nineteenth century until today. By examining how these works handle text and image, this thesis seeks to gain a deeper understanding about how image functions in the gokan genre. When text must be translated and the image altered from the original layout to make it comprehensible to modern readers, the resulting text and image combinations, or lack of image, offer insight into the importance of the role of image in gokan.

La présente étude a pour objet d’examiner les écrits d’André Breton sur l’art et d’éclairer l’importance des recherches opérées sur l’image dans les principales questions abordées par l’écrivain entre les premières années de sa carrière et la fin de la seconde guerre mondiale. Si le surréalisme a commencé comme un mouvement littéraire, il va de soi que les arts plastiques n’ont pas tardé à y occuper une place essentielle. De toute évidence, les succès du surréalisme en matière d’art plastique sont encouragés par la passion vive et constante de Breton lui-même en ce domaine. Les œuvres d’art ont souvent été pour Breton autant d’occasions d’écriture. Une part de ces textes est généralement considérée comme relevant de la critique d’art. Cependant les réflexions de l’écrivain sur l’art ne se limitent pas au genre critique. Certains essais et les récits autobiographiques rendent compte d’événements étonnants suscités par les œuvres d’art. La question artistique peut en outre prendre la place principale dans les textes argumentatifs et théoriques.Les écrits de Breton peuvent être situés dans la grande filiation de la modernité poétique, de Baudelaire à des contemporains comme Jacques Dupin ou Bernard Noël. Ces textes mettent en jeu le rapprochement et la concurrence entre poésie et peinture. Le propos de l’écrivain ne vise pourtant pas à accorder la supériorité à l’expression verbale plutôt qu’à la représentation visuelle ou vice-versa, mais à faire remarquer l’entrelacement du langage et de l’image. Breton s’efforce en outre de mettre à profit l’entrecroisement hiérographique de l’écriture et de l’image. Loin de chercher à dégager une conception cohérente de l’écrivain concernant les arts plastiques, notre analyse s’efforce de montrer comment les images visuelles ont au contraire pour objet de déstabiliser l’écriture de Breton et de la remettre en cause. Dans la première partie, nous nous pencherons sur les réflexions de Breton sur l’art au cours des années 1920, dont le développement traverse les frontières génériques des textes. Dans la partie suivante, nous constaterons qu’une place plus importante est accordée dans les années 1930 aux recherches sur les représentations visuelles, qui concernent alors directement les préoccupations principales de l’écrivain : le problème porte sur le désir inconscient, sa révélation et sa réalisation. Tandis que les tableaux sont susceptibles de mettre en place un nœud entre réalité et rêverie, l’ « objet surréaliste » consiste à déplacer sans cesse la frontière de ceux-ci. La dernière partie sera consacrée à l’apport des représentations visuelles dans la conception du « mythe surréaliste », qui occupe la place principale dans les préoccupations du groupe aux cours des années 1940. Le recours aux produits de l’imaginaire collectif n’empêche pas de profiter du caractère polyphonique des images pour en révéler le caractère mystificateur. En suivant ainsi l’évolution des recherches de Breton sur l’art plastique dans l’ordre chronologique, nos analyses voudront montrer que l’activité de Breton autour de l’art visuel entretenaient un rapport étroit avec l’écriture. L’intérêt conjoint pour les arts plastiques et pour la poésie n’aboutit pas à proclamer la suprématie de l’une sur les autres, encore moins à prétendre à la fusion des deux domaines. Inséparable de l’écriture, mais toujours en dehors ou plutôt en marge de celle-ci, les représentations visuelles ont pour effet de remettre les écrits en question en perturbant la lecture. Or, un tel décalage suppose incontestablement la participation du lecteur-spectateur qui, dans son embarras devant cette double incohérence, est convié à se mêler au jeu des textes et des images. L’écrivain s’efforce ainsi d’inviter le lecteur-spectateur à l’intérieur des champs surréalistes, en lui proposant de vivre la même expérience que lui. / The purpose of this study is to examine the writings of Andre Breton on the arts between the first years of his career and the end of The Second World War. If Surrealism began as a movement of literature, it goes without saying that arts have occupied an essential place in this group. Obviously, the success of the surrealism in the field of art is encouraged by the passion of Breton in this area. The art works inspired Breton to write many texts. A part of these texts is generally regarded as “art critic”. However the reflections of the writer on the art is not confined to this genre. Some essays and autobiographical stories tell episodes created by the art works. In addition, the artistic question may take the main place in the argumentative and theoretical texts.The writings of Breton may be located in the filiation of the modern poetry, from Baudelaire to contemporary poets as Jacques Dupin or Bernard Noël. These texts discuss the competitive relation between poetry and painting. The texts of Breton don’t try to give the superiority to the verbal expression rather than to the visual representation or vice versa, but to point out the interlacing of language and image. Far from seeking to find a coherent concept of the writer concerning the field of art, our analysis tries to show how the visual images have influence on the writing of Breton. In the first part, we will focus on the reflections of Breton on the art in the 1920s, the development of which crossed the borders of genres of texts. In the next part, we will find that a more important place is given in the 1930s to research on the visual representations, which then relate directly the main concerns of the writer : unconscious desire, its revelation and realization. The last part will be devoted to the contribution of visual representations in the “myth of Surrealism”, which occupied the important place in the concerns of the group during the 1940s. Following the evolution of Breton’s reflections on the plastic art in chronological order, our analysis will show that the activity of Breton around the visual art had a close relationship with his writing. The interest both for the plastic arts and for the poetry does not lead to proclaim the supremacy of one over the other, still less to pretend to the fusion of the two areas. Inseparable from the writing, but always outside or rather at the margin of the wrigting, the visual representations have the effect of putting the writings in question. However, such a discrepancy between texts and images implies the participation of the reader-spectator who, is invited to magnetic field of the Surrealism.

Developing intelligent systems that can recognize and express human affects is essential to bridge the gap between human and artificial intelligence. This thesis explores the creative and emotional frontiers of artificial intelligence. Specifically, in this thesis, we investigate the relation between the affective impact of visual stimuli and natural language by collecting and analyzing a new dataset called ArtEmis. Furthermore, capitalizing on this dataset, we demonstrate affective AI models that can emotionally talk about artwork and generate them given their affective descriptions. In text-to-image generation task, we present HyperCGAN: a conceptually simple and general approach for text-to-image synthesis that uses hypernetworks to condition a GAN model on text. In our setting, the generator and the discriminator weights are controlled by their corresponding hypernetworks, which modulate weight parameters based on the provided text query. We explore different mechanisms to modulate the layers depending on the underlying architecture of a target network and the structure of the conditioning variable.

Ce travail met en regard les recueils emblématiques de Henry Hawkins (The Devout Hart, 1633), Christopher Harvey (The School of the Heart, 1647) et John Hall (Emblems with Elegant Figures, 1648). Il confronte ainsi des auteurs de confessions différentes dans l’Angleterre de Charles Ier, où la question religieuse continue d’alimenter des conflits et des tensions. Pourtant, ces emblémistes s’intéressent à un motif commun, celui du cœur, envisagé comme une métonymie de l’homme dans son rapport à la transcendance. En replaçant ce discours cordial dans le contexte historique, culturel et idéologique dans lequel il émerge, notre travail tente de montrer les convergences d’auteurs aux intérêts a priori discordants. Surtout, il s’intéresse à la façon dont textes et gravures dialoguent dans ces recueils pour articuler un discours affectif puissant, capable d’emporter l’empathie et l’adhésion du lecteur/spectateur. Le livre d’emblèmes devient ainsi le support d’un exercice méditatif à travers lequel le dévot peut espérer retrouver le chemin de son âme et, peut-être, la ramener à Dieu. / This thesis brings together three emblem books : Henry Hawkins’ The Devout Hart (1633), Christopher Harvey’s The School of the Heart (1647) and John Hall’s Emblems with Elegant Figures (1648). By doing so, it confronts three writers who display denominational differences in King Charles I’s England, where the issue of religion still fuels intense debates and conflicts. Yet those emblematists share an interest in the motif of the heart, which they use as a metonymy for man in his relationship to transcendence. By replacing the discourse of the heart in the historical, cultural and ideological context in which it has emerged, this thesis attempts to show how three authors with presumably conflicting views end up converging. Moreover, it shows how engravings and texts interact to articulate a powerful affective discourse, which can engage both the reader/spectator’s sympathy and adherence. The emblem book thus works as the support of a meditative exercise through which the devout can hope to find again the way to his soul and bring it back to God.

Catholics is an artist's book, a limited-edition memoir that makes use of text, image, and tactility. It relates the author's Catholic upbringing as it interweaves several themes: Church history, pre-Christian mythology, and the places where such spiritualities resonate with twentieth-century pop culture.

Data collecting and preparing is generally considered a crucial process in data science projects. Especially for image data, adding semantic attributes when preparing image data provides much more insights for data scientists. In this project, we aim to implement a general-purpose central image data repository that allows image researchers to collect data with semantic properties as well as data query. One of our researchers has come up with the specific challenge of collecting images with weight data of infants in least developed countries with limited internet access. The rationale is to predict infant weights based on image data by applying Machine Learning techniques. To address the data collecting issue, I implemented a mobile application which features online and offline image and annotation upload and a web application which features image query functionality. This work is derived and partly decoupled from the previous project – ImageSfERe (Image Sharing for Epilepsy Research), which is a web-based platform to collect and share epilepsy patient imaging.

License Plate Recognition (LPR) systems are frequently utilized in various access controls and security applications. In this thesis, an experimental constraint based real-time License Plate Recognition system is designed, and implemented in Java platform. Many of the available constraint based methods worked under strict restrictions such as plate color, fixed illumination and designated routes, whereas, only the license plate geometry and format constraints are used in this developed system. These constraints are built on top of the current Turkish license plate regulations. The plate localization algorithm is based on vertical edge features where constraints are used to filter out non-text regions. Vertical and horizontal projections are used for character segmentation and Multi Layered Perceptron (MLP) based Optical Character Recognition (OCR) module has been implemented for character identification. The extracted license plate characters are validated against possible license plate formats during the recognition process. The system is tested both with Turkish and foreign license plate images including various plate orientation, image quality and size. An accuracy of 92% is achieved for license plate localization and %88 for character segmentation and recognition.