La bulle bleue d’un message texte, la fenêtre d’un navigateur web, les lignes de programmation dans un terminal informatique, les diagrammes affichés sur les écrans de contrôle d’un vaisseau spatial : de 2001 à 2020, ces images d’interfaces s’insinuent partout dans les films. En informatique, l’interface utilisateur est l’environnement virtuel utile à l’usager pour interagir avec la machine. Au cinéma, la représentation de l'interface est le moyen de figurer cette interaction tout en empruntant immédiatement aux images de la machine. Si les interfaces sont de plus en plus fréquentes au cinéma, leur représentation est néanmoins très normée, obéissant le plus souvent très fidèlement aux conventions des genres cinématographiques qui l’ont modelée. Pourtant, en dressant leur inventaire en quatre grands types et en observant quelques cas d’exceptions, leur représentation révèle une grammaire cinématographique dotée de moyens narratifs, de trucages filmiques et d’effets de montage nouveaux qui restent à définir et dont on mesurera la valeur ajoutée. / The blue bubble of a text message, the window of a web browser, the programming lines in a computer terminal, the diagrams displayed on the control screens of a spaceship: from 2001 to 2020, these images of interfaces are everywhere in the movies. In computer science, the user interface is the virtual environment useful to the user to interact with the machine. In cinema, the representation of the interface is the way to represent this interaction while immediately borrowing from the machine's images. If the interfaces are more and more frequent in the cinema, their representation is nevertheless very standardized, obeying most often very faithfully the conventions of the cinematographic genres that have shaped it. However, by drawing up an inventory of four main types and observing a few exceptions, their representation reveals a cinematographic grammar endowed with new narrative means, filmic tricks and editing effects that remain to be defined and whose added value will be measured.

Generative Adversarial Networks (GANs) are a very successful method for high-quality image synthesis and are a powerful tool to generate realistic images by learning their visual properties from a dataset of exemplars. However, the controllability of the generator output still poses many challenges. We propose several methods for achieving larger and/or higher visual quality in GAN outputs by combining latent space manipulations with image compositing operations: (1) GANs are inherently suitable for small-scale texture synthesis due to the generator’s capability to learn image properties of a limited domain such as the properties of a specific texture type at a desired level of detail. A rich variety of suitable texture tiles can be synthesized from the trained generator. Due to the convolutional nature of GANs, we can achieve largescale texture synthesis by tiling intermediate latent blocks, allowing the generation of (almost) arbitrarily large texture images that are seamlessly merged. (2) We notice that generators trained on heterogeneous data perform worse than specialized GANs, and we demonstrate that we can optimize multiple independently trained generators in such a way that a specialized network can fill in high-quality details for specific image regions, or insets, of a lower-quality canvas generator. Multiple generators can collaborate to improve the visual output quality and through careful optimization, seamless transitions between different generators can be achieved. (3) GANs can also be used to semantically edit facial images and videos, with novel 3D GANs even allowing for camera changes, enabling unseen views of the target. However, the GAN output must be merged with the surrounding image or video in a spatially and temporally consistent way, which we demonstrate in our method.

<p dir="ltr">3D effect synthesis is crucial in image editing to enhance realism or visual appeal. Unlike classical graphics rendering, which relies on complete 3D geometries, 3D effect synthesis in im- age editing operates solely with 2D images as inputs. This shift presents significant challenges, primarily addressed by data-driven methods that learn to synthesize 3D effects in an end-to-end manner. However, these methods face limitations in the diversity of 3D effects they can produce and lack user control. For instance, existing shadow generation networks are restricted to produc- ing hard shadows without offering any user input for customization.</p><p dir="ltr">In this dissertation, we tackle the research question: <i>how can we synthesize controllable and realistic 3D effects in image editing when only 2D information is available? </i>Our investigation leads to four contributions. First, we introduce a neural network designed to create realistic soft shadows from an image cutout and a user-specified environmental light map. This approach is the first attempt in utilizing neural network for realistic soft shadow rendering in real-time. Second, we develop a novel 2.5D representation Pixel Height, tailored for the nuances of image editing. This representation not only forms the foundation of a new soft shadow rendering pipeline that provides intuitive user control, but also generalizes the soft shadow receivers to be general shadow receivers. Third, we present the mathematical relationship between the Pixel Height representation and 3D space. This connection facilitates the reconstruction of normals or depth from 2D scenes, broadening the scope for synthesizing comprehensive 3D lighting effects such as reflections and refractions. A 3D-aware buffer channels are also proposed to improve the synthesized soft shadow quality. Lastly, we introduce Dr.Bokeh, a differentiable bokeh renderer that extends traditional bokeh effect algorithms with better occlusion modeling to correct flaws existed in existing methods. With the more precise lens modeling, we show that Dr.Bokeh not only achieves the state-of-the-art bokeh rendering quality, but also pushes the boundary of depth-from-defocus problem.</p><p dir="ltr">Our work in controllable 3D effect synthesis represents a pioneering effort in image editing, laying the groundwork for future lighting effect synthesis in various image editing applications. Moreover, the improvements to filtering-based bokeh rendering could significantly enhance com- mercial products, such as the portrait mode feature on smartphones.</p>

L'objectif de cette thèse est de faciliter le diagnostic du cancer de la vessie. Durant une cystoscopie, un endoscope est introduit dans la vessie pour explorer la paroi interne de l'organe qui est visualisée sur un écran. Cependant, le faible champ de vue de l'instrument complique le diagnostic et le suivi des lésions. Cette thèse présente des algorithmes pour la création de cartes bi- et tridimensionnelles à large champ de vue à partir de vidéo-séquences cystoscopiques. En utilisant les avancées récentes dans le domaine de la minimisation d'énergies discrètes, nous proposons des fonctions coût indépendantes des transformations géométriques requises pour recaler de façon robuste et précise des paires d'images avec un faible recouvrement spatial. Ces transformations sont requises pour construire des cartes lorsque des trajectoires d'images se croisent ou se superposent. Nos algorithmes détectent automatiquement de telles trajectoires et réalisent une correction globale de la position des images dans la carte. Finalement, un algorithme de minimisation d'énergie compense les faibles discontinuités de textures restantes et atténue les fortes variations d'illuminations de la scène. Ainsi, les cartes texturées sont uniquement construites avec les meilleures informations (couleurs et textures) pouvant être extraites des données redondantes des vidéo-séquences. Les algorithmes sont évalués quantitativement et qualitativement avec des fantômes réalistes et des données cliniques. Ces tests mettent en lumière la robustesse et la précision de nos algorithmes. La cohérence visuelle des cartes obtenues dépassent celles des méthodes de cartographie de la vessie de la littérature / The aim of this thesis is to facilitate bladder cancer diagnosis. The reference clinical examination is cystoscopy, where an endoscope, inserted into the bladder, allows to visually explore the organ's internal walls on a monitor. The main restriction is the small field of view (FOV) of the instrument, which complicates lesion diagnosis, follow-up and treatment traceability.In this thesis, we propose robust and accurate algorithms to create two- and three-dimensional large FOV maps from cystoscopic video-sequences. Based on recent advances in the field of discrete energy minimization, we propose transformation-invariant cost functions, which allow to robustly register image pairs, related by large viewpoint changes, with sub-pixel accuracy. The transformations linking such image pairs, which current state-of-the-art bladder image registration techniques are unable to robustly estimate, are required to construct maps with several overlapping image trajectories. We detect such overlapping trajectories automatically and perform non-linear global map correction. Finally, the proposed energy minimization based map compositing algorithm compensates small texture misalignments and attenuates strong exposure differences. The obtained textured maps are composed by a maximum of information/quality available from the redundant data of the video-sequence. We evaluate the proposed methods both quantitatively and qualitatively on realistic phantom and clinical data sets. The results demonstrate the robustness of the algorithms, and the obtained maps outperform state-of-the-art approaches in registration accuracy and global map coherence

L'objectif de cette thèse est de tirer parti d'une illumination totalement ou partiellement contrôlée pour enrichir l'acquisition vidéo de contenus tel que la reconstruction de la forme et de l'apparence. Aujourd'hui de nombreux travaux ont tenté d'atteindre cet objectif. Certains utilisent une illumination contrôlée et séquentielle pour obtenir des reconstructions de haute qualité de la forme et de la réflectance. En revanche, ces méthodes requièrent des dispositifs coûteuses et/ou ne fonctionnent pas en temps réel. Dans cette thèse, nous visions un système d'acquisition à bas coût, rapide et mobile, qui se veut non-seulement le moins intrusif possible mais aussi simple d'utilisation. La première contribution présentée dans cette thèse est une application de la méthode bien connue, intitulée stéréo photométrie, à la vidéo. De plus, comme une fréquence de trame élevée est nécessaire à une telle application, nous proposons une méthode permettant l'utilisation d'une illumination séquentielle avec des caméras rapides de type "electronic rolling shutter". Malgré les résultats intéressants obtenus, la qualité des reconstructions de l'apparence et de la forme n'étaient pas à la hauteur de nos espérances. De plus, la stéréo photométrie est une méthode qui, de nature, n'est pas très adaptée aux applications visées dans cette thèse. Pour notre seconde contribution, nous proposons une méthode de reconstruction de la forme (géométrie) ainsi que de la réflectance diffuse à partir d'une image (d'une séquence) en utilisant un système de capture hybride composé d'un capteur de profondeur (Kinect), d'une caméra grand public et d'un flash. L'objectif est de montrer qu'en combinant une acquisition RGB-D (image couleur + profondeur) avec illumination séquentielle, on peut obtenir une reconstruction qualitative de la forme et de la réflectance d'une scène dans le cas où l'éclairage n'est pas connu. Un couple d'images est capturé : une image non flashée (image sous une illumination ambiante) et une image flashée. Une image dont l'illumination ne provient que du flash (image flash pure) peut être calculée en soustrayant l'image non flashée de l'image flashée. Nous proposons un nouvel algorithme temps réel, qui, basé sur un modèle local d'illumination de notre flash et de l'image flash pure, améliore l'information de forme fournie par le capteur de profondeur tout en retrouvant les informations de réflectance diffuse. Notre dernière contribution concerne la composition automatique d'éclairage. L'éclairage est un élément clé de la photographie. Les professionnels travaillent régulièrement avec des systèmes d'éclairage complexes afin de capturer directement des images esthétiques. Récemment, certains photographes ont tenté une nouvelle approche : plutôt que photographier une scène directement sous un éclairage complexe, ils capturent la scène sous plusieurs éclairages simples, permettant ainsi un post-traitement permettant combiner les différentes illuminations de la scène. Cette approche apporte une nouvelle dimensionnalité intéressante au post-traitement. Cependant la combinaison des images requiert des compétences en matière de photographie, et l'acquisition sous différentes conditions d'éclairage n'en est pas moins fastidieuse. Nous proposons une méthode totalement automatisée, qui, à partir d'un modèle 3D (forme et albedo) reconstruit à partir de capture d'une scène réelle, produit virtuellement les images correspondant aux différentes conditions d'éclairages. Ensuite, ces images sont combinées automatiquement, à l'aide d'un algorithme génétique, pour correspondre à un style d'éclairage fourni par l'utilisateur sous forme d'une image cible de son choix. / The objective of this thesis is to take advantage of controlled illumination to enrich a video acquisition with shape and reflectance reconstructions. Today, a lot of works have tried to meet this objective. Some of them take advantage of sequential controlled illumintation to recover high quality shape and reflectance, however they either require a costly and very cumbersome fixed setup, and/or do not run in real-time. Our aim is a low cost, fast, mobile and simple acquisition setup which has to be the less intrusive possible so as to provide a greater ease of use. The first contribution of this thesis focuses on the application of the well known photometric stereo method to a video acquisition. Moreover, as a high frame rate is required by such an application, a method using sequential illumination with high frame rate cameras (electronic rolling shutter cameras) is also considered. Despite the interesting results provided by photometric stereo, we found that this latter did not provide enough qualitative results. Moreover, by its nature, photometric stereo is not really suitable for the range of applications targeted. We propose, as a second contribution, a method for recovering the shape (geometry) and the diffuse reflectance from an image (or video) using a hybrid setup consisting of a depth sensor (Kinect), a consumer camera and a partially controlled illumination (using a flash). The objective is to show how combining RGB-D acquisition with a sequential illumination is useful for shape and reflectance recovery. A pair of two images are captured : one non flashed (image under ambient illumination) and a flashed one. A pure flash image is computed by subtracting the non flashed image from the flashed image. We propose a novel and near real-time algorithm, based on a local illumination model of our flash and the pure flash image, to enhance geometry (from the noisy depth map) and recover reflectance information. Finally, our last contribution concerns an automatic method for light compositing, using rendered images. Lighting is a key element in photography. Professional photographers often work with complex lighting setups to directly capture an image close to the targeted one. Some photographers reversed this traditional workflow. Indeed, they capture the scene under several lighting conditions, then combine the captured images to get the expected one. Acquiring such a set of images is a tedious task and combining them requires some skill in photography. We propose a fully automatic method, that renders, based on a 3D reconstructed model (shape and albedo), a set of images corresponding to several lighting conditions. The resulting images are combined using a genetic optimization algorithm to match the desired lighting provided by the user as an image.

No description available.

{This thesis deals with fire detection in video by colour analysis and machine learning, specifically deep convolutional neural networks, using Caffe framework. The aim is to create a vast set of data that could be used as the base element of machine learning detection and create a detector usable in real application. For the purposes of the project a set of tools for fire sequences creation, their segmentation and automatic labeling is proposed and created together with a large test set of short sequences with artificial modelled fire.

Text in English, with abstracts and keywords in English, Zulu and Xhosa / Link to the dataset (catalogue): https://doi.org/10.25399/UnisaData.14101913.v1 / Digital image editing is rooted in the analog practices of photographic retouching from the late nineteenth century. This study interrogated how novel contributions of new media practice can inform understanding of the relationship between digital and analog media. The study also sought to explore new conceptual avenues in the creation of digital art that incorporates key aspects of both new and traditional media. This study employed a literature review of selected discourses related to new media studies. Specifically, the work of scholars Lev Manovich, Jay David Bolter, Richard Grusin, and Filipe Pais on the interplay between traditional and new media formed the cornerstone of the analysis. These discourses contextualise an analysis of several contemporary case studies of digital artists, with a particular focus on John Craig Freeman and the Oddviz collective. These works were selected for the way in which they destabilise conventional notions of digital photography in new media and the way digital content can be ‘displaced’ into a physical space. From this analysis several concepts arise that serve as distinguishing markers for media displacement. These themes include embodiment, memory, identity formation, autotopography, and intermediality. The dissertation concludes with an overview of my work that incorporates the concepts derived from my analysis of the case studies. It discusses how my exhibition Digital Tourist, a mixed media installation, makes use of photogrammetry and AR to displace the private connections of an individual life into the public space of the gallery. / Ukuhlelwa kwezithombe zezindaba zedijithali kususelwe emikhubeni ye-analokhu yokuthwebula kabusha izithombe kusukela ngasekupheleni kwekhulu leshumi nesishiyagalolunye leminyaka.. Lolu cwaningo luphenye ukuthi iminikelo yenoveli emisha yokwenziwa kwezezindaba ezintsha zingakwazisa kanjani ukuqonda kobudlelwano phakathi kwezindaba zedijithali ne-analokhu. Ucwaningo luphinde lwafuna ukubheka izindlela ezintsha zomqondo ekwakhiweni kobuciko bedijithali obufaka izinndaba ezibalulekile kokubili kwezokuxhumana nezendabuko ezintsha. Izinkulumo ezikhethiwe ezihlobene nezifundo zezindaba ezintsha zibuyekeziwe. Ngokuqondile, umsebenzi wezazi uLev Manovich, Jay David Bolter, Richard Grusin noFilipe Pais ekusebenzisaneni phakathi kwabezindaba bendabuko nabasha kwakha okuzobhekwa ngqo uma kuhlaziywa. Lezi zinkulumo zigxila ekuhlaziyweni kwezifundo zamanje zamaciko edijithali, kugxilwe kakhulu kuJohn Craig Freeman kanye neqoqo le-Oddviz. Le misebenzi yakhethwa ngendlela yokuthi ingazinzisi imiqondo ejwayelekile yokuthwebula izithombe zedijithali emithonjeni emisha kanye nokuthi okuqukethwe kwedijithali "kungahanjiswa kanjani" endaweni ebonakalayo. Ukusuka kulokhu kuhlaziywa kuvela imiqondo eminingana esebenza njengezimpawu ezihlukanisayo zokufuduswa kwabezindaba. Lezi zingqikithi zifaka phakathi ukwakheka, inkumbulo, ukwakheka kobunikazi, ukuziphendulela kanye nokuzibandakanya. Idezetheyishini iphetha ngokubuka konke ngomsebenzi wami ohlanganisa imiqondo esuselwe ekuhlaziyweni kwami kwezifundo zocwaningo. Ingxoxo ihlanganisa ukuthi umbukiso wami we-Zivakashi zeDijithali, ukufakwa kwabezindaba okuxubile, isebenzisa uhlelo lokuthwebula olusebenzisa ulimi noma ifothogrametri ne-AR ukukhipha ukuxhumana kwangasese kwempilo yomuntu ngamunye endaweni yomphakathi yegalari. / Ukuhlela imifanekiso yedijithali yinkqubo eyendeleyo, nowaqalwa kwiminyaka yokugqibela yenkulungwane yeshumi elinethoba, kwimisebenzi yezifaniso/yeanalogu ekuhlaziyweni kweefoto. Esi sifundo siphonononga ukuba igalelo elikhethekileyo leendlela ezintsha zonxibelelwano lwemiboniso/imidiya lingenza njani ukuqinisa ukuqonda unxulumano phakathi kwemiboniso yedijithali neyeanalogu. Kwakhona, esi sifundo sizama ukuphanda iindlela ezintsha ezisetyenziswa kubugcisa bedijithali neziquka imiba ephambili yemiboniso yale mihla neyakudala. From this analysis several concepts arise that serve as distinguishing markers for media displacement. These themes include embodiment, memory, identity formation, autotopography and intermediality. Kuphononongwe iingxoxo ezithile ezimalunga nezifundo zemiboniso yale mihla. Kuqwalaselwe ngakumbi imisebenzi yeengcali ooLev Manovich, Jay David Bolter, Richard Grusin kunye noFilipe Pais malunga nonxulumano phakathi kwemiboniso yakudala neyale mihla njengesiseko solu hlalutyo. Ezi ngxoxo zifaka emxholweni uhlalutyo lwezifundo zokuzekelisa zale mihla malunga nabazobi bale mihla, kugxininiswa kwindibanisela ka John Craig Freeman nekaOddviz. Le misebenzi ikhethwe ngenxa yokuba iyazichitha iingcinga eziqhelekileyo malunga nokufota ngedijithali kwimiboniso yale mihla nangendlela iziqulatho zedijithali “zinokushenxiswa” zisiwe kwindawo ebambekayo. Olu hlalutyo luveze iingcinga eziliqela nezisebenza njengeempawu zoshenxiso lwemiboniso. Imixholo iquka imifuziselo, ukukhumbula, ukwenziwa kwesazisi, ukuzazisa ngezinto onazo, unxulumano phakathi kwemiboniso eyahlukeneyo Le ngxelo yophando igqibela ngokushwankathela umsebenzi wam ohlanganisa iingcinga ezivele ekuhlalutyeni kwam izifundo ezingumzekelo. Ingxoxo ibonisa ukuba umboniso wengqokelela yemisebenzi yam owaziwa ngokuba yiDigital Tourist, ubusebenzise njani ubuchwepheshe ekuthiwa yifotogrametri (obokufumana ulwazi ngokuhlalutya imifanekiso) ekushenxiseni unxulumano lwabucala lobomi bomntu ibubeke kwindawo ebonwa nguwonkewonke apho kubukwa imifanekiso neefoto (igalari). / https://doi.org/10.25399/UnisaData.14101913.v1 / Arts and Music / M.A. (Visual Arts)