Recent advancements in inverse rendering have exhibited promising results for 3D representation, novel view synthesis, scene parameter reconstruction, and direct graphical asset generation and editing. Inverse rendering attempts to recover the scene parameters of interest from a set of camera observations by optimizing the photometric error between rendering model output and the true observation with appropriate regularization. The objective of this dissertation is to study inverse problems from several perspectives: (1) Software Framework: the general differentiable pipeline for solving physically-based or neural-based rendering problems, (2) Closed-Form: efficient and closed-form solutions in specific condition in inverse problems, (3) Representation Structure: hybrid 3D scene representation for efficient training and adaptive resource allocation, and (4) Robustness: enhanced robustness and accuracy from controlled lighting aspect. We aim to solve the following tasks: 1. How to address the challenge of rendering and optimizing scene parameters such as geometry, texture, and lighting, while considering multiple viewpoints from physically-based or neural 3D representations. To this end, we present a comprehensive software toolkit that provides support for diverse ray-based sampling and tracing schemes that enable the optimization of a wide range of targeting scene parameters. Our approach emphasizes the importance of maintaining differentiability throughout the entire pipeline to ensure efficient and effective optimization of the desired parameters. 2. Is there a 3D representation that has a fixed computational complexity or closed-form solution for forward rendering when the target has specific geometry or simplified lighting cases for better relaxing computational problems or reducing complexity. We consider multi-bounce reflection inside the plane transparent medium, and design differentiable polarization simulation engine that jointly optimize medium's parameters as well as the polarization state of reflection and transmission light. 3. How can we use our hybrid, learned 3D scene representation to solve inverse rendering problems for scene reconstruction and novel view synthesis, with a particular interest in several scientific fields, including density, radiance field, signed distance function, etc. 4. Unknown lighting condition significantly influence object appearance, to enhance the robustness of inverse rendering, we adopt invisible co-located lighting to further control lighting and suppress unknown lighting by jointly optimize separated channels of RGB and near infrared light, and enable accurate all scene parameters reconstruction from wider application environment. We also demonstrate the visually and quantitatively improved results for the aforementioned tasks and make comparisons with other state-of-the-art methods to demonstrate superior performance on representation and reconstruction tasks.

No description available.

<div>Many real-world domains are relational, consisting of objects (e.g., users and pa- pers) linked to each other in various ways. Because class labels in graphs are often only available for a subset of the nodes, semi-supervised learning for graphs has been studied extensively to predict the unobserved class labels. For example, we can pre- dict political views in a partially labeled social graph dataset and get expected gross incomes of movies in an actor/movie graph with a few labels. Recently, advances in representation learning for graph data have made great strides for the semi-supervised node classification. However, most of the methods have mainly focused on learning node representations by considering simple relational properties (e.g., random walk) or aggregating nearby attributes, and it is still challenging to learn complex inter- action patterns in partially labeled graphs and provide explanations on the learned representations. </div><div><br></div><div>In this dissertation, multiple methods are proposed to alleviate both challenges for semi-supervised node classification. First, we propose a graph neural network architecture, REGNN, that leverages local inferences for unlabeled nodes. REGNN performs graph convolution to enable label propagation via high-order paths and predicts class labels for unlabeled nodes. In particular, our proposed attention layer of REGNN measures the role equivalence among nodes and effectively reduces the noise, which is generated during the aggregation of observed labels from distant neighbors at various distances. Second, we also propose a neural network archi- tecture that jointly captures both temporal and static interaction patterns, which we call Temporal-Static-Graph-Net (TSGNet). The architecture learns a latent rep- resentation of each node in order to encode complex interaction patterns. Our key insight is that leveraging both a static neighbor encoder, that learns aggregate neigh- bor patterns, and a graph neural network-based recurrent unit, that captures complex interaction patterns, improves the performance of node classification. Lastly, in spite of better performance of representation learning on node classification tasks, neural network-based representation learning models are still less interpretable than the pre- vious relational learning models due to the lack of explanation methods. To address the problem, we show that nodes with high bridgeness scores have larger impacts on node embeddings such as DeepWalk, LINE, Struc2Vec, and PTE under perturbation. However, it is computationally heavy to get bridgeness scores, and we propose a novel gradient-based explanation method, GRAPH-wGD, to find nodes with high bridgeness efficiently. In our evaluations, our proposed architectures (REGNN and TSGNet) for semi-supervised node classification consistently improve predictive performance on real-world datasets. Our GRAPH-wGD also identifies important nodes as global explanations, which significantly change both predicted probabilities on node classification tasks and k-nearest neighbors in the embedding space after perturbing the highly ranked nodes and re-learning low-dimensional node representations for DeepWalk and LINE embedding methods.</div>

L’objectif de cette thèse est de décrire le réseau cérébral et la dynamique neuronale qui pourraient servir de support aux aversions alimentaires de type olfactives. Nous avons réalisé des enregistrements multisites de potentiel de champ locaux chez le rat vigile engagé dans cet apprentissage, en proposant deux modes de présentation de l’indice olfactif : à proximité de l’eau de boisson (distal) ou ingéré (distal-proximal). Après apprentissage, la présentation du seul indice distal induit l’émergence d’une activité oscillatoire de forte amplitude dans la bande de fréquence beta (15-40 Hz). Finement corrélée au comportement d’aversion de l’animal, cette activité est proposée comme la signature du réseau de structures fonctionnellement impliquées dans la reconnaissance de l’odeur en tant que signal. Nous montrons que ce réseau peut être plus ou moins étendu selon la façon dont le stimulus a été perçu lors du conditionnement: dans certaines aires (bulbe olfactif, cortex piriforme, amygdale basolatérale, cortex orbitofrontal) la modulation en puissance de l’activité beta se fait indépendamment du mode de conditionnement; dans d’autres aires (cortex insulaire, cortex infralimbique) ces changements ont lieu si et seulement si l’odeur a été ingérée. Complétés par l’étude des interactions fonctionnelles entre ces différentes structures dans la bande de fréquence considérée, ces résultats nous permettent de mieux comprendre comment un stimulus peut être représenté en mémoire dans un réseau cérébral en fonction de l’expérience que l’animal en a fait. / The goal of this thesis is to describe the dynamic of the neural network involved in food aversion based on olfactory cue. We performed multisite recordings of local field potential in the behaving rat engaged in such aversion learning and offered two modes of presentation of the olfactory cue: either close to drinking water (distal) or ingested (distal-proximal). After learning, the presentation of the distal cue alone induced the emergence of a powerful oscillation in the beta frequency band (15-40 Hz). Finely correlated with the aversive behavior of the animal, this activity has been proposed as the signature of the neural network functionally involved in odor signal recognition. We showed that this network may be more or less extended depending on how the stimulus has been experienced during conditioning: in some areas (olfactory bulb, piriform cortex, basolateral amygdala, orbitofrontal cortex), modulation of beta power were observed whatever the mode of odor presentation; in other areas (insular and infralimbic cortices) these changes took place only if the odor cue has been ingested. Associated to the analysis of transient oscillatory synchronizations between these different structures, these results allowed us to better understand how a stimulus could be represented in memory by a cerebral network depending on the way the animal had experienced it.