Indiana University-Purdue University Indianapolis (IUPUI) / Although current augmented, virtual, and mixed reality (AR/VR/MR) systems are facing advanced and immersive experience in the entertainment industry with countless media forms. Theses systems suffer a lack of correct direct and indirect illumination modeling where the virtual objects render with the same lighting condition as the real environment. Some systems are using baked GI, pre-recorded textures, and light probes that are mostly accomplished offline to compensate for precomputed real-time global illumination (GI). Thus, illumination information can be extracted from the physical scene for interactively rendering the virtual objects into the real world which produces a more realistic final scene in real-time. This work approaches the problem of visual coherence in AR by proposing a system that detects the real-world lighting conditions in dynamic scenes, then uses the extracted illumination information to render the objects added to the scene. The system covers several major components to achieve a more realistic augmented reality outcome. First, the detection of the incident light (direct illumination) from the physical scene with the use of computer vision techniques based on the topological structural analysis of 2D images using a live-feed 360-degree camera instrumented on an AR device that captures the entire radiance map. Also, the physics-based light polarization eliminates or reduces false-positive lights such as white surfaces, reflections, or glare which negatively affect the light detection process. Second, the simulation of the reflected light (indirect illumination) that bounce between the real-world surfaces to be rendered into the virtual objects and reflect their existence in the virtual world. Third, defining the shading characteristic/properties of the virtual object to depict the correct lighting assets with a suitable shadow casting. Fourth, the geometric properties of real-scene including plane detection, 3D surface reconstruction, and simple meshing are incorporated with the virtual scene for more realistic depth interactions between the real and virtual objects. These components are developed methods which assumed to be working simultaneously in real-time for photo-realistic AR. The system is tested with several lighting conditions to evaluate the accuracy of the results based on the error incurred between the real/virtual objects casting shadow and interactions. For system efficiency, the rendering time is compared with previous works and research. Further evaluation of human perception is conducted through a user study. The overall performance of the system is investigated to reduce the cost to a minimum.

La caractérisation fine des matériaux inorganiques nécessite d'avoir accès à des informations complémentaires de celles apportées par des techniques d'analyse élémentaire ou de diffraction. La spectroscopie infrarouge à transformée de Fourier permet de caractériser les liaisons covalentes et l'environnement des groupes fonctionnels dans les matériaux. C'est donc une technique de choix pour l'étude des matériaux hydratés, amorphes ou sujets à des phénomènes de vieillissement. En couplant cette technique à une platine de déplacement, il est possible de réaliser des cartographies des phases sur quelques centimètres carrés : c'est la microscopie infrarouge. Cette thèse développe plus particulièrement l'utilisation de la lumière réfléchie au travers de l'étude de la réflexion spéculaire et de la réflexion totale atténuée (ATR).Après une première partie se focalisant sur les méthodes d'acquisitions disponibles, une seconde partie s'attache à obtenir de manière non supervisée les cartographies chimiques associées aux concentrations relatives des différents composants présents dans la zone analysée. Des techniques de réduction de données et d'analyse factorielle sont mises en place afin d'estimer le nombre de composants chimiques et leurs spectres relatifs ; des problèmes de minimisation sous contraintes sont résolus pour extraire l'information chimique. La réflexion spéculaire ne nécessite aucun contact avec l‘échantillon et, de ce fait, n'entraine aucune altération de la surface analysée. C'est sur le papier une technique de choix pour suivre l'évolution d'un matériau. Cependant, elle souffre de la complexité d'interprétation liée à l'allure des spectres obtenus. Afin de développer la cartographie issue de la réflexion spéculaire, des modèles prenant en compte l'optique géométrique, l'optique ondulatoire, des corrections d'interférogrammes et des méthodes classiques d'homogénéisation ont été développés. Ce travail a permis d'aboutir à un modèle optique liant les spectres issus de la réflexion spéculaire avec les concentrations relatives des composants. Ce modèle tient compte de la polarisation, de l'angle d'incidence et utilise les constantes diélectriques du matériau. Ce modèle a été validé sur un matériau contenant trois composants distincts facilement identifiables en infrarouge et spécialement mis en forme pour cette étude. Ce modèle a ouvert la voie à l'utilisation innovante de lumières polarisées elliptiquement pour déterminer l'indice de réfraction complexe d'un matériau. Ainsi, des spectroscopes infrarouges couplés à un accessoire de contrôle de l'angle d'incidence peuvent être utilisés en complément de l'ellipsométrie / Complementary information, to that provided by elemental analysis and diffraction techniques, is needed to characterize inorganic materials. Fourier Transform Infrared spectroscopy enables to characterize covalent bonds and the environment of functional groups in materials. Thus, it is a technique of interest to study hydrated materials, amorphous materials or any materials, which may experience ageing phenomena. By combining this technique with a micrometric motorized stage, cartographies of chemical compounds can be obtained on several square millimeters: this is the infrared microscopy technique. This Ph.D. thesis focuses on the use of reflected light, in particular through the study of specular reflection and of Attenuated Total Reflectance (ATR). After a first part focused on the different acquisition set-ups, a second part covers the unsupervised methodologies of resolution employed to obtain chemical maps. They result in one map for each component present in the analyzed area. Dimensions reduction techniques and multivariate statistics techniques are implemented to estimate the number of components and their infrared spectra; minimization problems under constraints are solved to retrieve chemical information. When specular reflection is used to acquire spectra, no contact is made with the sample, thus no damage of the analyzed area occurs during the acquisition. A priori, it is a great technique to study the evolution of a material. However, this technique suffers from the complexity of interpretation of the resulting spectra. With the objective to democratize the use of specular reflection to obtain chemical maps, models based on geometrical optics and including diffraction, correction of interferograms and classical homogenization techniques have been developed. This work resulted in an optical model linking the angle of incidence, the polarization state and the dielectric optical constants of the material with the reflected light, which is measured. A model material, constituted of three distinct phases, detectable in the infrared range, has specially been fabricated to validate this optical model. This model set the stage for the use of elliptically polarized light in the determining of the complex refractive indices of materials in the infrared range. Thanks to this development, infrared spectroscopes, equipped with a classical set-up to control the angle of incidence, can now be used in addition to ellipsometry techniques

Although current augmented, virtual, and mixed reality (AR/VR/MR) systems are facing advanced and immersive experience in the entertainment industry with countless media forms. Theses systems suffer a lack of correct direct and indirect illumination modeling where the virtual objects render with the same lighting condition as the real environment. Some systems are using baked GI, pre-recorded textures, and light probes that are mostly accomplished offline to compensate for precomputed real-time global illumination (GI). Thus, illumination information can be extracted from the physical scene for interactively rendering the virtual objects into the real world which produces a more realistic final scene in real-time. This work approaches the problem of visual coherence in AR by proposing a system that detects the real-world lighting conditions in dynamic scenes, then uses the extracted illumination information to render the objects added to the scene. The system covers several major components to achieve a more realistic augmented reality outcome. First, the detection of the incident light (direct illumination) from the physical scene with the use of computer vision techniques based on the topological structural analysis of 2D images using a live-feed 360^o camera instrumented on an AR device that captures the entire radiance map. Also, the physics-based light polarization eliminates or reduces false-positive lights such as white surfaces, reflections, or glare which negatively affect the light detection process. Second, the simulation of the reflected light (indirect illumination) that bounce between the real-world surfaces to be rendered into the virtual objects and reflect their existence in the virtual world. Third, defining the shading characteristic/properties of the virtual object to depict the correct lighting assets with a suitable shadow casting. Fourth, the geometric properties of real-scene including plane detection, 3D surface reconstruction, and simple meshing are incorporated with the virtual scene for more realistic depth interactions between the real and virtual objects. These components are developed methods which assumed to be working simultaneously in real-time for photo-realistic AR. The system is tested with several lighting conditions to evaluate the accuracy of the results based on the error incurred between the real/virtual objects casting shadow and interactions. For system efficiency, the rendering time is compared with previous works and research. Further evaluation of human perception is conducted through a user study. The overall performance of the system is investigated to reduce the cost to a minimum.