We propose Instant High Dynamic Range Neural Radiance Fields (Instant HDR-NeRF), a method of learning high dynamic range (HDR) view synthesis from a set of low dynamic range (LDR) views with unknown and varying exposure and white balance in as little as minutes. Our method can render novel HDR views without ground-truth supervision, and novel LDR views in different exposure settings, including those that match the ground-truth LDR views. The key to our method is to model the physical process of the camera with two implicit MLPs: a radiance field and a monotonically increasing tone-mapper. Built upon Instant Neural Graphics Primitives (Instant-NGP), the radiance field encodes the scene geometry and radiance (from 0 to ∞), and outputs the densities and the radiance at locations along the camera ray. The monotonically increasing tone-mapper models the camera response function (CRF) where the radiance hits on the camera sensor and becomes a pixel value (from 0 to 255). The radiance at each location is combined with the learnable exposure parameters, optimized separately for each color band and for each image. A quantitative evaluation on benchmark datasets shows that our method outperforms prior HDR novel view synthesis methods in LDR rendering quality and training speed. To best of our knowledge, our method is also the first HDR radiance field that successfully recovers the ground-truth CRF with a low average error rate of 3.70%, while co-learning geometry, radiance, and exposures all at the same time through implicit functions. In practical applications, our method can produce high-fidelity 3D reconstruction of real-world scenes from images of varying exposure settings, which is particularly useful for casual capturing, where fixed settings aren’t guaranteed. The tone-mapper MLP can be easily controlled to simulate auto-exposure effects, making it useful in filming and video games. Furthermore, the HDR radiance maps produced by our method can be edited and tone-mapped according to user preferences.

40 light years from Earth an Earth sized exoplanet called Trappist 1d orbits the M-dwarf star called Trappist 1. Trappist 1d is located in the habitable zone where liquid water could exist on the surface of the planet which raises the question: Could Trappist 1d be habitable? Since it is not known what the planet looks like, several simulations of potentially habitable climates were made including different water levels and atmospheric pressures with Earth-like atmospheres. Real observations with JWST and VLT are currently being made for the light passing through Trappist 1d’s potential atmosphere. In order to interpret the data and make any conclusions about the habitability of Trappist 1d, simulated spectra need to be made for the different scenarios. The goal of this project was to produce radiance spectrum of how observations viewed through different instruments would look like for the different planetary scenarios. The result of the project gave spectra that were quite similar, but differed specifically in the depths of the lines, meaning that in theory it could be possible to distinguish between the planetary scenarios via observations. In reality, because of uncertainties in the observations, it is probably not possible to distinguish between the different planetary models, but it might be enough to conclude if the planet has an Earth like atmosphere containing CO2 and H2O or not. / 40 ljusår bort från jorden kretsar en jordlik planet vid namn Trappist 1d runt en röd dvärgstjärna. Trappist 1d ligger i den så kallade beboeliga zonen där det är möjligt för flytande vatten att existera på planetens yta. Detta medför frågan: Finns det förutsättningar för liv på Trappist 1d? Eftersom det inte är känt hur det ser ut på planeten har flera potentiellt beboeliga klimat simulerats för olika vattennivåer och atmosfärstryck med en jordlik atmosfär. Olika instrument på teleskopen JWST och VLT samlar för tillfället in data för observationer genom Trappist 1d:s potentiella atmosfär. För att kunna tolka datan och dra slutsatser om förutsättningarna för liv på Trappist 1d behövs simulerade spektrum att jämföra med. Målet med det här projektet är att producera simulerade radians spektrum för hur observationer med de olika instrumenten hade sett ut för de olika scenarierna. Resultatet gav spektrum som främst skiljde sig i djupet av linjerna i graferna, vilket betyder att i teorin är det möjligt att skilja mellan de olika scenariona för en observation. På grund av osäkerheter i observationen, är det troligtvis inte möjligt i praktiken att se exakt vilket scenario det tillhör, men det kan vara tillräckligt för att säga ifall planeten har en jordlik atmosfär som innehåller vatten och koldioxid eller ej.

Simuler efficacement l'éclairage global est l'un des problèmes ouverts les plus importants en infographie. Calculer avec précision les effets de l'éclairage indirect, causés par des rebonds secondaires de la lumière sur des surfaces d'une scène 3D, est généralement un processus coûteux et souvent résolu en utilisant des algorithmes tels que le path tracing ou photon mapping. Ces techniquesrésolvent numériquement l'équation du rendu en utilisant un lancer de rayons Monte Carlo. Ward et al. ont proposé une technique nommée irradiance caching afin d'accélérer les techniques précédentes lors du calcul de la composante indirecte de l'éclairage global sur les surfaces diffuses. Krivanek a étendu l'approche de Ward et Heckbert pour traiter le cas plus complexe des surfaces spéculaires, en introduisant une approche nommée radiance caching. Jarosz et al. et Schwarzhaupt et al. ont proposé un modèle utilisant le hessien et l'information de visibilité pour raffiner le positionnement des points de la cache dans la scène, raffiner de manière significative la qualité et la performance des approches précédentes. Dans ce mémoire, nous avons étendu les approches introduites dans les travaux précédents au problème du radiance caching pour améliorer le positionnement des éléments de la cache. Nous avons aussi découvert un problème important négligé dans les travaux précédents en raison du choix des scènes de test. Nous avons fait une étude préliminaire sur ce problème et nous avons trouvé deux solutions potentielles qui méritent une recherche plus approfondie. / Efficiently simulating global illumination is one of the most important open problems in computer graphics. Accurately computing the effects of indirect illumination, caused by secondary bounces of light off surfaces in a 3D scene, is generally an expensive process and often solved using algorithms such as path tracing or photon mapping. These approaches numerically solve the rendering equation using stochastic Monte Carlo ray tracing. Ward et al. proposed irradiance caching to accelerate these techniques when computing the indirect illumination component on diffuse surfaces. Krivanek extended the approach of Ward and Heckbert to handle the more complex case of glossy surfaces, introducing an approach referred to as radiance caching. Jarosz et al. and Schwarzhaupt et al. proposed a more accurate visibility-aware Hessian-based model to greatly improve the placement of records in the scene for use in an irradiance caching context, significantly increasing the quality and performance of the baseline approach. In this thesis, we extended similar approaches introduced in these aforementioned work to the problem of radiance caching to improve the placement of records. We also discovered a crucial problem overlooked in the previous work due to the choice of test scenes. We did a preliminary study of this problem, and found several potential solutions worth further investigation.

Les ombres sont un élément important pour la compréhension d'une scène. Grâce à elles, il est possible de résoudre des situations autrement ambigües, notamment concernant les mouvements, ou encore les positions relatives des objets de la scène. Il y a principalement deux types d'ombres: des ombres dures, aux limites très nettes, qui résultent souvent de lumières ponctuelles ou directionnelles; et des ombres douces, plus floues, qui contribuent à l'atmosphère et à la qualité visuelle de la scène. Les ombres douces résultent de grandes sources de lumière, comme des cartes environnementales, et sont difficiles à échantillonner efficacement en temps réel. Lorsque l'interactivité est prioritaire sur la qualité, des méthodes d'approximation peuvent être utilisées pour améliorer le rendu d'une scène à moindre coût en temps de calcul. Nous calculons interactivement les ombres douces résultant de sources de lumière environnementales, pour des scènes composées d'objets en mouvement et d'un champ de hauteurs dynamique. Notre méthode enrichit la méthode d'exponentiation des harmoniques sphériques, jusque là limitée aux bloqueurs sphériques, pour pouvoir traiter des champs de hauteurs. Nous ajoutons également une représentation pour les BRDFs diffuses et glossy. Nous pouvons ainsi combiner les visibilités et BRDFs dans un même espace, afin de calculer efficacement les ombres douces et les réflexions de scènes complexes. Un algorithme hybride, qui associe les visibilités en espace écran et en espace objet, permet de découpler la complexité des ombres de la complexité de la scène. / Shadows provide important visual cues to a viewer about the relative positions of objects in a scene, as well as certain properties of the lighting in an environment, such as orientation, size, and intensity. The importance of shadows in visual simulations is even more striking when any element of an environment, such as characters in a scene or the light sources themselves, are animated over time. The simulation of so-called "hard" shadows from small point or directional light sources is a very mature field in computer graphics, with many concrete and well-established solutions. On the other hand, efficiently approximating the shadowing effects from larger "area" light sources, such as ceiling lights or environment maps captured from the real world, remains an open problem. Indeed, in many applications, the availability of a high-performance solution to this problem trumps the need for an accurate solution. Our work aims to solve the problem of approximating soft shadows interactively, in scenes where the geometric elements and lighting are both allowed to be animated over time. We decompose dynamic scene elements into deformable objects, approximated with a collection of non-deformable animated spheres, and height field geometry. By leveraging a novel spherical harmonic basis-space exponentiation formulation, we are able to very quickly accumulate the shadowing effects from these many dynamic blockers, while also encoding their local reflectance behaviour in a similar reduced basis representation. Our proof-of-concept implementation uses a hybrid, multi-resolution image- and object-space visibility marching algorithm that decouples geometric complexity from radiometric complexity. We demonstrate our method on several scenes with dynamic blockers and complex illumination.

Depuis quelques décennies, les communautés d'informatique graphique et de vision ont contribué à l'émergence de technologies permettant la numérisation d'objets 3D. Une demande grandissante pour ces technologies vient des acteurs de la culture, notamment pour l'archivage, l'étude à distance et la restauration d'objets du patrimoine culturel : statuettes, grottes et bâtiments par exemple. En plus de la géométrie, il peut être intéressant de numériser la photométrie avec plus ou moins de détail : simple texture (2D), champ de lumière (4D), SV-BRDF (6D), etc. Nous formulons des solutions concrètes pour la création et le traitement de champs de lumière surfaciques représentés par des fonctions de radiance attachés à la surface.Nous traitons le problème de la phase de construction de ces fonctions à partir de plusieurs prises de vue de l'objet dans des conditions sur site : échantillonnage non structuré voire peu dense et bruité. Un procédé permettant une reconstruction robuste générant un champ de lumière surfacique variant de prévisible et sans artefacts à excellente, notamment en fonction des conditions d'échantillonnage, est proposé. Ensuite, nous suggérons un algorithme de simplification permettant de réduire la complexité mémoire et calculatoire de ces modèles parfois lourds. Pour cela, nous introduisons une métrique qui mesure conjointement la dégradation de la géométrie et de la radiance. Finalement, un algorithme d'interpolation de fonctions de radiance est proposé afin de servir une visualisation lisse et naturelle, peu sensible à la densité spatiale des fonctions. Cette visualisation est particulièrement bénéfique lorsque le modèle est simplifié. / Vision and computer graphics communities have built methods for digitizing, processing and rendering 3D objects. There is an increasing demand coming from cultural communities for these technologies, especially for archiving, remote studying and restoring cultural artefacts like statues, buildings or caves. Besides digitizing geometry, there can be a demand for recovering the photometry with more or less complexity : simple textures (2D), light fields (4D), SV-BRDF (6D), etc. In this thesis, we present steady solutions for constructing and treating surface light fields represented by hemispherical radiance functions attached to the surface in real-world on-site conditions. First, we tackle the algorithmic reconstruction-phase of defining these functions based on photographic acquisitions from several viewpoints in real-world "on-site" conditions. That is, the photographic sampling may be unstructured and very sparse or noisy. We propose a process for deducing functions in a manner that is robust and generates a surface light field that may vary from "expected" and artefact-less to high quality, depending on the uncontrolled conditions. Secondly, a mesh simplification algorithm is guided by a new metric that measures quality loss both in terms of geometry and radiance. Finally, we propose a GPU-compatible radiance interpolation algorithm that allows for coherent radiance interpolation over the mesh. This generates a smooth visualisation of the surface light field, even for poorly tessellated meshes. This is particularly suited for very simplified models.

Recent advancements in computer vision have led to the emergence of Neural Radiance Fields (NeRFs), a powerful tool for reconstructing photorealistic 3D scenes, even in dynamic settings. However, these methods struggle when dealing with human subjects, especially when the subject is partially obscured or not completely observable, resulting in inaccurate reconstructions of geometries and textures. To address this issue, this thesis evaluates state-of-the-art human modeling using implicit representations with partial observability of the subject. We then propose and test several novel methods to improve the generalization of these models, including the use of symmetry and Signed Distance Function (SDF) driven losses and leveraging prior knowledge from multiple subjects via a pre-trained model. Our results demonstrate that our proposed methods significantly improve the accuracy of the reconstructions, even in challenging ”in-the-wild” situations, both quantitatively and qualitatively. Our approach opens new opportunities for applications such as asset generation for video games and movies and improved simulations for autonomous driving scenarios from abundant in-the-wild monocular videos. In summary, our research presents a significant improvement to the state-of-the-art human modeling using implicit representations, with important implications for 3D Computer Vision (CV) and Neural Rendering and its applications in various industries. / De senaste framstegen inom datorseende har lett till uppkomsten av Neural Radiance Fields (NeRFs), ett kraftfullt verktyg för att rekonstruera fotorealistiska 3D-scener, även i dynamiska miljöer. Dessa metoder brister dock vid hantering av människor, särskilt när människan är delvis skymd eller inte helt observerbar, vilket resulterar i felaktiga rekonstruktioner av geometrier och texturer. För att ta itu med denna fråga, utvärderar denna avhandling toppmodern mänsklig modellering med hjälp av implicita representationer med partiell observerbarhet av ämnet. Vidare föreslår, samt testar vi, flertalet nya metoder för att förbättra generaliseringen av dessa modeller, inklusive användningen av symmetri och SDF-drivna förluster och utnyttjandet av förkunskaper från flera individer via en förtränad modell. Resultaten visar att våra föreslagna metoder avsevärt förbättrar rekonstruktionernas noggrannhet, även i utmanande ”in-the-wild” situationer, både kvantitativt och kvalitativt. Vårt tillvägagångssätt skapar nya möjligheter för applikationer som tillgångsgenerering för videospel och filmer och förbättrade simuleringar för scenarier för autonom körning från rikliga monokulära videor. Sammanfattningsvis, presenterar vår forskning en betydande förbättring av toppmodern modelleringen med hjälp av implicita representationer, med viktiga implikationer för 3D CV och neural rendering och dess tillämpningar i olika industrier.

Effective daylighting can substantially reduce the energy use and greenhouse gas emissions of commercial buildings. Daylight is also healthy for building occupants, and contributes to occupant satisfaction. When productivity improvements are considered, effective daylighting is also highly attractive financially. However, successful daylighting of sub-tropical buildings is a very difficult task, due to high direct irradiances and excessive solar shading. A device was created that combined effective solar shading and efficient daylight redirection. The micro-light guiding shade panel achieves all objectives of an optimal daylighting device placed on the façade of a sub-tropical, high rise office building. Its design is based on the principles of non-imaging optics. This provides highly efficient designs offering control over delivered illumination, within the constraints of the second law of thermodynamics. Micro-light guiding shade panels were constructed and installed on a test building. The tested devices delivered daylight deep into the building under all conditions. Some glare was experienced with a poorly chosen translucent material. Glare was eliminated by replacing this material. Construction of the panels could be improved by application of mass-manufacturing techniques including metal pressing. For the micro-light guiding shade panel to be utilised to its full potential, building designers must understand its impact on building performance early in the design process. Thus, the device must be modelled with lighting simulation software currently in use by building design firms. The device was successfully modelled by the RADIANCE lighting simulator. RADIANCE predictions compared well with measurements, providing bias generally less than 10%. Simulations greatly aided further development of the micro-light guiding shade panel. Several new RADIANCE algorithms were developed to improve daylight simulation in general.

La prédiction de l'émission infrarouge des jets de propulseurs composites aluminisés nécessite principalement trois étapes : le calcul des grandeurs aérothermochimiques du jet, la conversion de ces grandeurs en propriétés optiques (coefficient d'absorption, coefficient de diffusion, fonction de phase) puis la résolution de l'équation de transfert radiatif. Cette thèse,essentiellement consacrée à cette troisième étape, propose de nouvelles voies pour l'application des modèles de bande aux cas de milieux biphasiques et diffusants.D'une part, nous avons étendu ces modèles aux cas de milieux caractérisés par un déséquilibre thermique entre gaz et particules. D'autre part, nous avons proposé une méthode de séparation de la luminance en deux contributions, appelées respectivement non diffusée et diffusée, à partir d'une idée originale de Liu et al. La contribution non diffusée est solution de l'équation de transfert radiatif obtenue en ignorant les effets de la diffusion. Par conséquent, elle peut être résolue par une formulation en modèles de bande. Cette approche permet de réduire les erreurs de corrélations spectrales au seul terme de luminance diffusée.Nous avons montré l'intérêt de ces approches par comparaison avec une résolution de l'équation de transfert radiatif en raie par raie, sur des milieux représentatifs de situations de télédétection de jets. / Prediction of infrared emission of exhaust plumes from aluminized composite rocket, follows mainly three steps : calculating aero-thermo-chemical values in the plume, converting those valuesto optical properties (absorption coefficient, scattering coefficient and phase function) and resolving the radiative transfer equation. This thesis is mostly devoted to this third step, and proposes new ways to use band models on two-phases and scattering media.Firstly, we extended band models to cases with thermic non equilibrium between gas and particles. Secondly, we proposed a method consisting in splitting radiance in two parts, one called un-scattered and the other scattered, from an original idea of Liu et al. The un-scattered part is solution of the radiative transfer equation obtained by ignoring scattering. As a result, the unscattered radiance can be found by using band models. By this approach, errors on spectral correlations are only present on the scattered radiance.We show the interest of thoses approches by comparing them with a line by line resolutionof the radiative transfer equation, on media representative of remote sensing cases of rocket exhaust plumes.

This work presents a framework for shading of virtual objects using high dynamic range (HDR) light probe sequences in real-time. The method is based on using HDR environment map of the scene which is captured in an on-line process by HDR video camera as light probes. In each frame of the HDR video, an optimized CUDA kernel is used to project incident lighting into spherical harmonics in realtime. Transfer coefficients are calculated in an offline process. Using precomputed radiance transfer the radiance calculation reduces to a low order dot product between lighting and transfer coefficients. We exploit temporal coherence between frames to further smooth lighting variation over time. Our results show that the framework can achieve the effects of consistent illumination in real-time with flexibility to respond to dynamic changes in the real environment. We are using low-order spherical harmonics for representing both lighting and transfer functionsto avoid aliasing.

Human pose estimation in 3D is a large area within computer vision, with many application areas. A common approach is to first estimate the pose in 2D, resulting in a confidence heatmap, and then estimate the 3D pose using the most likely estimations in 2D. This may, however, cause problems in cases where pose estimates are more uncertain and the estimation of one point is far from the true position, for example when a limb is occluded. This thesis adapts the method Neural Radiance Fields (NeRF) to 2D confidence heatmaps in order to create an implicit representation of the uncertainty in 3D, thus attempting to make use of as much information in 2D as possible. The adapted method was evaluated on the Human3.6M dataset, and results show that this method outperforms a simple triangulation baseline, especially when the estimation in 2D is far from the true pose.