Ces dernières années, les contenus vidéos ont évolué très rapidement. En effet, les télévisions (TV) ont rapidement évolué vers l’Ultra Haute résolution (UHD), la Haute Fréquence d’images (HFR) ou la stéréoscopie (3D). La tendance actuelle est à l’imagerie à Haute Dynamique de luminance (HDR). Ces technologies permettent de reproduire des images beaucoup plus lumineuses que celles des écrans actuels. Chacune de ces améliorations représente une augmentation du coût de stockage et nécessite la création de nouveaux standards de compression vidéo, toujours plus performant. La majorité des consommateurs est actuellement équipé de TV ayant une Dynamique Standard (SDR) qui ne supportent pas les contenus HDR et ils vont lentement renouveler leurs écrans pour un HDR. Il est donc important de délivrer un signal HDR qui puisse être décodé par ces deux types d’écrans. Cette rétro compatibilité est rendue possible par un outil appelé TMO (Tone Mapping Operator) qui transforme un contenu HDR en une version SDR. Au travers de cette thèse, nous explorons de nouvelles méthodes pour améliorer la compression HDR rétro compatible. Premièrement, nous concevons un TMO qui optimise les performances d’un schéma de compression scalable où une couche de base et d’amélioration sont envoyées pour reconstruire les contenus HDR et SDR. Il est démontré que le TMO optimal dépend seulement de la couche SDR de base et que le problème de minimisation peut être séparé en deux étapes consécutives. Pour ces raisons, nous proposons ensuite un autre TMO conçu pour optimiser les performances d’un schéma de compression utilisant uniquement une couche de base mais avec un modèle amélioré et plus précis. Ces deux travaux optimisent des TMO pour images fixes. Par la suite, la thèse se concentre sur l’optimisation de TMO spécifiques à la vidéo. Cependant, on y démontre que l’utilisation d’une prédiction pondérée pour la compression SDR est aussi bon voir meilleur que d’utiliser un TMO optimisé temporellement. Pour ces raisons, un nouvel algorithme et de nouveaux modes de prédictions pondérées sont proposés pour gérer plus efficacement la large diversité des changements lumineux dans les séquences vidéos. / In recent years, video content evolved very quickly. Indeed, televisions (TV) quickly evolved to Ultra High Definition (UHD), High Frame Rate (HFR) or stereoscopy (3D). The recent trend is towards High Dynamic range (HDR). These new technologies allow the reproduction of much brighter images than for actual displays. Each of these improvements represents an increase in storage cost and therefore requires the creation of new video compression standards, always more efficient. The majority of consumers are currently equipped with Standard Dynamic Range (SDR) displays, that cannot handle HDR content. Consumers will slowly renew their display to an HDR one and it is therefore of great importance to deliver an HDR signal that can be decoded by both SDR and HDR displays. Such backward compatibility is provided by a tool called Tone Mapping Operator (TMO) which transforms an HDR content into an SDR version. In this thesis, we explore new methods to improve the backward compatible HDR compression. First, we design a Tone Mapping to optimize scalable compression scheme performances where a base and an enhancement layer are sent to reconstruct the SDR and HDR content. It is demonstrated that the optimum TMO only depends on the SDR base layer and that the minimization problem can be separated in two consecutive minimization steps. Based on these observations, we then propose another TMO designed to optimize the performances of compression schemes using only a base layer but with an enhanced and more precise model. Both of these works optimize TMO for still images. Thereafter, this thesis focuses on the optimization of video-specific TMO. However, we demonstrate that using a weighted prediction for the SDR compression is as good or even better than using a temporally optimized TMO. Therefore, we proposed a new weighted prediction algorithm and new weighted prediction modes to handle more efficiently the large diversity of brightness variations in video sequences.

Ces dernières années, les technologies d’écran se sont considérablement améliorées. Par exemple, le contraste des écrans à plage dynamique élevée (HDR) dépasse de loin la capacité d’un écran conventionnel. De plus, un écran à gamut de couleur étendu (WCG) peut couvrir un espace colorimétrique plus grand que jamais. L'évaluation de la qualité de ces nouveaux contenus est devenue un domaine de recherche actif, les métriques de qualité SDR classiques n'étant pas adaptées. Cependant, les études les plus récentes négligent souvent une caractéristique importante: les chrominances. En effet, les bases de données existantes contiennent des images HDR avec un gamut de couleur standard, négligeant ainsi l’augmentation de l’espace colorimétrique due au WCG et les artefacts chromatiques. La plupart des mesures de qualité HDR objectives non plus ne prennent pas en compte ces artefacts. Pour surmonter cette problématique, dans cette thèse, nous proposons deux nouvelles bases de données HDR/WCG annotés avec des scores subjectifs présentant des artefacts chromatique réaliste. En utilisant ces bases de données, nous explorons trois solutions pour créer des métriques HDR/WCG: l'adaptation des métrics de qualité SDR, l’extension colorimétrique d’une métrique HDR connue appelée HDR-VDP-2 et, enfin, la fusion de diverses métriques de qualité et de features colorimétriques. Cette dernière métrique présente de très bonnes performances pour prédire la qualité tout en étant sensible aux distorsions chromatiques. / To improve their ability to display astonishing images, screen technologies have been greatly evolving. For example, the contrast of high dynamic range rendering systems far exceed the capacity of a conventional display. Moreover, a Wide Color gamut display can cover a bigger color space than ever. Assessing the quality of these new content has become an active field of research as classical SDR quality metrics are not adapted. However, state-of-the-art studies often neglect one important image characteristics: chrominances. Indeed, previous databases contain HDR images with a standard gamut thus neglecting the increase of color space due to WCG. Due to their gamut, these databases are less prone to contain chromatic artifacts than WCG content. Moreover, most existing HDR objective quality metrics only consider luminance and are not considering chromatic artifacts. To overcome this problematic, in this thesis, we have created two HDR / WCG databases with annotated subjective scores. We focus on the creation of a realistic chromatic artifacts that can arise during compression. In addition, using these databases, we explore three solutions to create HDR / WCG metrics. First, we propose a method to adapt SDR metrics to HDR / WCG content. Then, we proposed an extension of a well-known HDR metric called HDR-VDP-2. Finally, we create a new metric based on the merger of various quality metric and color features. This last metric presents very good performance to predict quality while being sensitive to chromatic distortion.

Human vision is remarkable. By limiting the main concentration of high-acuity photoreceptors to the eye's central fovea region, we efficiently view the world by redirecting the fovea between points of interest using eye movements called <em>saccades</em>. <br /><br /> Part I describes a saccadic vision system prototype design. The dual-resolution saccadic camera detects objects of interest in a scene by processing low-resolution image information; it then revisits salient regions in high-resolution. The end product is a dual-resolution image in which background information is displayed in low-resolution, and salient areas are captured in high-acuity. This lends to a resource-efficient active vision system. <br /><br />Part II describes CMOS image sensor designs for active vision. Specifically, this discussion focuses on methods to determine regions of interest and achieve high dynamic range on the sensor.

Image based lighting, (IBL), is a computer graphics technique for creating photorealistic renderings of synthetic objects such that they can be placed into real world scenes. IBL has been widely recognized and is today used in commercial production pipelines. However, the current techniques only use illumination captured at a single point in space. This means that traditional IBL cannot capture or recreate effects such as cast shadows, shafts of light or other important spatial variations in the illumination. Such lighting effects are, in many cases, artistically created or are there to emphasize certain features, and are therefore a very important part of the visual appearance of a scene. This thesis and the included papers present methods that extend IBL to allow for capture and rendering with spatially varying illumination. This is accomplished by measuring the light field incident onto a region in space, called an Incident Light Field, (ILF), and using it as illumination in renderings. This requires the illumination to be captured at a large number of points in space instead of just one. The complexity of the capture methods and rendering algorithms are then significantly increased. The technique for measuring spatially varying illumination in real scenes is based on capture of High Dynamic Range, (HDR), image sequences. For efficient measurement, the image capture is performed at video frame rates. The captured illumination information in the image sequences is processed such that it can be used in computer graphics rendering. By extracting high intensity regions from the captured data and representing them separately, this thesis also describes a technique for increasing rendering efficiency and methods for editing the captured illumination, for example artificially moving or turning on and of individual light sources.

Enhancing low light videos has been quite a challenge over the years. A video taken in low light always has the issues of low dynamic range and high noise. This master thesis presents contribution within the field of low light video enhancement. Three models are proposed with different tone mapping algorithms for extremely low light low quality video enhancement. For temporal noise removal, a motion compensated kalman structure is presented. Dynamic range of the low light video is stretched using three different methods. In Model 1, dynamic range is increased by adjustment of RGB histograms using gamma correction with a modified version of adaptive clipping thresholds. In Model 2, a shape preserving dynamic range stretch of the RGB histogram is applied using SMQT. In Model 3, contrast enhancement is done using CLAHE. In the final stage, the residual noise is removed using an efficient NLM. The performance of the models are compared on various Objective VQA metrics like NIQE, GCF and SSIM. To evaluate the actual performance of the models subjective tests are conducted, due to the large number of applications that target humans as the end user of the video.The performance of the three models are compared for a total of ten real time input videos taken in extremely low light environment. A total of 25 human observers subjectively evaluated the performance of the three models based on the parameters: contrast, visibility, visually pleasing, amount of noise and overall quality. A detailed statistical evaluation of the relative performance of the three models is also provided.

Operadores de tone mapping reverso (RTMOs) realçam imagens e vídeos de baixa faixa dinâmica para visualização em monitores de alta faixa dinâmica. Um problema comum encontrado por operadores anteriores é a maneira com que tratam conteúdo sub ou superexposto. Sob tais condições, eles podem não ser eficientes, e até mesmo causar perda e reversão de contraste visível. Apresentamos uma classe de operadores de tone mapping reverso, baseados no filtro bilateral cruzado (cross bilateral filter), capazes de gerar imagens HDR de alta qualidade. Experimentos utilizando uma métrica objetiva de avaliação de imagens demostram que nosso método é a única técnica capaz de realçar detalhes perceptíveis ao longo de uma grande gama de exposições de imagem, a qual inclui desde imagens subexpostas até imagens superexpostas. / Reverse tone mapping operators (rTMOs) enhance low-dynamic-range images and videos for display on high dynamic range monitors. A common problem faced by previous rTMOs is the handling of under or overexposed content. Under such conditions, they may not be effective, and even cause loss and reversal of visible contrast. We present a class of local rTMOs based on cross bilateral filtering that is capable of generating highquality HDR images and videos for a wide range of exposure conditions. Experiments performed using an objective image quality metric show that our approach is the only single technique available that can gracefully enhance perceived details across a large range of image exposures.

The rise of organometal halide perovskite materials with extremely intriguing properties have opened a new horizon in the design of high speed and low price optoelectronic devices. The bandgap in the crystalline structure of these materials can be easily tuned for various applications and their dominant non-excitonic dynamics eliminate the requirement of a bulk or heterostructure for charge carrier separation. These unique properties increase the photo-sensitivity of perovskite-based optoelectronics and provide them with a low time constant, resulting in high precision fast devices. Realization of perovskite-based devices translates directly to inexpensive and simplified architectures of optoelectronic systems. In perovskite-based devices, costly silicon or wide bandgap semiconductor fabrication technology is largely replaced by solution processable methods. Their bandgap tunability allows the reduction of the required optical accessories and interconnects in optoelectronic components. For instance, a tuned perovskite-based detector can substitute a narrowband detecting system consisting of a conventional detector and its required optical accessories such as lenses and color filters. These properties of perovskite-based devices lead to the realization of inexpensive, low power and high-performance optoelectronic systems. In this work, the design of a narrowband, low noise, high performance and stable photodetector based on organic-inorganic hybrid perovskite structure is proposed. The full width at half maximum (FWHM) of the device would be in the nanometer range. The response of the device can be tuned using either different ratios of the lead salts or synthetic dyes (macromolecules) in the crystalline structure for color discrimination in machine vision and imaging applications. Non-excitonic photocarrier generation, tunability of the optical bandgap and low voltage requirements for charge carrier generation are the keys to the utility of this optoelectronic device. The goals of this project were to identify the required functional materials (lead salts and synthetic dyes based on their molecular structures) and optimize their performance; the study of their effect on the charge collection narrowing mechanism and bandwidth specifications defined for detectivity, linear dynamic range (LDR) and photoresponse speed. To achieve these goals, it was proposed to study the light detection properties as well as spectroscopic and semiconductor parameter characteristics of fabricated devices. The design considerations of such devices are versatile and may be modulated for different applications.

<p>Visualization is the process of constructing methods, which are able to synthesize interesting and informative images from data sets, to simplify the process of interpreting the data. In this thesis a new approach to construct meteorological visualization methods using neural network technology is described. The methods are trained with examples instead of explicitely designing the appearance of the visualization.</p><p>This approach is exemplified using two applications. In the fist the problem to compute an image of the sky for dynamic weather, that is taking account of the current weather state, is addressed. It is a complicated problem to tie the appearance of the sky to a weather state. The method is trained with weather data sets and images of the sky to be able to synthesize a sky image for arbitrary weather conditions. The method has been trained with various kinds of weather and images data. The results show that this is a possible method to construct weather visaualizations, but more work remains in characterizing the weather state and further refinement is required before the full potential of the method can be explored. This approach would make it possible to synthesize sky images of dynamic weather using a fast and efficient empirical method.</p><p>In the second application the problem of computing synthetic satellite images form numerical forecast data sets is addressed. In this case a mode is trained with preclassified satellite images and forecast data sets to be able to synthesize a satellite image representing arbitrary conditions. The resulting method makes it possible to visualize data sets from numerical weather simulations using synthetic satellite images, but could also be the basis for algorithms based on a preliminary cloud classification.</p> / Report code: LiU-Tek-Lic-2004:66.

We describe the design and implementation of a high dynamic range (HDR) imaging system capable of capturing RGB color images with a dynamic range of 10,000,000 : 1 at 25 frames per second. We use a highly programmable camera unit with high throughput A/D conversion, data processing and data output. HDR acquisition is performed by multiple exposures in a continuous rolling shutter progression over the sensor. All the different exposures for one particular row of pixels are acquired head to tail within the frame time, which means that the time disparity between exposures is minimal, the entire frame time can be used for light integration and the longest expo- sure is almost the entire frame time. The system is highly configurable, and trade-offs are possible between dynamic range, precision, number of exposures, image resolution and frame rate.

Human vision is remarkable. By limiting the main concentration of high-acuity photoreceptors to the eye's central fovea region, we efficiently view the world by redirecting the fovea between points of interest using eye movements called <em>saccades</em>. <br /><br /> Part I describes a saccadic vision system prototype design. The dual-resolution saccadic camera detects objects of interest in a scene by processing low-resolution image information; it then revisits salient regions in high-resolution. The end product is a dual-resolution image in which background information is displayed in low-resolution, and salient areas are captured in high-acuity. This lends to a resource-efficient active vision system. <br /><br />Part II describes CMOS image sensor designs for active vision. Specifically, this discussion focuses on methods to determine regions of interest and achieve high dynamic range on the sensor.