This dissertation focuses on the many issues that arise from the visual rendering problem. Of primary consideration is light transport simulation, which is known to be computationally expensive. Monte Carlo methods represent a simple and general class of algorithms often used for light transport computation. Unfortunately, the images resulting from Monte Carlo approaches generally suffer from visually unacceptable noise artifacts. The result of any light transport simulation is, by its very nature, an image of high dynamic range (HDR). This leads to the issues of the display of such images on conventional low dynamic range devices and the development of data compression algorithms to store and recover the corresponding large amounts of detail found in HDR images. This dissertation presents our contributions relevant to these issues. Our contributions to high dynamic range image processing include tone mapping and data compression algorithms. This research proposes and shows the efficacy of a novel level set based tone mapping method that preserves visual details in the display of high dynamic range images on low dynamic range display devices. The level set method is used to extract the high frequency information from HDR images. The details are then added to the range compressed low frequency information to reconstruct a visually accurate low dynamic range version of the image. Additional challenges associated with high dynamic range images include the requirements to reduce excessively large amounts of storage and transmission time. To alleviate these problems, this research presents two methods for efficient high dynamic range image data compression. One is based on the classical JPEG compression. It first converts the raw image into RGBE representation, and then sends the color base and common exponent to classical discrete cosine transform based compression and lossless compression, respectively. The other is based on the wavelet transformation. It first transforms the raw image data into the logarithmic domain, then quantizes the logarithmic data into the integer domain, and finally applies the wavelet based JPEG2000 encoder for entropy compression and bit stream truncation to meet the desired bit rate requirement. We believe that these and similar such contributions will make a wide application of high dynamic range images possible. The contributions to light transport simulation include Monte Carlo noise reduction, dynamic object rendering and complex scene rendering. Monte Carlo noise is an inescapable artifact in synthetic images rendered using stochastic algorithm. This dissertation proposes two noise reduction algorithms to obtain high quality synthetic images. The first one models the distribution of noise in the wavelet domain using a Laplacian function, and then suppresses the noise using a Bayesian method. The other extends the bilateral filtering method to reduce all types of Monte Carlo noise in a unified way. All our methods reduce Monte Carlo noise effectively. Rendering of dynamic objects adds more dimension to the expensive light transport simulation issue. This dissertation presents a pre-computation based method. It pre-computes the surface radiance for each basis lighting and animation key frame, and then renders the objects by synthesizing the pre-computed data in real-time. Realistic rendering of complex scenes is computationally expensive. This research proposes a novel 3D space subdivision method, which leads to a new rendering framework. The light is first distributed to each local region to form local light fields, which are then used to illuminate the local scenes. The method allows us to render complex scenes at interactive frame rates. Rendering has important applications in mixed reality. Consistent lighting and shadows between real scenes and virtual scenes are important features of visual integration. The dissertation proposes to render the virtual objects by irradiance rendering using live captured environmental lighting. This research also introduces a virtual shadow generation method that computes shadows cast by virtual objects to the real background. We finally conclude the dissertation by discussing a number of future directions for rendering research, and presenting our proposed approaches.

Vision has always been one of the most important cognitive tools of human beings. In this regard, the development of image sensors opens up the potential to view objects that our eyes cannot see. One of the most promising capability in some image sensors is their single-photon sensitivity that provides information at the ultimate fundamental limit of light. Time-resolved single-photon avalanche diode (SPAD) image sensors bring a new dimension as they measure the arrival time of incident photons with a precision in the order of hundred picoseconds. In addition to this characteristic, they can be fabricated in complementary metal-oxide-semiconductor (CMOS) technology enabling the integration of complex signal processing blocks at the pixel level. These unique features made CMOS SPAD sensors a prime candidate for a broad spectrum of applications. This thesis is dedicated to the optimization and characterization of quantum imagers based on the SPADs as part of the E.U. funded SUPERTWIN project to surpass the fundamental diffraction limit known as the Rayleigh limit by exploiting the spatio-temporal correlation of entangled photons. The first characterized sensor is a 32×32-pixel SPAD array, named “SuperEllen”, with in-pixel time-to-digital converters (TDC) that measure the spatial cross-correlation functions of a flux of entangled photons. Each pixel features 19.48% fill-factor (FF) in 44.64-μm pitch fabricated in a 150-nm CMOS standard technology. The sensor is fully characterized in several electro-optical experiments, in order to be used in quantum imaging measurements. Moreover, the chip is calibrated in terms of coincidence detection achieving the minimal coincidence window determined by the SPAD jitter. The second developed sensor in the context of SUPERTWIN project is a 224×272-pixel SPAD-based array called “SuperAlice”, a multi-functional image sensor fabricated in a 110-nm CMOS image sensor technology. SuperAlice can operate in multiple modes (time-resolving or photon counting or binary imaging mode). Thanks to the digital intrinsic nature of SPAD imagers, they have an inherent capability to achieve a high frame rate. However, running at high frame rate means high I/O power consumption and thus inefficient handling of the generated data, as SPAD arrays are employed for low light applications in which data are very sparse over time and space. Here, we present three zero-suppression mechanisms to increase the frame rate without adversely affecting power consumption. A row-skipping mechanism that is implemented in both SuperEllen and SuperAlice detects the absence of SPAD activity in a row to increase the duty cycle. A current-based mechanism implemented in SuperEllen ignores reading out a full frame when the number of triggered pixels is less than a user-defined value. A different zero-suppression technique is developed in the SuperAlice chip that is based on jumping through the non-zero pixels within one row. The acquisition of TDC-based SPAD imagers can be speeded up further by storing and processing events inside the chip without the need to read out all data. An on-chip histogramming architecture based on analog counters is developed in a 150-nm CMOS standard technology. The test structure is a 16-bin histogram with 9 bit depth for each bin. SPAD technology demonstrates its capability in other applications such as automotive that demands high dynamic range (HDR) imaging. We proposed two methods based on processing photon arrival times to create HDR images. The proposed methods are validated experimentally with SuperEllen obtaining >130 dB dynamic range within 30 ms of integration time and can be further extended by using a timestamping mechanism with a higher resolution.

This thesis concerns with the introduction to the problematics of images with high dynamic range (HDR) and possibilities of HDR images compression options for display on devices with a low dynamic range (LDR). In the introduction is described historical evolution of recording of reality. It is focusing towards point of view of physics, human visual perception and digital recording. There are described the ways of generating and holding of HDR images. The thesis is corncerned to the techniques of HDR compression, it means the tone-mapping. The different techniques of tone-mapping are explained and specific aproach is targeted to the gradient domain high dynamic range compresion.

Biologically inspired logarithmic CMOS sensors offer high dynamic range imaging capabilities without the difficulties faced by linear imagers. By compressing dynamic range while encoding contrast information, they mimic the human visual system’s response to photo stimuli in fewer bits than those used in linear sensors. Despite this prospect, logarithmic sensors suffer poor image quality due to illumination dependent fixed pattern noise (FPN), making individual pixels appear up to 100 times brighter or darker. This thesis is primarily concerned with alleviating FPN in logarithmic imagers in a simple and convenient way while undertaking a system approach to its origin, distribution and effect on the quality of monochrome and colour images, after FPN correction. Using the properties of the Human visual system, I propose to characterise the errors arising from FPN in a perceptually significant manner by proposing an error measure, never used before. Logarithmic operation over a wide dynamic range is first characterised using a new model; yi j =aj +bj ln(exp sqrt(cj +djxi)−1), where yi j is the response of the sensor to a light stimulus xi and aj, bj, cj and dj are pixel dependent parameters. Using a proposed correction procedure, pixel data from a monochromatic sensor array is FPN corrected to approximately 4% error over 5 decades of illumination even after digitisation - accuracy equivalent to four times the human eyes ability to just notice an illumination difference against a uniform background. By evaluating how error affects colour, the possibility of indiscernible residual colour error after FPN correction, is analytically explored using a standard set of munsell colours. After simulating the simple FPN correction procedure, colour quality is analysed using a Delta E76 perceptual metric, to check for perceptual discrepancies in image colour. It is shown that, after quantisation, the FPN correction process yields 1−2 Delta E76 error units over approximately 5 decades of illumination; colour quality being imperceptibly uniform in this range. Finally, tone-mapping techniques, required to compress high dynamic range images onto the low range of standard screens, have a predominantly logarithmic operation during brightness compression. A new Logr'Gb' colour representation is presented in this thesis, significantly reducing computational complexity, while encoding contrast information. Using a well-known tone mapping technique, images represented in this new format are shown to maintain colour accuracy when the green colour channel is compressed to the standard display range, instead of the traditional luminance channel. The trade off between colour accuracy and computation in this tone mapping approach is also demonstrated, offering a low cost alternative for applications with low display specifications.

<div>Event-Based cameras, also known as neuromophic cameras or dynamic vision sensors, are an imaging modality that attempt to mimic human eyes by asynchronously measuring contrast over time. If the contrast changes sufficiently then a 1-bit event is output, indicating whether the contrast has gone up or down. This stream of events is sparse, and its asynchronous nature allows the pixels to have a high dynamic range and high temporal resolution. However, these events do not encode the intensity of the scene, resulting in an inverse problem to estimate intensity images from the event stream. Hybrid event-based cameras, such as the DAVIS camera, provide a reference intensity image that can be leveraged when estimating the intensity at each pixel during an event. Normally, inverse problems are solved by formulating a forward and prior model and minimizing the associated cost, however, for this problem, the Plug and Play (P&P) algorithm is used to solve the inverse problem. In this case, P&P replaces the prior model subproblem with a denoiser, making the algorithm modular, easier to implement. We propose an idealized forward model that assumes the contrast steps measured by the DAVIS camera are uniform in size to simplify the problem. We show that the algorithm can swiftly reconstruct the scene intensity at a user-specified frame rate, depending on the chosen denoiser’s computational complexity and the selected frame rate.</div>

Within the realm of biosensing, DNA analysis has become an indispensable research tool in medicine, enabling the investigation of relationships among genes, proteins, and drugs. Conventional DNA microarray technology uses multiple lasers and complex optics, resulting in expensive and bulky systems which are not suitable for point-of-care medical diagnostics. The immobilization of DNA probes across the microarray substrate also results in substantial spatial variation. To mitigate the above shortcomings, this thesis presents a set of techniques developed for the CMOS image sensor for point-of-care spectrally-multiplexed fluorescent DNA sensing and other fluorescence biosensing applications. First, a CMOS tunable-wavelength multi-color photogate (CPG) sensor is presented. The CPG exploits the absorption property of a polysilicon gate to form an optical filter, thus the sensor does not require an external color filter. A prototype has been fabricated in a standard 0.35μm digital CMOS technology and demonstrates intensity measurements of blue (450nm), green (520nm), and red (620nm) illumination. Second, a wide dynamic range CMOS multi-color image sensor is presented. An analysis is performed for the wide dynamic-range, asynchronous self-reset with residue readout architecture where photon shot noise is taken into consideration. A prototype was fabricated in a standard 0.35μm CMOS process and is validated in color light sensing. The readout circuit achieves a measured dynamic range of 82dB with a peak SNR of 46.2dB. Third, a low-power CMOS image sensor VLSI architecture for use with comparator based ADCs is presented. By eliminating the in-pixel source follower, power consumption is reduced, compared to the conventional active pixel sensor. A 64×64 prototype with a 10μm pixel pitch has been fabricated in a 0.35μm standard CMOS technology and validated experimentally. Fourth, a spectrally-multiplexed fluorescence contact imaging microsystem for DNA analysis is presented. The microsystem has been quantitatively modeled and validated in the detection of marker gene sequences for spinal muscular atropy disease and the E. coli bacteria. Spectral multiplexing enables the two DNA targets to be simultaneously detected with a measured detection limit of 240nM and 210nM of target concentration at a sample volume of 10μL for the green and red transduction channels, respectively.

Within the realm of biosensing, DNA analysis has become an indispensable research tool in medicine, enabling the investigation of relationships among genes, proteins, and drugs. Conventional DNA microarray technology uses multiple lasers and complex optics, resulting in expensive and bulky systems which are not suitable for point-of-care medical diagnostics. The immobilization of DNA probes across the microarray substrate also results in substantial spatial variation. To mitigate the above shortcomings, this thesis presents a set of techniques developed for the CMOS image sensor for point-of-care spectrally-multiplexed fluorescent DNA sensing and other fluorescence biosensing applications. First, a CMOS tunable-wavelength multi-color photogate (CPG) sensor is presented. The CPG exploits the absorption property of a polysilicon gate to form an optical filter, thus the sensor does not require an external color filter. A prototype has been fabricated in a standard 0.35μm digital CMOS technology and demonstrates intensity measurements of blue (450nm), green (520nm), and red (620nm) illumination. Second, a wide dynamic range CMOS multi-color image sensor is presented. An analysis is performed for the wide dynamic-range, asynchronous self-reset with residue readout architecture where photon shot noise is taken into consideration. A prototype was fabricated in a standard 0.35μm CMOS process and is validated in color light sensing. The readout circuit achieves a measured dynamic range of 82dB with a peak SNR of 46.2dB. Third, a low-power CMOS image sensor VLSI architecture for use with comparator based ADCs is presented. By eliminating the in-pixel source follower, power consumption is reduced, compared to the conventional active pixel sensor. A 64×64 prototype with a 10μm pixel pitch has been fabricated in a 0.35μm standard CMOS technology and validated experimentally. Fourth, a spectrally-multiplexed fluorescence contact imaging microsystem for DNA analysis is presented. The microsystem has been quantitatively modeled and validated in the detection of marker gene sequences for spinal muscular atropy disease and the E. coli bacteria. Spectral multiplexing enables the two DNA targets to be simultaneously detected with a measured detection limit of 240nM and 210nM of target concentration at a sample volume of 10μL for the green and red transduction channels, respectively.

Les technologies CMOS représentent aujourd’hui plus de 90% du marché des capteurs d’images : elles permettent d’intégrer des systèmes intelligents dans une seule puce (SoC = System-On-Chip) et ouvrent la voie à l’intégration d’algorithmes de plus en plus complexes dans les dernières générations de capteurs. Des techniques telles que la reconstruction grande dynamique nécessitent d’acquérir plusieurs images avec un même capteur et de les recombiner. Ces nouvelles contraintes nécessitent d’augmenter drastiquement le débit d’images pour des capteurs de tailles conséquentes (Jusqu'à 30 Mpixels), ainsi que d’augmenter la résolution du convertisseur analogique numérique (jusqu’à 14 bits). Cela crée une demande forte en techniques de conversion analogique-numérique. Ces techniques doivent obéir en même temps aux contraintes de performance notamment la vitesse, la résolution, le faible bruit, la faible consommation et l'intégrabilité mais aussi aux contraintes de qualité d'image impactées directement par la chaine de conversion analogique-numérique en plus de la technologie du pixel. D'ici découle une double problématique pour le sujet:- Etudier et déterminer les limites atteignables en termes de performance sur les différents axes précités.- Gestion du fonctionnement massivement parallèle lié à la structure inhérente des capteurs d'image en vue d'avoir une qualité d'image irréprochable. / CMOS technologies represent nowadays more than 90% of image sensors market given their features namely the possibility of integrating entire intelligent systems on the same chip (SoC = System-On-Chip). Thereby, allowing the implementation of more and more complex algorithms in the new generations of image sensors.New techniques have emerged like high dynamic range reconstruction which requires the acquisition of several images to build up one, thus multiplying the frame rate.These new constraints require a drastic increase of image rate for sensors ofconsiderable size (Up to 30 Mpix and more). At the same time, the ADCresolution has to be increased to be able to extract more details (until 14 bits).With all these demanding specifications, analog-to-digital conversion capabilities have to be boosted as far as possible.These capabilities can be distinguished into two main research axes representing the pillars of the PhD work, namely:+ The study of the reachable limits in terms of performance: Speed, Resolution,Low Noise, Low power consumption and small design pitch.+ The management of the highly parallel operation linked to the structure of animage sensor. Solutions have to be found so as to avoid image artefacts andpreserve the image quality.

Master's thesis is focused on capturing of low dynamic range images using common devices such as camera and its multiple exposure. The main part of thesis is dedicated to composing these images to HDR image, inclusive sequence of images of static scenes, but also dynamic ones. Next part describes tone mapping used for display HDR image on LDR monitors. Moreover, there is given design and implementation of application solving problems mentioned earlier. In the end, the implemented application is evaluated and the possible continuation of this work is stated.

Fyzikálně přijatelné osvětlení v reálném čase je často dosaženo použitím aproximací. Současné metody často aproximují globální osvětlení v prostoru obrazu s využitím schopností moderních grafických karet. Dva techniky z této kategorie, screen-space ambient occlusion a screen-space directional occlusion jsou popsány detailněji v této práci. Screen-space directional occlusion je zobecněná verze screen-space ambient occlusion s podporou jednoho difúzního odrazu a závislostí na směrové informaci světla. Hlavním cílem projektu bylo experimentování s těmito metodami. Pro uniformní distribuci náhodných vzorek pro obě metody byla použita Halton sekvence. Pro potlačení šumu je použita bilaterální filtrace, která bere do úvahy geometrické vlastnosti scény. Metody jsou dál zrychleny použitím nižších rozlišení pro výpočet. Rekonstrukce výsledků do původní velikosti pro vytvoření konečného obrazu je realizována pomoci joint bilateral upsamplingu. Kromě metod globálního osvětlení byly v práci použity aj metody pro mapování stínů a HDR osvětlení.