En imagerie couleur, un système d’acquisition capture une scène avec une haute résolution spatiale mais une résolution spectrale limitée. L’imagerie hyperspectrale permet d’acquérir la scène avec une grande résolution spectrale. Un système d’acquisition hyperspectrale est un ensemble complexe et il est difficile de l’utiliser pour acquérir des données dans une situation où les conditions d’imageries ne sont pas contrôlées. De plus, ces systèmes sont chers et souvent encombrants ou difficiles à manipuler. À cause de ces problèmes, l’utilisation de l’imagerie hyperspectrale n’a pas encore été beaucoup utilisée en vision assistée par ordinateur, et la plupart des systèmes de vision utilise l’imagerie couleur.L’imagerie multispectrale propose une solution intermédiaire, elle permet de capturer une information moins résolue selon la dimension spectrale, comparée à l’hyperspectrale, tout en préservant la résolution spatiale. Ces systèmes sont moins encombrants et moins difficiles à maitriser grâce aux récentes avancées technologiques, et arrivent sur le marché en tant que produits commerciaux. On peut citer les matrices de filtres spectraux (spectral filter arrays) qui permettent l’acquisition en temps réel d’images multispectrales grâce à l’utilisation d’unecaméra de complexité similaire à une caméra couleur. Jusqu’ici, les informations capturées par ces systèmes étaient considérées de la même manière que les imageurs hyperspectraux en champ proche, c’est à dire que pour utiliser l’information au mieux, les conditions d’acquisitions devaient être connues et le système calibré, en particulier pour l’éclairage de la scène et la dynamique de la scène.Afin d’élargir l’utilisation de l’imagerie multispectrale pour la vision par ordinateur dans des conditions générales, je propose dans cette thèse de développer les méthodes calculatoires en imagerie couleur (computational color imaging) et de les adapter aux systèmes d’imagerie multispectraux. Une caractéristique très puissante de l’imagerie couleur est de proposer un rendu constant des couleurs de la surface d’un objet à travers différentes conditions d’acquisition via l’utilisation d’algorithmes et divers traitements de l’information.Dans cette thèse, j’étends la notion de constance des couleurs et de balance des blancs de l’imagerie couleur à l’imagerie multispectrale. J’introduis le terme de constance de l’information spectrale (multispectral constancy).Je propose la construction d’un ensemble d’outils permettant la représentation constante de l’information spectrale à travers le changement d’éclairage. La validité de ces outils est évaluée à travers la reconstruction de la réflectance spectrale des objets lorsque l’éclairage change. Nous avons également acquis de nouvelles images hyperspectrales et multispectrales mises à disposition de la communauté.Ces outils et données permettront de favoriser la généralisation de l’utilisation de l’imagerie multispectrale en champ proche dans les applications classiques utilisant traditionnellement l’imagerie couleur et de sortir ce mode d’imagerie des laboratoires. L’avantage en vision par ordinateur est une meilleure analyse de la réflectance de la surface des objets et donc un avantage certain dans les tâches de classification et d’identification de matériaux. / A conventional color imaging system provides high resolution spatial information and low resolution spectral data. In contrast, a multispectral imaging system is able to provide both the spectral and spatial information of a scene in high resolution. A multispectral imaging system is complex and it is not easy to use it as a hand held device for acquisition of data in uncontrolled conditions. The use of multispectral imaging for computer vision applications has started recently but is not very efficient due to these limitations. Therefore, most of the computer vision systems still rely on traditional color imaging and the potential of multispectral imaging for these applications has yet to be explored.With the advancement in sensor technology, hand held multispectral imaging systems are coming in market. One such example is the snapshot multispectral filter array camera. So far, data acquisition from multispectral imaging systems require specific imaging conditions and their use is limited to a few applications including remote sensing and indoor systems. Knowledge of scene illumination during multispectral image acquisition is one of the important conditions. In color imaging, computational color constancy deals with this condition while the lack of such a framework for multispectral imaging is one of the major limitation in enabling the use of multispectral cameras in uncontrolled imaging environments.In this work, we extend some methods of computational color imaging and apply them to the multispectral imaging systems. A major advantage of color imaging is the ability of providing consistent color of objects and surfaces across varying imaging conditions. In this work, we extend the concept of color constancy and white balancing from color to multispectral images, and introduce the term multispectral constancy.The validity of proposed framework for consistent representation of multispectral images is demonstrated through spectral reconstruction of material surfaces from the acquired images. We have also presented a new hyperspectral reflectance images dataset in this work. The framework of multispectral constancy will make it one step closer for the use of multispectral imaging in computer vision applications, where the spectral information, as well as the spatial information of a surface will be able to provide distinctive useful features for material identification and classification tasks.

No / In the companion study (C. Ripamonti et al., 2004), we present data that measure the effect of surface slant on perceived lightness. Observers are neither perfectly lightness constant nor luminance matchers, and there is considerable individual variation in performance. This work develops a parametric model that accounts for how each observer¿s lightness matches vary as a function of surface slant. The model is derived from consideration of an inverse optics calculation that could achieve constancy. The inverse optics calculation begins with parameters that describe the illumination geometry. If these parameters match those of the physical scene, the calculation achieves constancy. Deviations in the model¿s parameters from those of the scene predict deviations from constancy. We used numerical search to fit the model to each observer¿s data. The model accounts for the diverse range of results seen in the experimental data in a unified manner, and examination of its parameters allows interpretation of the data that goes beyond what is possible with the raw data alone.

Previsualization is a class of techniques for creating approximate previews of a movie sequence in order to visualize a scene prior to shooting it on the set. Often these techniques are used to convey the artistic direction of the story in terms of cinematic elements, such as camera movement, angle, lighting, dialogue, and character motion. Essentially, a movie director uses previsualization (previs) to convey movie visuals as he sees them in his "minds-eye". Traditional methods for previs include hand-drawn sketches, Storyboards, scaled models, and photographs, which are created by artists to convey how a scene or character might look or move. A recent trend has been to use 3D graphics applications such as video game engines to perform previs, which is called 3D previs. This type of previs is generally used prior to shooting a scene in order to choreograph camera or character movements. To visualize a scene while being recorded on-set, directors and cinematographers use a technique called On-set previs, which provides a real-time view with little to no processing. Other types of previs, such as Technical previs, emphasize accurately capturing scene properties but lack any interactive manipulation and are usually employed by visual effects crews and not for cinematographers or directors. This dissertation's focus is on creating a new method for interactive visualization that will automatically capture the on-set lighting and provide interactive manipulation of cinematic elements to facilitate the movie maker's artistic expression, validate cinematic choices, and provide guidance to production crews. Our method will overcome the drawbacks of the all previous previs methods by combining photorealistic rendering with accurately captured scene details, which is interactively displayed on a mobile capture and rendering platform. This dissertation describes a new hardware and software previs framework that enables interactive visualization of on-set post-production elements. A three-tiered framework, which is the main contribution of this dissertation is; 1) a novel programmable camera architecture that provides programmability to low-level features and a visual programming interface, 2) new algorithms that analyzes and decomposes the scene photometrically, and 3) a previs interface that leverages the previous to perform interactive rendering and manipulation of the photometric and computer generated elements. For this dissertation we implemented a programmable camera with a novel visual programming interface. We developed the photometric theory and implementation of our novel relighting technique called Symmetric lighting, which can be used to relight a scene with multiple illuminants with respect to color, intensity and location on our programmable camera. We analyzed the performance of Symmetric lighting on synthetic and real scenes to evaluate the benefits and limitations with respect to the reflectance composition of the scene and the number and color of lights within the scene. We found that, since our method is based on a Lambertian reflectance assumption, our method works well under this assumption but that scenes with high amounts of specular reflections can have higher errors in terms of relighting accuracy and additional steps are required to mitigate this limitation. Also, scenes which contain lights whose colors are a too similar can lead to degenerate cases in terms of relighting. Despite these limitations, an important contribution of our work is that Symmetric lighting can also be leveraged as a solution for performing multi-illuminant white balancing and light color estimation within a scene with multiple illuminants without limits on the color range or number of lights. We compared our method to other white balance methods and show that our method is superior when at least one of the light colors is known a priori.

Cette thèse traite de l’utilisation de la couleur en vision par ordinateur. Deux problèmes sont étudiés : - l’estimation d’illuminants, - la mise en correspondance de descripteurs locaux pour la comparaison d’images couleur. Les surfaces achromatiques renvoient un spectre lumineux ayant la même distribution fréquentielle que le spectre de l’illuminant. Les détecter permet donc de recouvrer l'illuminant. En supposant que l’ensemble des couleurs que peut prendre un illuminant est limité (équation de Planck), il est possible de sélectionner les pixels appartenant à une surface potentiellement grise. Une méthode de vote est alors appliquée, permettant de sélectionner un ou plusieurs illuminants. L’algorithme final possède de nombreux avantages : il est efficace, intuitif, ne nécessite pas de phase d’apprentissage et requiert peu de paramètres, qui s’avèrent stables. De plus, la méthode de vote permet de s’adapter aux cas où plusieurs sources lumineuses éclairent la scène photographiée. Les descripteurs locaux sont des outils puissants pour comparer les images. Cependant, le rôle de la couleur dans l’étape d’appariement a fait l’objet de peu d’études. Le problème principal que nous considérons ici est celui de l’apport de la couleur pour l’appariement de descripteurs locaux. Un usage local de la couleur permet-il de désambiguïser les situations où la luminance seule est insuffisante ? Et si oui, dans quelles proportions ? Nous proposons quatre descripteurs permettant de décrire de manière détaillée le contexte local de points clés dans l’optique de les apparier. Les expériences proposées montrent clairement l’apport de la couleur pour la mise en correspondance locale. / This thesis addresses the use of color in image processing and computer vision. Two problems are studied: - illuminant estimation, - local descriptors matching for color images comparison. Achromatic surfaces are defined as surfaces reflecting a spectrum with the same frequency dsitribution than the illuminant. Consequently, revovering these surfaces allows to estimate the illuminant. Assuming that the range of colors taken by an illuminant is limited (Planck equation), it is possible to select the pixels belonging to a potentially gray surface. A voting procedure is then applied to select one or more illuminants. The proposed algorithm has many advantages: it is effective, intuitive, does not rely on a learning phase and requires only few parameters. In addition, the voting procedure can be adapted to handle cases where multiple light sources of different colors illuminate the scene. Local descriptors are powerful tools to compare images. However, few studies concern the influence of color in the matching step. The main problem that we consider here is the contribution of the color matching of local descriptors. Does the local use of color allow to disambiguate situations where the luminance alone is insufficient ? And if so, how much? We propose four descriptors to precisely describe the local context of key points in the matching step. The main idea developed here is that accurate information can only be obtained by describing both the color distributions and transitions between colors. The many experiences presented clearly show the positive contribution of color to the reliability of the local matching.

The quality of a digital image pipeline relies greatly on its color reproduction which should at a minimum handle the color constancy, and the final judgment of the excellence of the pipeline is made through subjective observations by humans. This dissertation addresses a few topics surrounding the color processing of digital image pipelines from a practical point of view. Color processing fundamentals will be discussed in the beginning to form a background understanding for the topics that follow.A memory color assisted illuminant estimation algorithm is then introduced after a review of memory colors and some modeling techniques. Spectral sensitivity of the camera is required by many color constancy algorithms but such data is often not readily available. To tackle this problem, an alternative method to the spectral characterization for color constancy parameter calibration is proposed. Hue control in color reproduction can be of great importance especially when memory colors are concerned. A hue constrained matrix optimization algorithm is introduced to address this issue, followed by a psychophysical study to systematically arrive at a recommendation for the optimized preferred color reproduction. At the end, a color constancy algorithm for high dynamic range scenes observing multiple illuminants is proposed. / Graduation date: 2012