The development of machine vision algorithms for inspection and machine guidance has traditionally relied on the knowledge and experience of the developers as most of the techniques are based on heuristics and trial and error. This is especially problematic in the area of natural products where variability of the products is the rule rather than the exception. Humans are particularly good in functioning in this arena and in this thesis we look at the development of techniques derived from the functions of the human visual system (HVS). We first identify the significant processes in the HVS and highlight those that we believe are germane to the problems of interest. We then develop computational techniques using these methods and demonstrate their applicability to practical problems. This thesis uses the knowledge that the HVS is considered to consist of three sequential operations (sensing; encoding/transfer; and image interpretation) as a basis for developing a parallel procedure for a machine vision system. We have found that outputs derived from a simulation of the behaviors of receptive fields in the retina and in the higher levels of the brain can generate useful and robust features. Equivalent processes are then developed for machine applications under the guidance of a human operator to identify the areas of interest in the scene for the problem under consideration. Specifically we use the processes for encoding/transfer of data from the retina to develop techniques to enhance color contrasts, and compute color image features that are useful for defect detection and identification in real world images. This is accomplished by a transformation from image space to a characteristic response space that improves the robustness of classification. In this thesis the approach developed is applied to two industrial problems in the quality monitoring of meat and vegetables. The first problem concerns the quality monitoring of breast butterflies and the other the detection of defects on the surface of citrus. The approach is shown to derive algorithms that are robust and can be implemented at high rates of speed. Additionally we also identify a model within which further developments can be conducted as we learn more about the functioning of the HVS.

<p>The trichromatic principle of representing color has for a long time been dominating in color imaging. The reason is the trichromatic nature of human color vision, but as the characteristics of typical color imaging devices are different from those of human eyes, there is a need to go beyond the trichromatic approach. The interest for multi-channel imaging, i.e. increasing the number of color channels, has made it an active research topic with a substantial potential of application.</p><p>To achieve consistent color imaging, one needs to map the imaging-device data to the device-independent colorimetric representations CIEXYZ or CIELAB, the key concept of color management. As the color coordinates depend not only on the reflective spectrum of the object but also on the spectral properties of the illuminant, the colorimetric representation suffers from metamerism, i.e. objects of the same color under a specific illumination may appear different when they are illuminated by other light sources. Furthermore, when the sensitivities of the imaging device differ from the CIE color matching functions, two spectra that appear different for human observers may result in identical device response. On contrary, in multispectral imaging, color is represented by the object’s physical characteristics namely the spectrum which is illuminant independent. With multispectral imaging, different spectra are readily distinguishable, no matter they are metameric or not. The spectrum can then be transformed to any color space and be rendered under any illumination.</p><p>The focus of the thesis is high quality image-acquisition in colorimetric and multispectral formats. The image acquisition system used is an experimental system with great flexibility in illumination and image acquisition setup. Besides the conventional trichromatic RGB filters, the system also provides the possibility of acquiring multi-channel images, using 7 narrowband filters. A thorough calibration and characterization of all the components involved in the image acquisition system is carried out. The spectral sensitivity of the CCD camera, which can not be derived by direct measurements, is estimated using least squares regression, optimizing the camera response to measured spectral reflectance of carefully selected color samples.</p><p>To derive mappings to colorimetric and multispectral representations, two conceptually different approaches are used. In the model-based approach, the physical model describing the image acquisition process is inverted, to reconstruct spectral reflectance from the recorded device response. In the empirical approach, the characteristics of the individual components are ignored, and the functions are derived by relating the device response for a set of test colors to the corresponding colorimetric and spectral measurements, using linear and polynomial least squares regression.</p><p>The results indicate that for trichromatic imaging, accurate colorimetric mappings can be derived by the empirical approach, using polynomial regression to CIEXYZ and CIELAB. Because of the media-dependency, the characterization functions should be derived for each combination of media and colorants. However, accurate spectral data reconstruction requires for multi-channel imaging, using the model-based approach. Moreover, the model-based approach is general, since it is based on the spectral characteristics of the image acquisition system, rather than the characteristics of a set of color samples.</p> / Report code: LiU-TEK-LIC- 2006:70

To reproduce color images in print, the continuous tone image is first transformed into a binary halftone image, producing various colors by discrete dots with varying area coverage. In halftone prints on paper, physical and optical dot gains generally occur, making the print look darker than expected, and making the modeling of halftone color reproduction a challenge. Most available models are based on macroscopic color measurements, averaging the reflectance over an area that is large in relation to the halftone dots. The aim of this study is to go beyond the macroscopic approach, and study halftone color reproduction on a micro-scale level, using high resolution images of halftone prints. An experimental imaging system, combining the accuracy of color measurement instruments with a high spatial resolution, opens up new possibilities to study and analyze halftone color prints. The experimental image acquisition offers a great flexibility in the image acquisition setup. Besides trichromatic RGB filters, the system is also equipped with a set of 7 narrowband filters, for multi-channel images. A thorough calibration and characterization of all the components in the imaging system is described. The spectral sensitivity of the CCD camera, which can not be derived by direct measurements, is estimated using least squares regression. To reconstruct spectral reflectance and colorimetric values from the device response, two conceptually different approaches are used. In the model-based characterization, the physical model describing the image acquisition process is inverted, to reconstruct spectral reflectance from the recorded device response. In the empirical characterization, the characteristics of the individual components are ignored, and the functions are derived by relating the device response for a set of test colors to the corresponding colorimetric and spectral measurements, using linear and polynomial least squares regression techniques. Micro-scale images, referring to images whose resolution is high in relation to the resolution of the halftone, allow for measurements of the individual halftone dots, as well as the paper between them. To capture the characteristics of large populations of halftone dots, reflectance histograms are computed as well as 3D histograms in CIEXYZ color space. The micro-scale measurements reveal that the reflectance for the halftone dots, as well as the paper between the dots, is not constant, but varies with the dot area coverage. By incorporating the varying micro-reflectance in an expanded Murray-Davies model, the nonlinearity caused by optical dot gain can be accounted for without applying the nonphysical exponentiation of the reflectance values, as in the commonly used Yule-Nielsen model. Due to their different intrinsic nature, physical and optical dot gains need to be treated separately when modeling the outcome of halftone prints. However, in measurements of reflection colors, physical and optical dot gains always co-exist, making the separation a difficult task. Different methods to separate the physical and optical dot gain are evaluated, using spectral reflectance measurements, transmission scans and micro-scale images. Further, the relation between the physical dot gain and the halftone dot size is investigated, demonstrated with FM halftones of various print resolutions. The physical dot gain exhibits a clear correlation with the dot size and the dot gain increase is proportional to the increase in print resolution. The experimental observations are followed by discussions and a theoretical explanation.

Within the home environment there exits a host of digital color imaging (DCI) system configurations. The combination of devices and software at the consumer's desktop with devices and services at a remote location (e.g., Print at Kodak), creates a complex interaction of device, contextual, and observer characteristics. In particular, the cathode-ray tube (CRT) display has the potential to influence consumers' perceptions of image quality and their subsequent image manipulation activities. Depending on the inherent color bias and apparent contrast of the CRT, extensive image manipulation may occur, significantly altering the digital values of the image. Output generated by a remote printer will reflect any image manipulation undertaken by the consumer. If manipulation was extensive, what the consumer receives from a remote printer will appear quite different from the softcopy version and thus, may be deemed unacceptable. This research was designed to address the softcopy-hardcopy matching issues that arise from the home DCI system configuration just described. The primary study examined how the CRT display influenced perceived color quality of photographs generated at two points in a DCI system; on-screen photographs (softcopy) and photographic quality prints (hardcopy). CRT gamma, color temperature, and excitation purity were manipulated using an orthogonal, blocked, central composite design. Twenty-two Eastman Kodak Company employees viewed 6 photographs under each of the 15 CRT conditions. Participants rated the color quality of each softcopy photograph, then were given an opportunity to edit color balance, brightness, and contrast for each photograph. The edited photos were printed and rated once again for color quality and acceptability. Results indicated that monitor calibration influenced perceived softcopy color quality, softcopy editing behavior, and subsequent perceived hardcopy color quality. Perception of softcopy color quality ratings was determined predominantly by the CRT gamma level. Participants responded to CRT color balance differences through their editing behavior. In some cases, edits were large enough to significantly and negatively impact perceived hardcopy color quality. Gamma in particular, was the most significant predictor of hardcopy color quality ratings and rejection rates. Additional differences were observed between first- and third-party photographs. Results from this research may be applied to the development of monitor calibration tools, scene balancing algorithms, and software, for the purpose of accommodating consumer image manipulation behavior, in the context of the home DCI system presented herein. / Ph. D.

The trichromatic principle of representing color has for a long time been dominating in color imaging. The reason is the trichromatic nature of human color vision, but as the characteristics of typical color imaging devices are different from those of human eyes, there is a need to go beyond the trichromatic approach. The interest for multi-channel imaging, i.e. increasing the number of color channels, has made it an active research topic with a substantial potential of application. To achieve consistent color imaging, one needs to map the imaging-device data to the device-independent colorimetric representations CIEXYZ or CIELAB, the key concept of color management. As the color coordinates depend not only on the reflective spectrum of the object but also on the spectral properties of the illuminant, the colorimetric representation suffers from metamerism, i.e. objects of the same color under a specific illumination may appear different when they are illuminated by other light sources. Furthermore, when the sensitivities of the imaging device differ from the CIE color matching functions, two spectra that appear different for human observers may result in identical device response. On contrary, in multispectral imaging, color is represented by the object’s physical characteristics namely the spectrum which is illuminant independent. With multispectral imaging, different spectra are readily distinguishable, no matter they are metameric or not. The spectrum can then be transformed to any color space and be rendered under any illumination. The focus of the thesis is high quality image-acquisition in colorimetric and multispectral formats. The image acquisition system used is an experimental system with great flexibility in illumination and image acquisition setup. Besides the conventional trichromatic RGB filters, the system also provides the possibility of acquiring multi-channel images, using 7 narrowband filters. A thorough calibration and characterization of all the components involved in the image acquisition system is carried out. The spectral sensitivity of the CCD camera, which can not be derived by direct measurements, is estimated using least squares regression, optimizing the camera response to measured spectral reflectance of carefully selected color samples. To derive mappings to colorimetric and multispectral representations, two conceptually different approaches are used. In the model-based approach, the physical model describing the image acquisition process is inverted, to reconstruct spectral reflectance from the recorded device response. In the empirical approach, the characteristics of the individual components are ignored, and the functions are derived by relating the device response for a set of test colors to the corresponding colorimetric and spectral measurements, using linear and polynomial least squares regression. The results indicate that for trichromatic imaging, accurate colorimetric mappings can be derived by the empirical approach, using polynomial regression to CIEXYZ and CIELAB. Because of the media-dependency, the characterization functions should be derived for each combination of media and colorants. However, accurate spectral data reconstruction requires for multi-channel imaging, using the model-based approach. Moreover, the model-based approach is general, since it is based on the spectral characteristics of the image acquisition system, rather than the characteristics of a set of color samples. / Report code: LiU-TEK-LIC- 2006:70

Les travaux de cette thèse sont menés dans le cadre d’un projet porté par le centre de recherche LGI2P, (IMT Mines Ales). L’objectif final de ce projet vise à permettre le retour à l’emploi et à améliorer le quotidien des personnes malvoyantes atteintes de rétinite pigmentaire et de glaucome. Le dispositif final est conçu pour aider les personnes dans leurs activités en mobilité : détection d’obstacles, recherche d’indices visuels, en adressant les problèmes liés à l’éblouissement et à l’héméralopie dont souffrent ces utilisateurs potentiels.La contribution de cette thèse au projet se situe sur plusieurs plans. Tout d’abord, il était demandé de définir les paramètres caractérisant la vision résiduelle de chaque utilisateur. En effet, chaque patient a ses propres conditions de confort lumineux, qui dépendent en particulier de l’état d’avancement de sa pathologie : à chaque étape de l’évolution de leur maladie, les patients ont des limites spécifiques de luminosité minimale en dessous de laquelle ils ne perçoivent plus les détails dans une scène, et aussi de la luminosité maximale au-dessus de laquelle ils ressentent gêne et douleur. La définition de ces limitations en luminosité va permettre de paramétrer le dispositif et l’adapter à chaque utilisateur. Mais il n’existe pas de méthode pour mesurer ces niveaux de luminance limite. Nous avons donc participé à la conception et au développement de tests dédiés, et à la réalisation d’essais sur des sujets déficients visuels, dans le cadre d’un essai clinique piloté par le CHU de Nîmes et l’ARAMAV (institut spécialisé dans la rééducation fonctionnelle basse vision), pour l’aspect médical. Nous avons également proposé un nouveau test pour mesurer la sensibilité au contraste chromatique, toujours dans le but de mieux adapter les images affichées à la vision des utilisateurs.Nous avons ensuite travaillé sur la mise au point d’un prototype du dispositif (caméras et visiocasque de réalité virtuelle). Pour cela, nous avons dû choisir les équipements de capture et d’affichage d’images. Un travail de calibration colorimétrique sur ces équipements nous permet de relier grandeurs numériques (code RGB) et grandeurs physiques (luminance et chrominance). Cette étape est nécessaire pour réaliser les tests précités dans des conditions physiquement connues. Elle nous permet également de définir les caractéristiques physiques que devront posséder les équipements qui seront choisis pour réaliser le produit final, s’ils sont différents de ceux utilisés pendant nos travaux.Enfin, nous avons abordé la question des traitements à appliquer au signal capturé par la caméra. Nous avons proposé des traitements en temps réel sur la luminosité dans le but d’augmenter la luminosité dans les zones sombres de l’image et de baisser la luminosité dans les zones qui éblouissent le patient. Nous avons montré les limitations de l’imagerie classique et la nécessité de travailler sur des images HDR (high dynamic range) Nous avons comparé plusieurs méthodes pour permettre l’affichage de ces images HDR sur les écrans de plus faible dynamique, en recherchant les caractéristiques de l’image que ces méthodes doivent préserver au mieux, et en prenant en compte les performances visuelles des utilisateurs potentiels. Nous avons aussi proposé des traitements sur la couleur en augmentant le contraste et la saturation pour rendre les images mieux perceptibles par les patients qui souffrent de troubles de vision des couleurs. / This thesis is part of a project conducted by the LGI2P research center (IMT Mines Ales). The project's final aim is to help people with vision disorders suffering from retinitis pigmentosa and glaucoma get back to work and improve their daily lives. The final device is designed to help people in their mobility activities: detecting obstacles, searching for visual signals, by addressing problems related to dazzling and haemeralopia affecting these potential users.The research of this thesis has several contributions to the project. First of all, parameters characterizing the residual vision of each user had to be defined. Indeed, each patient has his own light comfort conditions, which depend in particular on his pathology's progress: at each stage of the evolution of their disease, patients have specific minimum luminosity limits below which they no longer perceive the details in a scene, and also on the maximum luminosity above which they feel discomfort and pain. The definition of these limitations in luminosity will make it possible to parameterize the device and adapt it to each user. But there is no method to measure these limiting luminance levels. We have therefore participated in the design and development of specialized tests, and in the conduct of trials on visually impaired subjects, as part of a clinical trial led by the Nîmes University Hospital and the ARAMAV (institute specializing in low vision functional rehabilitation), for medical research. We have also proposed a new test to measure sensitivity to chromatic contrast, always with the aim of better adjusting the images displayed to users' vision.Then, we developed a prototype of the device (cameras and virtual reality video headset). In order to achieve these results, we had to choose the image capture and display equipment. A colorimetric calibration work on these equipments allowed us to link digital quantities (RGB code) and physical quantities (luminance and chrominance). This stage is required to perform the above tests under physically known conditions. It also allowed us to define the physical characteristics of the equipment that would be selected to produce the final product, whether they are different from those required during our work.Finally, we discussed the processing to be applied to the signal captured by the camera. We have proposed real-time brightness treatments to increase brightness in dark areas of the image and decrease brightness in areas that dazzle the patient. We have presented the limitations of conventional imaging and the necessity to work on HDR (high dynamic range) images. We have compared several methods to allow the display of these HDR images on screens with lower dynamic range, looking for the image characteristics that these methods should better preserve, and taking into consideration the visual performance of potential users. We have also suggested color treatments by increasing contrast and saturation to make images more perceptible to patients with color vision disorders.

Au cours de ce travail de thèse expérimental, nous nous sommes intéressés à l’étude de la localisation nucléaire et nucléolaire de la protéine de la nucléocapside (NC) du VIH-1. Des études antérieures menées au laboratoire avaient mis en évidence une très forte accumulation de la NC dans les nucléoles. Ce compartiment nucléaire est connu pour être ciblé par de nombreux virus afin de promouvoir leur réplication. Des expériences de microscopie électronique avaient révélé la structure complexe du nucléole et montré qu’il est composé de trois sous-compartiments : les centres fibrillaires, le compartiment fibrillaire dense et le compartiment granulaire dans lesquels se déroule la synthèse des ribosomes. Afin de caractériser la localisation de la NC dans ces trois sous-compartiments, nous avons développé une approche de microscopie optique à haute résolution permettant d’obtenir des images à deux couleurs avec une résolution spatiale améliorée. Pour cela, nous avons mis au point un protocole qui permet d’utiliser simultanément une protéine fluorescente photocommutable et un fluorophore organique introduit par immunomarquage. Après avoir minimisé les aberrations optiques et corrigé les dérives mécaniques inhérentes au montage, nous avons visualisé simultanément la localisation de la NC surexprimée dans des cellules HeLa avec des marqueurs spécifiques des trois sous-compartiments nucléolaires (immunomarquage). La microscopie de fluorescence à haute résolution a permis de résoudre pour la première fois les différents compartiments et de montrer que la NC se localise préférentiellement dans le compartiment granulaire. Finalement, des expériences préliminaires avec des cellules vivantes ont permis de mettre en évidence que la NC est transportée de manière active dans le noyau et qu’elle pourrait interagir directement avec des protéines nucléolaires / During this experimental thesis work, we investigated the nuclear and nucleolar localization of the nucleocapsid protein (NC) of HIV-1. Previous studies performed in our laboratory evidenced a strong accumulation of NC in a subnuclear structure called nucleolus. Playing role in multiple cellular processes, nucleolus is often targeted by viruses to promote their replication. Electron microscopy revealed three nucleolar components (fibrillar centers, dense fibrillar component and granular component) associated to specific steps of the ribosome biogenesis. To characterize the distribution of the NC in these three sub-compartments and therefore shed light on the nucleolar localization of NC during the replication cycle, we developed a high-resolution optical microscopy approach. After having minimized the optical aberrations and corrected the mechanical drifts inherent to the imaging setup, the NC-mEos2 fusion protein overexpressed in HeLa cells was visualized simultaneously with immunolabeled nucleolar markers. The use of high-resolution fluorescence microscopy enabled us to resolve for the first time the three nucleolar compartments and to demonstrate the preferential localization of NC in the granular compartment of nucleolus. Finally, preliminary experiments performed with living cells showed that NC is actively transported in the nucleus and therefore may interact directly with nucleolar proteins.

Color vision deficiency (CVD) is the inability or limited ability to recognize colors and discriminate between them. A person with this condition perceives a narrower range of colors compared to a person with a normal color vision. A growing number of researchers are striving to improve the quality of life for CVD patients. Finding cure, making rectification equipment, providing simulation tools and applying color transformation methods are among the efforts being made by researchers in this field. In this study we concentrate on recoloring digital images in such a way that users with CVD, especially dichromats, perceive more details from the recolored images compared to the original image. The main focus is to give the CVD user a chance to find information within the picture which they could not perceive before. However, this transformed image might look strange or unnatural to users with normal color vision. During this color transformation process, the goal is to keep the overall contrast of the image constant while adjusting the colors that might cause confusion for the CVD user. First, each pixel in the RGB-image is converted to HSV color space in order to be able to control hue, saturation and intensity for each pixel and then safe and problematic hue ranges need to be found. The method for recognizing these ranges was inspired by a condition called “unilateral dichromacy” in which the patient has normal color vision in one eye and dichromacy in another. A special grid-like color card is designed, having constant saturation and intensity over the entire image, while the hue smoothly changes from one block to another to cover the entire hue range. The next step is to simulate the way this color card is perceived by a dichromatic user and finally to find the colors that are perceived identically from two images and the ones that differ too much. This part makes our method highly customizable and we can apply it to other types of CVD, even personalize it for the color vision of a specific observer. The resulting problematic colors need to be dealt with by shifting the hue or saturation based on some pre-defined rules. The results for the method have been evaluated both objectively and subjectively. First, we simulated a set of images as they would be perceived by a dichromat and compared them with simulated view of our transformed images. The results clearly show that our recolored images can eliminate a lot of confusion from user and convey more details. Moreover, an online questionnaire was created and 39 users with CVD confirmed that the transformed images allow them to perceive more information compared to the original images.

Le cerveau est un organe complexe composé de neurones et des cellules non-neuronales. La communication entre les neurones a lieu via les synapses, dont le remodelage morphologique est considéré essentiel pour le traitement et le stockage des informations dans le cerveau des mammifères. Récemment, ce point de vue neuro-centré de la fonction synaptique a évolué, en prenant également en compte les processus gliaux à proximité immédiate de la synapse. Cependant, comme leur structure est bien en deçà de la résolution spatiale de la microscopie optique conventionnelle, les progrès dans les enquêtes dans leur environnement physiologique, le cerveau intact, ont été entravés. En effet, on sait peu sur les variations nanométriques de la morphologie des épines dendritiques et l'interaction avec les processus gliaux, et, finalement, comment elles affectent la transmission synaptique in vivo. Dans cette thèse, nous cherchons à visualiser la dynamique de la nano-morphologie des épines dendritiques et les processus gliaux dans le cortex à tonneaux de souris in vivo. Nous avons donc mis en place l’imagerie super-résolution 2P-STED en temps réel, ce qui permet une haute résolution spatiale et la pénétration profonde des tissus, chez la souris anesthésiée in vivo. Nous montrons que la nano-morphologie des épines est diversifiée, variable, mais globalement stable, et que les différences dans la morphologie des épines peut avoir un effet sur leur compartimentation in vivo. En outre, la mise en œuvre de l’imagerie super-résolution en double couleur in vivo et le développement d'une approche de marquage astrocytaire, nous ont permis de fournir la caractérisation à l'échelle nanométrique des interactions neurone-glie. Ces résultats apportent un aperçu sans précédent dans la dynamique de la synapse à l'échelle nanométrique in vivo, et ouvrent la voie à une meilleure compréhension de la façon dont les réarrangements morphologiques des synapses contribuent à la physiologie du cerveau. / The brain is a complex organ consisting of neurons and non-neuronal cells. Communication between neurons takes place via synapses, whose morphological remodeling is thought to be crucial for information processing and storage in the mammalian brain. Recently, this neuro-centric view of synaptic function has evolved, also taking into account the glial processes in close vicinity of the synapse. However, as their structure is well below the spatial resolution of conventional light microscopy, progress in investigating them in a physiological environment, the intact brain, has been impeded. Indeed, little is known on the nanoscale morphological variations of dendritic spines, the interaction with glial processes, and how these affect synaptic transmission in vivo. Here, we aim to visualize the dynamic nano-morphology of dendritic spines in mouse somatosensory cortex in vivo. We implemented super-resolution 2P-STED time-lapse imaging, which allows for high spatial resolution and deep tissue penetration, in anesthetized mice, and show that the nano-morphology of spines is diverse, variable, but on average stable, and that differences in spine morphology can have an effect on spine biochemical compartmentalization in vivo. Moreover, implementation of dual color in vivo super-resolution imaging and a novel astrocytic labeling approach provided the first steps towards nanoscale characterization of neuron-glia interactions in vivo. These findings bring new insights in synapse dynamics at the nanoscale in vivo, and our methodological endeavors help pave the way for a better understanding of how nanoscale aspects of spine morphology and their dynamics might contribute to brain physiology and animal behavior.