One of the main objectives in digital imaging is to mimic the capabilities of the human eye, and perhaps, go beyond in certain aspects. However, the human visual system is so versatile, complex, and only partially understood that no up-to-date imaging technology has been able to accurately reproduce the capabilities of the it. The extraordinary capabilities of the human eye have become a crucial shortcoming in digital imaging, since digital photography, video recording, and computer vision applications have continued to demand more realistic and accurate imaging reproduction and analytic capabilities. Over decades, researchers have tried to solve the colour constancy problem, as well as extending the dynamic range of digital imaging devices by proposing a number of algorithms and instrumentation approaches. Nevertheless, no unique solution has been identified; this is partially due to the wide range of computer vision applications that require colour constancy and high dynamic range imaging, and the complexity of the human visual system to achieve effective colour constancy and dynamic range capabilities. The aim of the research presented in this thesis is to enhance the overall image quality within an image signal processor of digital cameras by achieving colour constancy and extending dynamic range capabilities. This is achieved by developing a set of advanced image-processing algorithms that are robust to a number of practical challenges and feasible to be implemented within an image signal processor used in consumer electronics imaging devises. The experiments conducted in this research show that the proposed algorithms supersede state-of-the-art methods in the fields of dynamic range and colour constancy. Moreover, this unique set of image processing algorithms show that if they are used within an image signal processor, they enable digital camera devices to mimic the human visual system s dynamic range and colour constancy capabilities; the ultimate goal of any state-of-the-art technique, or commercial imaging device.

La détection de changements dans une scène est l’un des problèmes les plus complexes en télédétection. Il s’agit de détecter des modifications survenues dans une zone géographique donnée par comparaison d’images de cette zone acquises à différents instants. La comparaison est facilitée lorsque les images sont issues du même type de capteur c’est-à-dire correspondent à la même modalité (le plus souvent optique multi-bandes) et possèdent des résolutions spatiales et spectrales identiques. Les techniques de détection de changements non supervisées sont, pour la plupart, conçues spécifiquement pour ce scénario. Il est, dans ce cas, possible de comparer directement les images en calculant la différence de pixels homologues, c’est-à-dire correspondant au même emplacement au sol. Cependant, dans certains cas spécifiques tels que les situations d’urgence, les missions ponctuelles, la défense et la sécurité, il peut s’avérer nécessaire d’exploiter des images de modalités et de résolutions différentes. Cette hétérogénéité dans les images traitées introduit des problèmes supplémentaires pour la mise en œuvre de la détection de changements. Ces problèmes ne sont pas traités par la plupart des méthodes de l’état de l’art. Lorsque la modalité est identique mais les résolutions différentes, il est possible de se ramener au scénario favorable en appliquant des prétraitements tels que des opérations de rééchantillonnage destinées à atteindre les mêmes résolutions spatiales et spectrales. Néanmoins, ces prétraitements peuvent conduire à une perte d’informations pertinentes pour la détection de changements. En particulier, ils sont appliqués indépendamment sur les deux images et donc ne tiennent pas compte des relations fortes existant entre les deux images. L’objectif de cette thèse est de développer des méthodes de détection de changements qui exploitent au mieux l’information contenue dans une paire d’images observées, sans condition sur leur modalité et leurs résolutions spatiale et spectrale. Les restrictions classiquement imposées dans l’état de l’art sont levées grâce à une approche utilisant la fusion des deux images observées. La première stratégie proposée s’applique au cas d’images de modalités identiques mais de résolutions différentes. Elle se décompose en trois étapes. La première étape consiste à fusionner les deux images observées ce qui conduit à une image de la scène à haute résolution portant l’information des changements éventuels. La deuxième étape réalise la prédiction de deux images non observées possédant des résolutions identiques à celles des images observées par dégradation spatiale et spectrale de l’image fusionnée. Enfin, la troisième étape consiste en une détection de changements classique entre images observées et prédites de mêmes résolutions. Une deuxième stratégie modélise les images observées comme des versions dégradées de deux images non observées caractérisées par des résolutions spectrales et spatiales identiques et élevées. Elle met en œuvre une étape de fusion robuste qui exploite un a priori de parcimonie des changements observés. Enfin, le principe de la fusion est étendu à des images de modalités différentes. Dans ce cas où les pixels ne sont pas directement comparables, car correspondant à des grandeurs physiques différentes, la comparaison est réalisée dans un domaine transformé. Les deux images sont représentées par des combinaisons linéaires parcimonieuses des éléments de deux dictionnaires couplés, appris à partir des données. La détection de changements est réalisée à partir de l’estimation d’un code couplé sous condition de parcimonie spatiale de la différence des codes estimés pour chaque image. L’expérimentation de ces différentes méthodes, conduite sur des changements simulés de manière réaliste ou sur des changements réels, démontre les avantages des méthodes développées et plus généralement de l’apport de la fusion pour la détection de changements / Change detection is one of the most challenging issues when analyzing remotely sensed images. It consists in detecting alterations occurred in a given scene from between images acquired at different times. Archetypal scenarios for change detection generally compare two images acquired through the same kind of sensor that means with the same modality and the same spatial/spectral resolutions. In general, unsupervised change detection techniques are constrained to two multiband optical images with the same spatial and spectral resolution. This scenario is suitable for a straight comparison of homologous pixels such as pixel-wise differencing. However, in somespecific cases such as emergency situations, punctual missions, defense and security, the only available images may be of different modalities and of different resolutions. These dissimilarities introduce additional issues in the context of operational change detection that are not addressedby most classical methods. In the case of same modality but different resolutions, state-of-the artmethods come down to conventional change detection methods after preprocessing steps appliedindependently on the two images, e.g. resampling operations intended to reach the same spatialand spectral resolutions. Nevertheless, these preprocessing steps may waste relevant informationsince they do not take into account the strong interplay existing between the two images. The purpose of this thesis is to study how to more effectively use the available information to work with any pair of observed images, in terms of modality and resolution, developing practicalcontributions in a change detection context. The main hypothesis for developing change detectionmethods, overcoming the weakness of classical methods, is through the fusion of observed images. In this work we demonstrated that if one knows how to properly fuse two images, it is also known how to detect changes between them. This strategy is initially addressed through a change detection framework based on a 3-step procedure: fusion, prediction and detection. Then, the change detection task, benefiting from a joint forward model of two observed images as degradedversions of two (unobserved) latent images characterized by the same high spatial and highspectral resolutions, is envisioned through a robust fusion task which enforces the differencesbetween the estimated latent images to be spatially sparse. Finally, the fusion problem isextrapolated to multimodal images. As the fusion product may not be a real quantity, the process is carried out by modelling both images as sparse linear combinations of an overcomplete pair of estimated coupled dictionaries. Thus, the change detection task is envisioned through a dual code estimation which enforces spatial sparsity in the difference between the estimated codes corresponding to each image. Experiments conducted in simulated realistically and real changes illustrate the advantages of the developed method, both qualitatively and quantitatively, proving that the fusion hypothesis is indeed a real and effective way to deal with change detection

Die Detektion des Pankreaskarzinoms oder die Unterscheidung zwischen einem Karzinom und einer chronischen Pankreatitis bleibt ein diagnostisches Problem. Jedes bisher eingesetzte bildgebende Verfahren hat seine Vor- und Nachteile. Die Bildfusion macht sich die verschiedenen Stärken und Schwächen der einzelnen Bildgebungen zum Vorteil und kombiniert diese mit dem Ziel, die Stärken zu addieren und die Schwächen zu reduzieren. In dieser Arbeit werden Bilder der CT und der Positronenemissionstomographie miteinander kombiniert, um sowohl eine hohe anatomische Auflösung zu erreichen als auch auf Stoffwechseleigenschaften des Gewebes zurückgreifen zu können. Die Auswertung ergab, dass die Bildfusion hinsichtlich der Karzinomdetektion der CT und der PET überlegen ist. Auch für Umgebungsinfiltration, Lymphknotenbefall und Fernmetastasen ergab die Auswertung bessere Ergebnisse als für die jeweiligen Einzelmodalitäten. Die Bildfusion kann die Akkuratheit der verschiedenen Untersuchungen verbessern und ohne technisch großen Aufwand wie auch ohne weitere Patientenbelastung um wertvolle Information ergänzen. Damit stellt die Bildfusion eine wertvolle Methode zur Erkennung und Visualisierung vor allem kleiner und schwer zu lokalisierender Neoplasien dar. Ebenfalls bietet sie eine genaue Möglichkeit zur Biopsie- und Bestrahlungsplanung. / The differentiation between pancreatic carcinoma and inflammatory pancreatic diseases remains a diagnostic problem. Every imaging method available has its own lack of information. Image-fusion uses the strength and the disadvantage of each imaging method. Trough the combination of the images advantages are combined whereas the disadvantages are reduced. In this study images of CT and PET were fused to archive a high anatomic resolution and the information about tissue metabolism. The evaluation showed the superiority of fused images in comparison to the single modalities regarding the detection of lesions, the infiltration of surrounding tissue and lymph nodes as well as detection of distant metastases. Image fusion improves accuracy of CT and PET without huge technical effort or patient-strain while providing useful accessory information. For this reason image-fusion presents a valuable method for detection and visualisation of particularly small lesions in early tumor stage. Furthermore it offers administrable information for biopsy and radiation-planning.

In this thesis, a modified 2D Discrete Cosine Transform based electro-optic and IR image fusion algorithm is proposed and implemented on an FPGA platform. The platform is a custom FPGA board which uses ALTERA Stratix III family FPGA. The algorithm is also compared with state of the art image fusion algorithms by means of an image fusion software application GUI developed in Matlab&reg / . The proposed algorithm principally takes corresponding 4x4 pixel blocks of two images to be fused and transforms them by means of 2D Discrete Cosine Transform. Then, the L2 norm of each block is calculated and used as the weighting factor for the AC values of the fused image block. The DC value of the fused block is the arithmetic mean of the DC coefficients of both input blocks. Based on this mechanism, the whole two images are processed in such a way that the output image is a composition of the processed 4x4 blocks. The proposed algorithm performs well compared to the other state of the art image fusion algorithms both in subjective and objective quality evaluations. In hardware, v the implemented algorithm can accept input videos as fast as 65 MHz pixel clock with a resolution of 1024x768 @60 Hz.

Le but de cette thèse est de construire une image couleur de haute qualité, contenant un faible niveau de bruit et d'aliasing, à partir d'une grande séquence (e.g. des centaines) d'images RAW prises avec un appareil photo grand public. C’est un problème complexe nécessitant d'effectuer à la volée du dématriçage, du débruitage et de la super-résolution. Les algorithmes existants produisent des images de haute qualité, mais le nombre d'images d'entrée est limité par des coûts de calcul et de mémoire importants. Dans cette thèse, nous proposons un algorithme de fusion d'images qui les traite séquentiellement de sorte que le coût mémoire ne dépend que de la taille de l'image de sortie. Après un pré-traitement, les images mosaïquées sont recalées en utilisant une méthode en deux étapes que nous introduisons. Ensuite, une image couleur est calculée par accumulation des données irrégulièrement échantillonnées en utilisant une régression à noyau classique. Enfin, le flou introduit est supprimé en appliquant l'inverse du filtre équivalent asymptotique correspondant (que nous introduisons). Nous évaluons la performance et la précision de chaque étape de notre algorithme sur des données synthétiques et réelles. Nous montrons que pour une grande séquence d'images, notre méthode augmente avec succès la résolution et le bruit résiduel diminue comme prévu. Nos résultats sont similaires à des méthodes plus lentes et plus gourmandes en mémoire. Comme la génération de données nécessite une méthode d'interpolation, nous étudions également les méthodes d'interpolation par polynôme trigonométrique et B-spline. Nous déduisons de cette étude de nouvelles méthodes d'interpolation affinées / The aim of this thesis is to build a high-quality color image, containing a low level of noise and aliasing, from a large sequence (e.g. hundreds or thousands) of RAW images taken with a consumer camera. This is a challenging issue requiring to perform on the fly demosaicking, denoising and super-resolution. Existing algorithms produce high-quality images but the number of input images is limited by severe computational and memory costs. In this thesis we propose an image fusion algorithm that processes the images sequentially so that the memory cost only depends on the size of the output image. After a preprocessing step, the mosaicked (or CFA) images are aligned in a common system of coordinates using a two-step registration method that we introduce. Then, a color image is computed by accumulation of the irregularly sampled data using classical kernel regression. Finally, the blur introduced is removed by applying the inverse of the corresponding asymptotic equivalent filter (that we introduce).We evaluate the performance and the accuracy of each step of our algorithm on synthetic and real data. We find that for a large sequence of RAW images, our method successfully performs super-resolution and the residual noise decreases as expected. We obtained results similar to those obtained by slower and memory greedy methods. As generating synthetic data requires an interpolation method, we also study in detail the trigonometric polynomial and B-spline interpolation methods. We derive from this study new fine-tuned interpolation methods

Imagens da anatomia interna são essenciais para as práticas médicas. Estabelecer correlação entre elas, é um importante procedimento para diagnóstico e tratamento. Nessa dissertação, é proposta uma abordagem para correlacionar dados multidimensionais de mesma modalidade de aquisição baseando-se somente nas informações de intensidade de pixels e voxels. O trabalho foi dividido em duas fases de implementação. Na primeira, foi explorado o problema de similaridade entre imagens médicas usando a perspectiva de análise de qualidade de imagem. Isso levou ao desenvolvimento de uma técnica de dois passos que estabelece um equilíbrio entre a velocidade de processamento e precisão de duas abordagens conhecidas. Avaliou-se a qualidade e aplicabilidade do algoritmo e, na segunda fase, o método foi estendido para analisar similaridade e encontrar a localização de uma imagem arbitrária (2D) em um volume (3D). A solução minimiza o número virtualmente infinito de possíveis orientações transversais e usa otimizações para reduzir a carga de trabalho e entregar resultados precisos. Uma visualização tridimensional volumétrica funde o volume (3D) com a imagem (2D) estabelecendo uma correspondência entre os dados. Uma análise experimental demonstrou que, apesar da complexidade computacional do algoritmo, o uso de amostragem, tanto na imagem quanto no volume, permite alcançar um bom equilíbrio entre desempenho e precisão, mesmo quando realizada com conjuntos de dados de baixa intensidade de gradiente. / Images of the inner anatomy are essential for clinical practice. To establish a correlation between them is an important procedure for diagnosis and treatment. In this thesis, we propose an approach to correlate within-modality 2D and 3D data from ordinary acquisition protocols based solely on the pixel/voxel information. The work was divided into two development phases. First, we explored the similarity problem between medical images using the perspective of image quality assessment. It led to the development of a 2-step technique that settles the compromise between processing speed and precision of two known approaches. We evaluated the quality and applicability of the 2-step and, in the second phase, we extended the method to use similarity analysis to, given an arbitrary slice image (2D), find the location of this slice within the volume data (3D). The solution minimizes the virtually infinite number of possible cross section orientations and uses optimizations to reduce the computational workload and output accurate results. The matching is displayed in a volumetric three-dimensional visualization fusing the 3D with the 2D. An experimental analysis demonstrated that despite the computational complexity of the algorithm, the use of severe data sampling allows achieving a great compromise between performance and accuracy even when performed with low gradient intensity datasets.

Imagens da anatomia interna são essenciais para as práticas médicas. Estabelecer correlação entre elas, é um importante procedimento para diagnóstico e tratamento. Nessa dissertação, é proposta uma abordagem para correlacionar dados multidimensionais de mesma modalidade de aquisição baseando-se somente nas informações de intensidade de pixels e voxels. O trabalho foi dividido em duas fases de implementação. Na primeira, foi explorado o problema de similaridade entre imagens médicas usando a perspectiva de análise de qualidade de imagem. Isso levou ao desenvolvimento de uma técnica de dois passos que estabelece um equilíbrio entre a velocidade de processamento e precisão de duas abordagens conhecidas. Avaliou-se a qualidade e aplicabilidade do algoritmo e, na segunda fase, o método foi estendido para analisar similaridade e encontrar a localização de uma imagem arbitrária (2D) em um volume (3D). A solução minimiza o número virtualmente infinito de possíveis orientações transversais e usa otimizações para reduzir a carga de trabalho e entregar resultados precisos. Uma visualização tridimensional volumétrica funde o volume (3D) com a imagem (2D) estabelecendo uma correspondência entre os dados. Uma análise experimental demonstrou que, apesar da complexidade computacional do algoritmo, o uso de amostragem, tanto na imagem quanto no volume, permite alcançar um bom equilíbrio entre desempenho e precisão, mesmo quando realizada com conjuntos de dados de baixa intensidade de gradiente. / Images of the inner anatomy are essential for clinical practice. To establish a correlation between them is an important procedure for diagnosis and treatment. In this thesis, we propose an approach to correlate within-modality 2D and 3D data from ordinary acquisition protocols based solely on the pixel/voxel information. The work was divided into two development phases. First, we explored the similarity problem between medical images using the perspective of image quality assessment. It led to the development of a 2-step technique that settles the compromise between processing speed and precision of two known approaches. We evaluated the quality and applicability of the 2-step and, in the second phase, we extended the method to use similarity analysis to, given an arbitrary slice image (2D), find the location of this slice within the volume data (3D). The solution minimizes the virtually infinite number of possible cross section orientations and uses optimizations to reduce the computational workload and output accurate results. The matching is displayed in a volumetric three-dimensional visualization fusing the 3D with the 2D. An experimental analysis demonstrated that despite the computational complexity of the algorithm, the use of severe data sampling allows achieving a great compromise between performance and accuracy even when performed with low gradient intensity datasets.

Imagens da anatomia interna são essenciais para as práticas médicas. Estabelecer correlação entre elas, é um importante procedimento para diagnóstico e tratamento. Nessa dissertação, é proposta uma abordagem para correlacionar dados multidimensionais de mesma modalidade de aquisição baseando-se somente nas informações de intensidade de pixels e voxels. O trabalho foi dividido em duas fases de implementação. Na primeira, foi explorado o problema de similaridade entre imagens médicas usando a perspectiva de análise de qualidade de imagem. Isso levou ao desenvolvimento de uma técnica de dois passos que estabelece um equilíbrio entre a velocidade de processamento e precisão de duas abordagens conhecidas. Avaliou-se a qualidade e aplicabilidade do algoritmo e, na segunda fase, o método foi estendido para analisar similaridade e encontrar a localização de uma imagem arbitrária (2D) em um volume (3D). A solução minimiza o número virtualmente infinito de possíveis orientações transversais e usa otimizações para reduzir a carga de trabalho e entregar resultados precisos. Uma visualização tridimensional volumétrica funde o volume (3D) com a imagem (2D) estabelecendo uma correspondência entre os dados. Uma análise experimental demonstrou que, apesar da complexidade computacional do algoritmo, o uso de amostragem, tanto na imagem quanto no volume, permite alcançar um bom equilíbrio entre desempenho e precisão, mesmo quando realizada com conjuntos de dados de baixa intensidade de gradiente. / Images of the inner anatomy are essential for clinical practice. To establish a correlation between them is an important procedure for diagnosis and treatment. In this thesis, we propose an approach to correlate within-modality 2D and 3D data from ordinary acquisition protocols based solely on the pixel/voxel information. The work was divided into two development phases. First, we explored the similarity problem between medical images using the perspective of image quality assessment. It led to the development of a 2-step technique that settles the compromise between processing speed and precision of two known approaches. We evaluated the quality and applicability of the 2-step and, in the second phase, we extended the method to use similarity analysis to, given an arbitrary slice image (2D), find the location of this slice within the volume data (3D). The solution minimizes the virtually infinite number of possible cross section orientations and uses optimizations to reduce the computational workload and output accurate results. The matching is displayed in a volumetric three-dimensional visualization fusing the 3D with the 2D. An experimental analysis demonstrated that despite the computational complexity of the algorithm, the use of severe data sampling allows achieving a great compromise between performance and accuracy even when performed with low gradient intensity datasets.

Cette thèse aborde les principaux aspects applicatifs en matière de restauration et amélioration d'images. A travers une approche progressive, deux nouveaux paradigmes sont introduits : la mise en place d'une déconvolution et d'un débruitage simultanés avec une amélioration de cohérence, et la fusion avec débruitage. Ces paradigmes sont définis dans un cadre théorique d'approches EDP - variationnelles. Le premier paradigme représente une étape intermédiaire dans la validation et l'analyse du concept de restauration et d'amélioration combinées, tandis que la deuxième proposition traitant du modèle conjoint fusion-débruitage illustre les avantages de l'utilisation d'une approche parallèle en traitement d'images, par opposition aux approches séquentielles. Ces deux propositions sont théoriquement et expérimentalement formalisées, analysées et comparées avec les approches les plus classiques, démontrant ainsi leur validité et soulignant leurs caractéristiques et avantages. / This thesis addresses key issues of current image restoration and enhancement methodology, and through a progressive approach introduces two new image processing paradigms, i.e., concurrent image deblurring and denoising with coherence enhancement, and joint image fusion and denoising, defined within a Partial Differential Equation -variational theoretical setting.The first image processing paradigm represents an intermediary step in validating and testing the concept of compound image restoration and enhancement, while the second proposition, i.e., the joint fusion-denoising model fully illustrates the advantages of using concurrent image processing as opposed to sequential approaches.Both propositions are theoretically formalized and experimentally analyzed and compared with the similar existing methodology, proving thus their validity and emphasizing their characteristics and advantages when considered an alternative to a sequential image processing chain.

S rychlým rozvojem technologie multispektrálního zobrazování v posledních desetiletích obrázky získané zobrazovacími systémy obsahují nejen barevná pásma RGB v každodenním životě, ale také mají multispektrální barevná pásma a vysoké prostorové rozlišení v multispektrálních obrazových datech. Díky tomu obrázky obsahují bohaté informace o charakteristických cílových oblastech. Fúze obrazu je také důležitou větví v oblasti zpracování obrazu, kde je více obrázků ze stejné oblasti ve stejné výšce sloučeno do jednoho obrazu. Poté se zlepší korelace mezi spektrálními informacemi multispektrálních obrazů. Aby se informace na obrázku neztratily. Tato práce obsahuje popis návrhu a implementace multispektrálního obrazového systému, předzpracování multispektrálních obrazů, fúzi multispektrálních obrazů a analýzu hlavních komponent. Nakonec je představeno hodnocení celého systému.