<p dir="ltr">This dissertation presents four distinct studies in the fields of image processing and machine learning, focusing on applications ranging from quality assessment for raster images in scanned document and virtual reality facial expression tracking to compression for continual learning and food image classification. First, we shift the traditional focus of image quality assessment (IQA) from natural images to scanned documents, proposing a machine learning-based classification method to evaluate the visual quality of scanned raster images. We enhance the classifier's performance using augmented data generated through noise models simulating scanning degradation. Second, we address the challenges of virtual facial animation in immersive VR, developing a domain adversarial training model to generate domain invariant features and combined it with manifold learning methods for accurate facial action unit (AU) intensity estimation from partially occluded facial images. Third, we explore the use of image compression to increase buffer capacity in continual machine learning systems, thereby enhancing exemplar diversity and mitigating catastrophic forgetting. Our approach includes a new framework that selects compression rate and algorithm, showing significant improvements in image classification accuracy on the CIFAR-100 and ImageNet datasets. Finally, we combine class-activation maps with neural image compression in food image classification systems to adapt to continuously evolving data, extending buffer size and enhancing data diversity, which is validated on food-specific datasets and shows potential for broader applications in continual machine learning systems. Together, these studies demonstrate the versatility of image processing and machine learning techniques in addressing complex and varied challenges across different domains.</p>

<p>Osnovni doprinos ove doktorske disertacije je razvoj algortima i sistema za objektivnu procenu vizuelnog kvaliteta slike uzimajući u obzir najvažnija moguća oštećenja kao što su zamućenje ivica (oštrina) i poremećaj prirodnog izgleda teksture objekata na slici sa jedne strane i uticaj sadržaja slike (procenta ivica u slici) na procenu kvaliteta sa druge strane. Dakle, hipoteza izneta u ovom radu je da je potreban multiparametarski pristup da bi se dobila objektivna procena kvaliteta slike koja je što približnija subjektivnoj proceni.</p>

Thesis (PhD (Electronic Engineering))--University of Stellenbosch, 2009. / The downloading of images is a bottleneck in the image acquisition chain for low earth orbit, remote sensing satellites. An on-board image quality assessment system could optimise use of available downlink time by prioritising images for download, based on their quality. An image quality assessment system based on measuring image degradations is proposed. Algorithms for estimating degradations are investigated. The degradation types considered are cloud cover, additive sensor noise and the defocus extent of the telescope. For cloud detection, the novel application of heteroscedastic discriminant analysis resulted in better performance than comparable dimension reducing transforms from remote sensing literature. A region growing method, which was previously used on-board a micro-satellite for cloud cover estimation, is critically evaluated and compared to commonly used thresholding. The thresholding method is recommended. A remote sensing noise estimation algorithm is compared to a noise estimation algorithm based on image pyramids. The image pyramid algorithm is recommended. It is adapted, which results in smaller errors. A novel angular spectral smoothing method for increasing the robustness of spectral based, direct defocus estimation is introduced. Three existing spectral based defocus estimation methods are compared with the angular smoothing method. An image quality assessment model is developed that models the mapping of the three estimated degradation levels to one quality score. A subjective image quality evaluation experiment is conducted, during which more than 18000 independent human judgements are collected. Two quality assessment models, based on neural networks and splines, are tted to this data. The spline model is recommended. The integrated system is evaluated and image quality predictions are shown to correlate well with human quality perception.

Diagnostic radiology plays an important role in the assessment and treatment of neonates, mainly premature, requiring intensive care in the Special Baby Care Unit (SBCU), because they have highly mitotic state of their cells and thus they are more radiosensitive. It is often necessary to perform a large number of X-ray examinations depending upon the infant's birth-weight, gestational age and respiratory problems. It is therefore important to ensure that radiation doses from radiographic examinations carried out in neo-natal units are kept to a minimum while maintaining the quality of radiographic images in a high level. An optimization study on radiation dose and image quality in neonatal radiography is presented. Our sample consists of 135 neonatal radiographic examinations, which performed on 54 neonates. All examinations were performed using the same mobile unit and under manual exposure control. Neonates were categorized into four groups depending on birth-weight. ESD was estimated from the exposure parameters (kVp, mAs) and tube output and also with using of Dose-Area Product (DAP). For the evaluation of image quality, our sample consists of 195 images (75 screen film images, 60 CR images in printed form and 60 CR images in electronic form) were assessed by two observers and were based on the visibility of certain anatomical features using a five-grade scale. ESDoutput values increased with increasing weight and ranged from 16.8 μGy to 64.7 μGy, with a mean value of 36 μGy for all radiographs. Similarly, ESDDAP values ranged from 14.8 μGy to 48.5 μGy with a mean of 29 μGy. Analyzing, the mean ESD for CR images was found 34.8 μGy and for screen film images 36.9 μGy. ESD values for CR images have the same behavior as ESD values for SF images. However, the majority of the acquired values are lower than the proposed Dose Reference Levels by Commission of European Communities (CEC: 80 μGy) and National Radiological Protection Board (NRPB: 50 μGy). Image quality evaluation revealed the feasibility of achieving a diagnostically satisfactory image using both low and high tube voltage techniques, with the latter resulting in reduced ESDs. The results suggest that the use of high tube voltage techniques could result in further reductions in neonatal dose, without image quality degradation, underlying the requirement for establishing standard examination protocols for neonatal radiography with respect to neonatal weight. / Η Διαγνωστική ακτινολογία παίζει σημαντικό ρόλο στην αξιολόγηση και τη θεραπεία των νεογνών, κυρίως των πρόωρων, καθώς απαιτείται η εντατική φροντίδα τους στην Ειδική Μονάδα Φροντίδας Νεογνών, λόγω της μεγάλης μιτωτικής δραστηριότητας των κυττάρων τους και κατα συνέπεια της ακτινοευαισθησίας τους. Είναι συχνά απαραίτητο να πραγματοποιηθεί ένας μεγάλος αριθμός ακτινογραφικών εξετάσεων που εξαρτώνται από το βάρος γέννησης, την περίοδο κύησης και τα αναπνευστικά προβλήματα. Επομένως, είναι σημαντικό να εξασφαλιστεί ότι οι δόσεις ακτινοβολίας από ακτινογραφικές εξετάσεις που πραγματοποιούνται σε μονάδες νεογνών περιορίζονται στο ελάχιστο, ενώ η ποιότητα των ακτινογραφικών εικόνων διατηρείται σε υψηλά επίπεδα. Μια μελέτη για τη βελτιστοποίηση της δόσης της ακτινοβολίας και της ποιότητας της εικόνας σε ακτινογραφίες νεογνών παρουσιάζεται παρακάτω. Το δείγμα μας αποτελείται από 135 ακτινογραφικές εξετάσεις νεογνών, οι οποίες πραγματοποιήθηκαν σε 54 νεογνά. Όλες οι εξετάσεις πραγματοποιήθηκαν χρησιμοποιώντας την ίδια φορητή ακτινογραφική μονάδα και με χειροκίνητο έλεγχο έκθεσης. Τα νεογνά ταξινομήθηκαν σε τέσσερις ομάδες ανάλογα με το βάρος γέννησης. Η Επιφανειακή δόση (ESD) εκτιμήθηκε από τις παραμέτρους της έκθεσης (kVp, mΑs), αλλά και με τη χρήση του DAP. Για την αξιολόγηση της ποιότητας της εικόνας, το δείγμα αποτελούνταν από 195 εικόνες (75 συμβατικές, 60 ψηφιακές (CR) σε έντυπη μορφή και 60 ψηφιακές εικόνες (CR) σε ηλεκτρονική μορφή) οι οποίες αξιολογήθηκαν από δύο παρατηρητές και βασίστηκαν στην ορατότητα ορισμένων ανατομικών χαρακτηριστικών χρησιμοποιώντας μια πενταβάθμια κλίμακα. Οι ESDoutput τιμές αυξάνονται με την αύξηση του βάρους και κυμαίνονται από 16.8μGy σε 64.7μGy, με μέση τιμή 36μGy για όλες τις ακτινογραφίες. Ομοίως, οι ESDDAP τιμές κυμαίνονται από 14.8 μGy σε 48.5 μGy, με μέση τιμή 29 μGy. Αναλυτικά, η μέση τιμή ESD για τις ψηφιακές (CR) εικόνες βρέθηκε 34.8μGy και για τις συμβατικές 36.9μGy. Η ESD για CR εικόνες έχει στατιστικά την ίδια συμπεριφορά με την ESD για SF εικόνες. Η πλειοψηφία των αποκτηθέντων τιμών είναι χαμηλότερες από τα Διαγνωστικά Επίπεδα Αναφοράς που έχουν προταθεί από την Επιτροπή της Ευρωπαϊκής Ένωσης (CEC: 80μGy) και το National Radiological Protection Board (NRPB: 50μGy) για προσθοπίσθιες ακτινογραφίες θώρακος νεογνών. Η αξιολόγηση της ποιότητας της εικόνας αποκάλυψε την δυνατότητα επίτευξης μιας διαγνωστικά ικανοποιητικής εικόνας χρησιμοποιώντας τόσο χαμηλές όσο και υψηλές τάσεις, με τις τελευταίες να οδηγούν σε μείωση των επιφανειακών δόσεων (ESDs). Τα αποτελέσματα δείχνουν ότι η χρήση τεχνικών υψηλής τάσης μπορούν να οδηγήσουν σε περαιτέρω μείωση των δόσεων στα νεογνά, χωρίς να υποβαθμίζεται η ποιότητα της εικόνας, τα οποία βασίζονται στην απαίτηση καθορισμού τυποποιημένων πρωτοκόλλων εξέτασης για ακτινογραφίες σε νεογνά σε σχέση με το βάρος τους.

In our society with its aging population, the design and implementation of a highperformance distributed multi-sensor and information system for autonomous physical services become more and more important. In line with this, this thesis proposes an Intelligent Multi-Sensor System, IMSS, that surveys a human activities space to detect and identify a target for a specific service. The subject of this thesis covers three main aspects related to the set-up of an IMSS: an improved depth measurement and reconstruction method and its related uncertainty, a surveillance and tracking algorithm and finally a way to validate and evaluate the proposed methods and algorithms. The thesis discusses how a model of the depth spatial quantisation uncertainty can be implemented to optimize the configuration of a sensor system to capture information of the target objects and their environment with required specifications. The thesis introduces the dithering algorithm which significantly reduces the depth reconstruction uncertainty. Furthermore, the dithering algorithm is implemented on a sensor-shifted stereo camera, thus simplifying depth reconstruction without compromising the common stereo field of view. To track multiple targets continuously, the Gaussian Mixture Probability Hypothesis Density, GM-PHD, algorithm is implemented with the help of vision and Radio Frequency Identification, RFID, technologies. The performance of the tracking algorithm in a vision system is evaluated by a circular motion test signal. The thesis introduces constraints to the target space, the stereo pair characteristics and the depth reconstruction accuracy to optimize the vision system and to control the performance of surveillance and 3D reconstruction through integer linear programming. The human being within the activity space is modelled as a tetrahedron, and a field of view in spherical coordinates are used in the control algorithms. In order to integrate human behaviour and perception into a technical system, the proposed adaptive measurement method makes use of the Fuzzily Defined Variable, FDV. The FDV approach enables an estimation of the quality index based on qualitative and quantitative factors for image quality evaluation using a neural network. The thesis consists of two parts, where Part I gives an overview of the applied theory and research methods used, and Part II comprises the eight papers included in the thesis.

Contexte : Le cancer du sein est le cancer le plus fréquent et le deuxième cancer le plus mortel chez la femme. Les techniques d'imagerie constituent un élément essentiel pour le dépistage, le diagnostic, la stadification et le traitement du cancer du sein. L'imagerie par résonance magnétique avec injection de produit de contraste (CE-MRI) est actuellement la technique d'imagerie standard pour la détection du développement vasculaire anormal et des prises de contraste des lésions mammaires. CE-MRI est cependant très coûteuse et peu disponible. De plus, sa résolution spatiale pourrait être insuffisante pour la détection de certains types de lésions, et ne permet pas d'imager les amas de microcalcifications. Le développement de l'angiomammographie double-énergie (CESM) a permis l'utilisation des produits de contraste intraveineux en clinique avec des appareils conventionnels de mammographie. Cependant, CESM est une technique de projection 2D et présente, par conséquence, des limites pour décrire la structure 3D interne des lésions et pour fournir une information fonctionnelle 3D précise.La tomosynthèse numérique du sein avec injection de produit de contraste (CE-DBT) et le scanner dédié du sein avec injection de produit de contraste (CE-bCT) sont deux techniques d'imagerie actuellement en investigation par des groupes de recherche académiques et industriels. Il est cependant anticipé que le potentiel quantitatif de la CE-DBT soit limité, en raison de la faible résolution en profondeur due à l'ouverture angulaire limitée de la DBT. CE-bCT, avec sa résolution spatiale quasi-isotrope et son intensité de signal proportionnelle au coefficient d'atténuation linéaire, est supposée offrir une information quantitative plus précise, bien qu'une utilisation à faible dose de radiation reste toujours un défi.Objectifs : L'objectif de cette thèse a été d'étudier la faisabilité de la méthode CE-bCT et sa capacité à détecter et localiser des tumeurs vascularisées, ainsi que d'offrir de l'information morphologique et fonctionnelle précise sur les tumeurs. Pour comprendre la valeur ajoutée de la CE-bCT par rapport à CE-DBT, le potentiel quantitatif des deux méthodes a également été comparé. Nos études ont été réalisées grâce à des simulations par ordinateur, validées par des mesures expérimentales.Méthodes : Dans un premier temps, une plateforme de simulation capable de modéliser différentes techniques d'imagerie du sein par rayons X, et fournissant des images radiographiques de fantômes numériques simples et complexes, a été implémentée et validée. Deuxièmement, une étude d'optimisation pour la technique CE-bCT basée sur une approche double-énergie a été réalisée, dans le but de maximiser la qualité des images équivalentes-iode ainsi que des images morphologiques. Enfin, le potentiel quantitatif des méthodes CE-bCT et CE-DBT a été comparé au travers de l'évaluation de la détectabilité, de la caractérisation, de la localisation et de la mesure de l'étendue 3D des lésions iodées. Dans une étude impliquant des observateurs humains, la détectabilité et la caractérisation des lésions iodées de différentes tailles, formes et concentrations ont été comparées entre CE-bCT et CE-DBT, grâce à l'utilisation d'un fantôme anthropomorphique numérique du sein.Conclusions : Les études de simulation menées pendant cette thèse suggèrent que le scanner dédié du sein avec injection de produit de contraste iodé pourrait être une technique réalisable pour la détection, localisation et caractérisation des tumeurs du sein, pour un niveau de dose comparable à une mammographie standard. Bien que les comparaisons préliminaires avec CE-DBT suggèrent une performance comparable sur la détection et caractérisation, l'information 3D complète combinée avec une haute résolution spatiale font de CE-bCT une évolution intéressante de CESM vers une évaluation quantitative 3D des prises de contraste, et une alternative potentielle à CE-MRI pour certaines indications cliniques. / Background: Worldwide, breast cancer is the most common cancer and second deadliest cancer in women. Diagnostic imaging techniques are a critical part for screening, diagnosis, tumor staging and cancer therapy of the breast. Contrast-Enhanced Magnetic Resonance Imaging (CE-MRI) is the current standard imaging technique allowing detection of abnormal vascular development and lesion contrast uptake. CE-MRI is however very costly and not widely available. Moreover, its spatial resolution might not be sufficient to depict certain types of lesions including microcalcifications. The development of Contrast-Enhanced Spectral Mammography (CESM) has made the clinical use of intravenous contrast enhancement with conventional mammography possible. However, CESM is a 2D projection technique and therefore presents limitations to depict the 3D internal structures of lesions and to provide accurate quantitative 3D functional information.Contrast-Enhanced Digital Breast Tomosynthesis (CE-DBT) and dedicated Contrast-Enhanced Breast CT (CE-bCT) are two breast imaging modalities currently under investigation by academic and industrial research groups. It is however anticipated that the quantitative potential of CE-DBT is limited, due to the inherent low depth-resolution of limited opening angle DBT modality. CE-bCT with quasi-isotropic spatial resolution and voxel signal intensity proportional to the linear attenuation coefficient is believed to offer more accurate quantitative information, though a low-dose operation is still a challenge.Objectives: The purpose of this thesis has been to study the technical feasibility of CE-bCT and its potential to accurately depict and localize tumors, as well as to provide accurate quantitative morphological and functional imaging information about tumors, at low radiation dose levels. To understand the incremental value of CE-bCT over CE-DBT, the quantitative potential of both technologies have been compared. This investigation has been performed through computer simulations.Methods: At first, a simulation platform capable of modeling various X-ray breast imaging techniques and providing radiographic images of simple and complex computational phantoms was developed and validated. Secondly, an optimization study of a CE-bCT technique based on a dual-energy approach was performed, aiming to maximize image quality of iodine-enhanced and morphological images. Finally, the quantitative potential of CE-bCT and CE-DBT was compared through the assessment of iodine-enhanced lesion detectability, characterization, localization and 3D extent measurement. In a human observer study, depiction and characterization of iodine-enhanced lesions of different sizes, shapes and iodine uptakes was compared between CE-bCT and CE-DBT using a mesh-based anthropomorphic breast phantom.Conclusions: The simulation studies in this PhD thesis suggest that dual-energy iodine-injected CE-bCT could be a feasible technique for breast tumor depiction, localization and characterization, with dose levels comparable to standard mammography. While preliminary comparisons with CE-DBT suggests comparable depiction and characterization performance, the fully 3D information combined with high spatial resolution confirms CE-bCT as an interesting low-dose evolution of CESM toward 3D quantitative assessment of contrast uptakes and potential alternative to

Este trabalho tem por objetivo comparar os seguintes métodos de compressão de imagens de impressão digital: transformada discreta do cosseno (DCT), transformada de wavelets de Haar, transformada de wavelets de Daubechies e transformada de wavelets de quantização escalar (WSQ). O propósito da comparação é identificar o método que resulta numa menor perda de dados, para a maior taxa de compressão possível. São utilizadas as seguintes métricas para avaliação da qualidade da imagem para os métodos: erro quadrático médio (ERMS), a relação sinal e ruído (SNR) e a relação sinal ruído de pico (PSNR). Para as métricas utilizadas a DCT apresentou os melhores resultados, seguida pela WSQ. No entanto, o melhor tempo de compressão e a melhor qualidade das imagens recuperadas avaliadas pelo software GrFinger 4.2, foram obtidos com a técnica WSQ. / This research aims to compare the following fingerprint image compression methods: the discrete cosseno transform (DCT), Haar wavelet transform, Daubechies wavelets transform and wavelet scalar quantization (WSQ). The main interest is to find out the technique with the smallest distortion and higher compression ratio. Image quality is measured using peak signal-to-noise ratio (PSNR), signal-to-noise ratio (SNR) and root mean square (ERMS). Image quality using these metrics showed best results for the DCT followed by WSQ, although the WSQ had the best compression time and presented the best quality when evaluated by the GrFinger 4.2 software.

O propósito deste trabalho tem sido a utilização das técnicas de Desenho Óptico para o desenvolvimento de sistemas ópticos altamente otimizados. Após a confeccção destes sistemas ópticos torna-se necessário avaliar o desempenho do sistema construído, para isto tem sido utilizado uma das técnicas mais modernas para avaliação de qualidade da imagem óptica, a função de transferência óptica FTO, mais especificamente a função de transferência de modulação FTM. FTM representa o módulo da função complexa FTO e está associada à medida direta e quantitativa da qualidade de imagem, descrevendo a estrutura da imagem como uma função da freqüência espacial. Estas duas técnicas são as ferramentas fundamentais para os desenhistas ópticos. Instrumentos oftálmicos como o microscópio cirúrgico, sistema óptico para adaptação de uma câmera CCD e a lâmpada de fenda são desenvolvidos e analisados por estas técnicas neste trabalho. / The purpose of this work has been the use of design optical techniques to development of highly optimized optical systems. After the fabrication of these systems, it is necessary to evaluate the performance of building systems. A modern technique used to evaluate the quality of optical image is the optical transfer function OTF, more exactly the modulation transfer function MTF. MTF is the modulus of the complex function OTF. MTF is associated the direct and quantitative measure of the image quality and it describes the image structure as a function of spatial frequency. This bides techniques are the tools of optical designers. Ophthalmic instruments like surgery microscope, optical system of CCD camera adaptation and slit lamps are developed and analyzed for this techniques in this work.

Imagens de documentos são documentos digitalizados com conteúdo textual. Estes documentos são compostos de caracteres e diagramação, apresentando características comuns entre si, como a presença de bordas e limites no formato de cada caractere. A relação entre as características das imagens de documentos e os impactos do processo de compressão com respeito à fidelidade visual são analisadas nesse trabalho. Métricas objetivas são empregadas na análise das características das imagens de documentos, como a medida da atividade da imagem (IAM) no domínio espacial dos pixels, e a verificação da medida de atividade espectral (SAM) no domínio espectral. Os desempenhos das técnicas de compressão de imagens baseada na transformada discreta de cosseno (DCT) e na transformada discreta de Wavelet (DWT) são avaliados sobre as imagens de documentos ao aplicar diferentes níveis de compressão sobre as mesmas, para cada técnica. Os experimentos são realizados sobre imagens digitais de documentos impressos e manuscritos de livros e periódicos, explorando texto escritos entre os séculos 16 ao século 19. Este material foi coletado na biblioteca Brasiliana Digital (www.brasiliana.usp.br), no Brasil. Resultados experimentais apontam que as medidas de atividade nos domínios espacial e espectral influenciam diretamente a fidelidade visual das imagens comprimidas para ambas as técnicas baseadas em DCT e DWT. Para uma taxa de compressão fixa de uma imagem comprimida em ambas técnicas, a presença de valores superiores de IAM e níveis menores de SAM na imagem de referência resultam em menor fidelidade visual, após a compressão. / Document images are digitized documents with textual content. These documents are composed of characters and their layout, with common characteristics among them, such as the presence of borders and boundaries in the shape of each character. The relationship between the characteristics of document images and the impact of the compression process with respect to visual fidelity are analyzed herein. Objective metrics are employed to analyze the characteristics of document images, such as the Image Activity Measure (IAM) in the spatial domain, and assessment of Spectral Activity Measure (SAM) in the spectral domain. The performance of image compression techniques based on Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) are evaluated from document images by applying different compression levels for each technique to these images. The experiments are performed on digital images of printed documents and manuscripts of books and magazines, exploring texts written from the 16th to the 19th century. This material was collected in the Brasiliana Digital Library in Brazil. Experimental results show that the activity measures in spatial and spectral domains directly influence the visual fidelity of compressed images for both the techniques based on DCT and DWT. For a fixed compression ratio for both techniques on a compressed image, higher values of IAM and low levels of SAM in the reference image result in less visual fidelity after compression.

Este trabalho tem por objetivo comparar os seguintes métodos de compressão de imagens de impressão digital: transformada discreta do cosseno (DCT), transformada de wavelets de Haar, transformada de wavelets de Daubechies e transformada de wavelets de quantização escalar (WSQ). O propósito da comparação é identificar o método que resulta numa menor perda de dados, para a maior taxa de compressão possível. São utilizadas as seguintes métricas para avaliação da qualidade da imagem para os métodos: erro quadrático médio (ERMS), a relação sinal e ruído (SNR) e a relação sinal ruído de pico (PSNR). Para as métricas utilizadas a DCT apresentou os melhores resultados, seguida pela WSQ. No entanto, o melhor tempo de compressão e a melhor qualidade das imagens recuperadas avaliadas pelo software GrFinger 4.2, foram obtidos com a técnica WSQ. / This research aims to compare the following fingerprint image compression methods: the discrete cosseno transform (DCT), Haar wavelet transform, Daubechies wavelets transform and wavelet scalar quantization (WSQ). The main interest is to find out the technique with the smallest distortion and higher compression ratio. Image quality is measured using peak signal-to-noise ratio (PSNR), signal-to-noise ratio (SNR) and root mean square (ERMS). Image quality using these metrics showed best results for the DCT followed by WSQ, although the WSQ had the best compression time and presented the best quality when evaluated by the GrFinger 4.2 software.