A avaliação objetiva da qualidade de imagens é de especial importância em diversas aplicações, por exemplo na compressão de imagens, onde pode ser utilizada para regular a taxa que deve ser empregada para que haja a máxima compressão (permitindo perda de dados) sem comprometer a qualidade final; outro exemplo é na inserção de marcas dágua, isto é, introdução de informações descritivas utilizadas para atestar a autenticidade de uma imagem, que devem ser invisíveis para o observador. O SSIM (Structural SIMilarity) é uma métrica de avaliação objetiva da qualidade de imagens de referência completa projetada para imagens em tons de cinza. Esta dissertação investiga sua aplicação na avaliação de imagens coloridas. Para tanto, inicialmente é feito um estudo do SSIM utilizando quatro diferentes espaços de cores RGB, YCbCr, Lαβ e CIELAB. O SSIM é primeiramente calculado nos canais individuais desses espaços de cores. Em seguida, com inspiração no trabalho desenvolvido em (1) são testadas formas de se combinar os valores SSIM obtidos para cada canal em um valor único os chamados SSIM Compostos. Finalmente, a fim de buscar melhores correlações entre SSIM e avaliação subjetiva, propomos a utilização da mínima diferença de cor perceptível, calculada utilizando o espaço de cores CIELAB, conjuntamente com o SSIM. Para os testes são utilizados três bancos de dados de imagens coloridas, LIVE, IVC e TID, a fim de se conferir consistência aos resultados. A avaliação dos resultados é feita utilizando as métricas empregadas pelo VQEG (Video Quality Experts Group) para a avaliação da qualidade de vídeos, com uma adaptação. As conclusões do trabalho sugerem que os melhores resultados para avaliação da qualidade de imagens coloridas usando o SSIM são obtidas usando os canais de luminância dos espaços de cores YCbCr, Lαβ e especialmente o CIELAB. Também se concluiu que a utilização da mínima diferença de cor perceptível contribui para o melhoramento dos resultados da avaliação objetiva. / Objective image quality evaluation is of special interest in many image applications, for example for image compression, where it can be used to control the rate in order to keep a tradeoff between lost of data and image quality; another example is in the application of watermarks, i.e., introduction of descriptive information used to guarantee the authenticity of an image, that must be invisible to the observer who looks at the image. SSIM (Structural SIMilarity) index is a full-reference image quality assessment metric developed to evaluate gray images. This work investigates the application of SSIM in the evaluation of color images. Therefore, four different color spaces are tested RGB, YCbCr, Lαβ e CIELAB. Initially SSIM is calculated individually for each one of color spaces channels. Then, inspired in (1), the results of the SSIM in the individual channels are combined in a unique result the so called Composite SSIM. Finally, in order to improve the correlations between, calculated using CIELAB color space, together with SSIM. Three color image databases, LIVE, IVC and TID, were employed in the tests in order to confer solidity to the results. The evaluation of the results is made using VQEG (Video Quality Experts Group) methodology, developed for video quality evaluation with an adaptation regarding the time dimension that does not exist in the image domain. The conclusions from the work were that SSIM performs better in the evaluation of color images when applied to luminance channel of YCbCr, Lαβ and especially to CIELAB color spaces. It was also concluded that the use of just noticeable difference concept improve objective assessment results.

<div>This thesis consists of four sections which are related with four research projects.</div><div><br></div><div>The first section is about Page Classification. In this section, we extend our previous approach which could classify 3 classes of pages: Text, Picture and Mixed, to 5 classes which are: Text, Picture, Mixed, Receipt and Highlight. We first design new features to define those two new classes and then use DAG-SVM to classify those 5 classes of images. Based on the results, our algorithm performs well and is able to classify 5 types of pages.</div><div><br></div><div>The second section is about Fading Detection. In this section, we develop an algorithm that can automatically detect fading for both text and non-text region. For text region, we first do global alignment and then perform local alignment. After that, we create a 3D color node system, assign each connected component to a color node and get the color difference between raster page connected component and scanned page connected. For non-text region, after global alignment, we divide the page into "super pixels" and get the color difference between raster super pixels and testing super pixels. Compared with the traditional method that uses a diagnostic page, our method is more efficient and effective.</div><div><br></div><div>The third section is about CNN Based Emotion Recognition. In this section, we build our own emotion recognition classification and regression system from scratch. It includes data set collection, data preprocessing, model training and testing. We extend the model to real-time video application and it performs accurately and smoothly. We also try another approach of solving the emotion recognition problem using Facial Action Unit detection. By extracting Facial Land Mark features and adopting SVM training framework, the Facial Action Unit approach achieves comparable accuracy to the CNN based approach.</div><div><br></div><div>The forth section is about Saliency Based Image Quality Assessment and Cropping. In this section, we propose a method of doing image quality assessment and recomposition with the help of image saliency information. Saliency is the remarkable region of an image that attracts people's attention easily and naturally. By showing everyday examples as well as our experimental results, we demonstrate the fact that, utilizing the saliency information will be beneficial for both tasks.</div>

Diplomová práca sa zaoberá detekciou a hodnotením skreslených snímok v retinálnych obrazových dátach. Teoretická časť obsahuje stručné zhrnutie anatómie oka a metód hodnotenia kvality obrazov všeobecne, ako aj konkrétne hodnotenie retinálnych obrazov. Praktická časť bola vypracovaná v programovacom jazyku Python. Obsahuje predspracovanie dostupných retinálnych obrazov za účelom vytvorenia vhodného datasetu. Ďalej je navrhnutá metóda hodnotenia troch typov šumu v skreslených retinálnych obrazoch, presnejšie pomocou Inception-ResNet-v2 modelu. Táto metóda nebola prijateľná a navrhnutá bola teda iná metóda pozostávajúca z dvoch krokov - klasifikácie typu šumu a následného hodnotenia úrovne daného šumu. Pre klasifikáciu typu šumu bolo využité filtrované Fourierove spektrum a na hodnotenie obrazu boli využité príznaky extrahované pomocou ResNet50, ktoré vstupovali do regresného modelu. Táto metóda bola ďalej rozšírená ešte o krok detekcie zašumených snímok v retinálnych sekvenciách.

Ces dernières années, les technologies d’écran se sont considérablement améliorées. Par exemple, le contraste des écrans à plage dynamique élevée (HDR) dépasse de loin la capacité d’un écran conventionnel. De plus, un écran à gamut de couleur étendu (WCG) peut couvrir un espace colorimétrique plus grand que jamais. L'évaluation de la qualité de ces nouveaux contenus est devenue un domaine de recherche actif, les métriques de qualité SDR classiques n'étant pas adaptées. Cependant, les études les plus récentes négligent souvent une caractéristique importante: les chrominances. En effet, les bases de données existantes contiennent des images HDR avec un gamut de couleur standard, négligeant ainsi l’augmentation de l’espace colorimétrique due au WCG et les artefacts chromatiques. La plupart des mesures de qualité HDR objectives non plus ne prennent pas en compte ces artefacts. Pour surmonter cette problématique, dans cette thèse, nous proposons deux nouvelles bases de données HDR/WCG annotés avec des scores subjectifs présentant des artefacts chromatique réaliste. En utilisant ces bases de données, nous explorons trois solutions pour créer des métriques HDR/WCG: l'adaptation des métrics de qualité SDR, l’extension colorimétrique d’une métrique HDR connue appelée HDR-VDP-2 et, enfin, la fusion de diverses métriques de qualité et de features colorimétriques. Cette dernière métrique présente de très bonnes performances pour prédire la qualité tout en étant sensible aux distorsions chromatiques. / To improve their ability to display astonishing images, screen technologies have been greatly evolving. For example, the contrast of high dynamic range rendering systems far exceed the capacity of a conventional display. Moreover, a Wide Color gamut display can cover a bigger color space than ever. Assessing the quality of these new content has become an active field of research as classical SDR quality metrics are not adapted. However, state-of-the-art studies often neglect one important image characteristics: chrominances. Indeed, previous databases contain HDR images with a standard gamut thus neglecting the increase of color space due to WCG. Due to their gamut, these databases are less prone to contain chromatic artifacts than WCG content. Moreover, most existing HDR objective quality metrics only consider luminance and are not considering chromatic artifacts. To overcome this problematic, in this thesis, we have created two HDR / WCG databases with annotated subjective scores. We focus on the creation of a realistic chromatic artifacts that can arise during compression. In addition, using these databases, we explore three solutions to create HDR / WCG metrics. First, we propose a method to adapt SDR metrics to HDR / WCG content. Then, we proposed an extension of a well-known HDR metric called HDR-VDP-2. Finally, we create a new metric based on the merger of various quality metric and color features. This last metric presents very good performance to predict quality while being sensitive to chromatic distortion.

an instance has become exceedingly more time and resource consuming. To solve this issue, cloud computing is being used to train and serve the models. However, there’s a gap in research where these cloud computing platforms have been evaluated for these tasks. This thesis aims to investigate the inference task of an image quality assessment algorithm on different Machine Learning as a Service architecture. The quantitative metrics that are being used for the comparison are latency, inference time, throughput, carbon Footprint, and cost. The utilization of Machine Learning has a wide range of applications, with one of its most popular areas being Image Recognition or Image Classification. To effectively classify an image, it is imperative that the image is of high quality. This requirement is not always met, particularly in situations where users capture images through their mobile devices or other equipment. In light of this, there is a need for an image quality assessment, which can be achieved through the implementation of an Image Quality Assessment Model such as BRISQUE. When hosting BRISQUE in the cloud, there is a plethora of hardware options to choose from. This thesis aims to conduct a benchmark of these hardware options to evaluate the performance and sustainability of BRISQUE’s image quality assessment on various cloud hardware. The metrics for evaluation include inference time, hourly cost, effective cost, energy consumption, and emissions. Additionally, this thesis seeks to investigate the feasibility of incorporating sustainability metrics, such as energy consumption and emissions, into machine learning benchmarks in cloud environments. The results of the study reveal that the instance type from GCP was generally the best-performing among the 15 tested. The Image Quality Assessment Model appeared to benefit more from a higher number of cores than a high CPU clock speed. In terms of sustainability, it was observed that all instance types displayed a similar level of energy consumption, however, there were variations in emissions. Further analysis revealed that the selection of region played a significant role in determining the level of emissions produced by the cloud environment. However, the availability of such sustainability data is limited in a cloud environment due to restrictions imposed by cloud providers, rendering the inclusion of these metrics in Machine Learning benchmarks in cloud environments problematic. / Maskininlärning kan användas till en mängd olika saker. Ett populärt verksamhetsområde inom maskininlärning är bildigenkänning eller bildklassificering. För att utföra bildklassificering på en bild krävs först en bild av god kvalitet. Detta är inte alltid fallet när användare tar bilder i en applikation med sina telefoner eller andra enheter. Därför är behovet av en bildkvalitetskontroll nödvändigt. BRISQUE är en modell för bildkvalitetsbedömning som gör bildkvalitetskontroller på bilder, men när man hyr plats för den i molnet finns det många olika hårdvarualternativ att välja mellan. Denna uppsats avser att benchmarka denna hårdvara för att se hur BRISQUE utför inferens på dessa molnhårdvaror både när det gäller prestanda och hållbarhet där inferensens tid, timpris, effektivt pris, energiförbrukning och utsläpp är de insamlade mätvärdena. Avhandlingen söker också att undersöka möjligheten att inkludera hållbarhetsmetriker som energiförbrukning och utsläpp i en maskininlärningsbenchmark i molnmiljöer. Resultaten av studien visar att en av GCPs instanstyper var generellt den bäst presterande bland de 15 som testades. Bildkvalitetsbedömningsmodellen verkar dra nytta av ett högre antal kärnor mer än en hög CPU-frekvens. Vad gäller hållbarhet observerades att alla instanstyper visade en liknande nivå av energianvändning, men det fanns variationer i utsläpp. Ytterligare analys visade att valet av region hade en betydande roll i bestämningen av nivån av utsläpp som producerades av molnmiljön. Tillgången till sådana hållbarhetsdata är begränsade i en molnmiljö på grund av restriktioner som ställs av molnleverantörer vilket skapar problem om dessa mätvärden ska inkluderas i maskininlärningsbenchmarks i molnmiljöer.

Biometric systems allow identification of human persons based on physiological or behavioral characteristics, such as voice, handprint, iris or facial characteristics. The use of face and iris recognition as a way to authenticate user’s identities has been a topic of research for years. Present iris recognition systems require that subjects stand close (<2m) to the imaging camera and look for a period of about three seconds until the data are captured. This cooperative behavior is required in order to capture quality images for accurate recognition. This will eventually restrict the amount of practical applications where iris recognition can be applied, especially in an uncontrolled environment where subjects are not expected to cooperate such as criminals and terrorists, for example. For this reason, this thesis develops a collection of methods to deal with low quality face and iris images and that can be applied for face and iris recognition in a non-cooperative environment. This thesis makes the following main contributions: I. For eye and face tracking in low quality images, a new robust method is developed. The proposed system consists of three parts: face localization, eye detection and eye tracking. This is accomplished using traditional image-based passive techniques such as shape information of the eye and active based methods which exploit the spectral properties of the pupil under IR illumination. The developed method is also tested on underexposed images where the subject shows large head movements. II. For iris recognition, a new technique is developed for accurate iris segmentation in low quality images where a major portion of the iris is occluded. Most existing methods perform generally quite well but tend to overestimate the occluded regions, and thus lose iris information that could be used for identification. This information loss is potentially important in the covert surveillance applications we consider in this thesis. Once the iris region is properly segmented using the developed method, the biometric feature information is calculated for the iris region using the relative entropy technique. Iris biometric feature information is calculated using two different feature decomposition algorithms based on Principal Component Analysis (PCA) and Independent Component Analysis (ICA). III. For face recognition, a new approach is developed to measure biometric feature information and the changes in biometric sample quality resulting from image degradations. A definition of biometric feature information is introduced and an algorithm to measure it proposed, based on a set of population and individual biometric features, as measured by a biometric algorithm under test. Examples of its application were shown for two different face recognition algorithms based on PCA (Eigenface) and Fisher Linear Discriminant (FLD) feature decompositions.

Den enorma ökningen av multicasttjänster har visat begränsningarna hos traditionella nätverkshanteringsverktyg vid multicastkvalitetsövervakning. Det behövs någon annan form av övervakningsteknik som inte är en hårdvaruinriktad lösning så som ökad länkgenomströmmning, buffertlängd och kapacitet för att förbättra kundupplevelsen. I rapporten undersöks användningen av biblioteken FFmpeg, och OpenCV samt no-reference image quality assessemnt algoritmen BRISQUE för att förbättra tjänstekvaliteten och kundupplevelsen. Genom att upptäcka kvalitetsbrister hos bildrutor samt bitfel i videoströmmen kan QoS och QoE förbättras. Uppgiftens ändamål är att i realtid detektera avvikelser i bildkvalitet och bitfel i en multicastvideoström för att sedan notifiera tjänsteleverantören med hjälp av SNMP traps. Undersökningen visar positiva resultat med en hybridlösning med användning av både BRISQUE och FFmpeg då båda ensamma inte är tillräckligt anpassade för multimediaövervakning. FFmpeg har möjligheten att detektera avkodningsfel som oftast beror på allvarliga bitfel, och BRISQUE algoritmen utvecklades för att analysera bilder och bestämma bildkvaliteten. Enligt testresultaten kan BRISQUE användas för multicastvideoanalysering eftersom att den subjektiva bildkvaliteten kan bestämmas med god pålitlighet. Kombinationen av dessa metoder har visat bra resultat men behöver undersökas mer för användning av multicastövervakning. / The enormous increase in multicast services has shown the limitations of traditional network management tools in multicast quality monitoring. There is a need for new monitoring technologies that are not hardware-based solutions such as increased link throughput, buffer length and capacity to enhance the quality of experience. This paper examines the use of FFmpeg, and OpenCV as well the no-reference image quality assessment algorithm BRISQUE to improve the quality of service as well as the quality of experience. By detecting image quality deficiencies as well as bit errors in the video stream, the QoS and QoE can be improved. The purpose of this project was to develop a monitoring system that has the ability to detect fluctuations in image quality and bit errors in a multicast video stream in real time and then notify the service provider using SNMP traps. The tests performed in this paper shows positive results when using the hybrid solution proposed in this paper, both BRISQUE and FFmpeg alone are not sufficiently adapted for this purpose. FFmpeg has the ability to detect decoding errors that usually occurs due to serious bit errors and the BRISQUE algorithm was developed to analyse images and determine the subjective image quality. According to the test results BRISQUE can be used for multicast video analysis because the subjective image quality can be determined with good reliability. The combination of these methods has shown good results but needs to be investigated and developed further.

<p>An on-line quality assessment system in the industry is essential to prevent artifacts and guide manufacturing processes. Some well-developed systems can diagnose problems and help control the output qualities. However, some of the conventional methods are limited in time consumption and cost of expensive human labor. So, more efficient solutions are needed to guide future decisions and improve productivity. This thesis focuses on developing two image-based machine learning systems to accelerate the manufacturing process: one is to benefit nitrate sensor fabrication, and the other is to help image quality control for inkjet printers.</p> <p><br></p> <p>In the first work, we propose a system for predicting the nitrate sensor's performance based on non-contact images. Nitrate sensors are commonly used to reflect the nitrate levels of soil conditions in agriculture. In a roll-to-roll system, for manufacturing thin-film nitrate sensors, varying characteristics of the ion-selective membrane on screen-printed electrodes are inevitable and affect sensor performance. It is essential to monitor the sensor performance in real-time to guarantee the quality of the sensor. We also develop a system for predicting the sensor performance in on-line scenarios and making the neural networks efficiently adapt to the new data.</p> <p><br></p> <p>Streaks are the number one image quality problem in inkjet printers. In the second work, we focus on developing an efficient method to model and predict missing jets, which is the main contributor to streaks. In inkjet printing, the missing jets typically increase over printing time, and the print head needs to be purged frequently to recover missing jets and maintain print quality. We leverage machine learning techniques for developing spatio-temporal models to predict when and where the missing jets are likely to occur. The prediction system helps the inkjet printers make more intelligent decisions during customer jobs. In addition, we propose another system that will automatically identify missing jet patterns from a large-scale database that can be used in a diagnostic system to identify potential failures.</p>

Purpose of this thesis is the quality check of correct button text display on photographed monitors. These photographs contain a variety of image distortions which complicates the following image graphic element recognition. This paper outlines several possibilities to detect buttons on forms and further elaborates on the implemented detection based on contour shapes description. After buttons are found, their defects are detected subsequently. Additionally, this thesis describes an automatic identification of picture with the highest quality for documentation purposes.

<p>Osnovni doprinos ove doktorske disertacije je razvoj algortima i sistema za objektivnu procenu vizuelnog kvaliteta slike uzimajući u obzir najvažnija moguća oštećenja kao što su zamućenje ivica (oštrina) i poremećaj prirodnog izgleda teksture objekata na slici sa jedne strane i uticaj sadržaja slike (procenta ivica u slici) na procenu kvaliteta sa druge strane. Dakle, hipoteza izneta u ovom radu je da je potreban multiparametarski pristup da bi se dobila objektivna procena kvaliteta slike koja je što približnija subjektivnoj proceni.</p>