Research conducted in the aviation industry includes two major areas, increased safety and a reduction of the environmental footprint. This thesis investigates the possibilities of increased situational awareness with computer vision in avionics systems. Image fusion methods are evaluated with appropriate pre-processing of three image sensors, one in the visual spectrum and two in the infra-red spectrum. The sensor setup is chosen to cope with the different weather and operational conditions of an aircraft, with a focus on the final approach and landing phases. Extensive image quality assessment metrics derived from a systematic review is applied to provide a precise evaluation of the image quality of the fusion methods. A total of four image fusion methods are evaluated, where two are convolutional network-based, using the networks for feature extraction in the detailed layers. Other approaches with visual saliency maps and sparse representation are also evaluated. With methods implemented in MATLAB, results show that a conventional method implementing a rolling guidance filter for layer separation and visual saliency map provides the best results. The results are further confirmed with a subjective ranking test, where the image quality of the fusion methods is evaluated further.

abstract: The depth richness of a scene translates into a spatially variable defocus blur in the acquired image. Blurring can mislead computational image understanding; therefore, blur detection can be used for selective image enhancement of blurred regions and the application of image understanding algorithms to sharp regions. This work focuses on blur detection and its application to image enhancement. This work proposes a spatially-varying defocus blur detection based on the quotient of spectral bands; additionally, to avoid the use of computationally intensive algorithms for the segmentation of foreground and background regions, a global threshold defined using weak textured regions on the input image is proposed. Quantitative results expressed in the precision-recall space as well as qualitative results overperform current state-of-the-art algorithms while keeping the computational requirements at competitive levels. Imperfections in the curvature of lenses can lead to image radial distortion (IRD). Computer vision applications can be drastically affected by IRD. This work proposes a novel robust radial distortion correction algorithm based on alternate optimization using two cost functions tailored for the estimation of the center of distortion and radial distortion coefficients. Qualitative and quantitative results show the competitiveness of the proposed algorithm. Blur is one of the causes of visual discomfort in stereopsis. Sharpening applying traditional algorithms can produce an interdifference which causes eyestrain and visual fatigue for the viewer. A sharpness enhancement method for stereo images that incorporates binocular vision cues and depth information is presented. Perceptual evaluation and quantitative results based on the metric of interdifference deviation are reported; results of the proposed algorithm are competitive with state-of-the-art stereo algorithms. Digital images and videos are produced every day in astonishing amounts. Consequently, the market-driven demand for higher quality content is constantly increasing which leads to the need of image quality assessment (IQA) methods. A training-free, no-reference image sharpness assessment method based on the singular value decomposition of perceptually-weighted normalized-gradients of relevant pixels in the input image is proposed. Results over six subject-rated publicly available databases show competitive performance when compared with state-of-the-art algorithms. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2019

No description available.

No description available.

No description available.

In recent years, the rapid development of autonomous vehicles (AVs) has necessitated the need for high-quality perception systems. Perception is a fundamental requirement for AVs, with cameras and LiDARs being commonly used sensors for environmental understanding and localization. However, there is a research gap in assessing the quality of AVs perception systems. To address this gap, this dissertation proposes a novel paradigm for evaluating AVs perception quality by studying the perception quality of cameras and LiDARs sensors. Our proposed paradigm aims to provide a comprehensive assessment of the quality of perception systems used in AVs.To achieve our research goals, we first validate the concept of surrounding environmental complexity through subjective experiments that rate complexity scores. In this study, we propose a neural network to classify complexity. Subsequently, we study image-based perception quality assessment by using image saliency and 2D object detection algorithms to create an image-based quality index. We then develop a neural network model to regress the proposed quality index score. Furthermore, we extend our research to LiDAR-based point cloud quality assessment by using the image-based saliency map as guidance to generate a point cloud quality index score. We then develop a neural network model to regress the score. Finally, we validate the proposed perception quality index with a novel designed AVs perception algorithm. In conclusion, this dissertation makes a significant contribution to the field of AVs perception by proposing a new paradigm for assessing perception quality. Our research findings can be used to improve the overall performance and safety of AVs, which has significant implications for the transportation industry and society as a whole. / Doctor of Philosophy / This dissertation delves into the fundamentals of autonomous vehicles (AVs), which is perception, with the aim of developing a new paradigm for evaluating the quality of perception algorithms. AVs are the dream of humanity, and perception is the fundamental requirement for achieving their full potential. Our research proposes a new approach to assessing the quality of perception algorithms, which can have significant implications for the performance and safety of AVs. By studying the perception algorithm quality, we aim to identify areas for improvement, leading to better AV performance and enhancing user trust. Our findings highlight the importance of perception in the development of AVs and demonstrate the need for continuous evaluation and improvement of the perception algorithms used in AVs.

Depth-Image-Based Rendering (DIBR) est une technologie fondamentale dans plusieurs applications liées à la 3D, telles que la vidéo en mode point de vue libre (FVV), la réalité virtuelle (VR) et la réalité augmentée (AR). Cependant, l'évaluation de la qualité des vues synthétisées par DIBR a également posé de nouveaux problèmes, car ce processus induit de nouveaux types de distorsions, qui sont intrinsèquement différentes des distorsions provoquées par le codage vidéo. Ce travail est destiné à mieux évaluer la qualité des vues synthétisées par DIBR en multimédia immersif. Au chapitre 2, nous proposons deux métriques complètements sans référence (NR). Le principe de la première métrique NR NIQSV consiste à utiliser plusieurs opérations morphologiques d’ouverture et de fermeture pour détecter et mesurer les distorsions, telles que les régions floues et l’effritement. Dans la deuxième métrique NR NIQSV+, nous améliorons NIQSV en ajoutant un détecteur de “black hole” et une détection “stretching”.Au chapitre 3, nous proposons deux métriques de référence complète pour traiter les distorsions géométriques à l'aide d'un masque de désocclusion et d'une méthode de correspondance de blocs multi-résolution. Au chapitre 4, nous présentons une nouvelle base de données d'images synthétisée par DIBR avec ses scores subjectifs associés. Ce travail se concentre sur les distorsions uniquement induites par différentes méthodes de synthèse de DIBR qui déterminent la qualité d’expérience (QoE) de ces applications liées à DIBR. En outre, nous effectuons également une analyse de référence des mesures d'évaluation de la qualité objective de pointe pour les vues synthétisées par DIBR sur cette base de données. Le chapitre 5 conclut les contributions de cette thèse et donne quelques orientations pour les travaux futurs. / Depth-Image-Based Rendering (DIBR) is a fundamental technology in several 3D-related applications, such as Free viewpoint video (FVV), Virtual Reality (VR) and Augmented Reality (AR). However, new challenges have also been brought in assessing the quality of DIBR-synthesized views since this process induces some new types of distortions, which are inherently different from the distortions caused by video coding. This work is dedicated to better evaluate the quality of DIBRsynthesized views in immersive multimedia. In chapter 2, we propose a completely No-reference (NR) metric. The principle of the first NR metrics NIQSV is to use a couple of opening and closing morphological operations to detect and measure the distortions, such as “blurry regions” and “crumbling”. In the second NR metric NIQSV+, we improve NIQSV by adding a “black hole” and a “stretching” detection. In chapter 3, we propose two Fullreference metrics to handle the geometric distortions by using a dis-occlusion mask and a multi-resolution block matching methods.In chapter 4, we present a new DIBR-synthesized image database with its associated subjective scores. This work focuses on the distortions only induced by different DIBR synthesis methods which determine the quality of experience (QoE) of these DIBR related applications. In addition, we also conduct a benchmark of the state-of-the-art objective quality assessment metrics for DIBR-synthesized views on this database. The chapter 5 concludes the contributions of this thesis and gives some directions of future work.

[pt] O Reconhecimento Facial é o processo de identificação de uma pessoa a partir da imagem de sua face. Na forma mais usual, o processo de identificação consiste em extrair informações dessa imagem e compará-las com informações relativas a outras imagens armazenadas numa base de dados e por fim indicar na saída a imagem da base mais similar à imagem de entrada. O desempenho desse processo está diretamente ligado à qualidade das imagens, tanto das que estão armazenadas na base de dados, quanto da imagem do indivíduo cuja identidade está sendo determinada. Por isso, convém que a qualidade das imagens faciais seja avaliada antes que estas sejam submetidas ao procedimento de reconhecimento. A maioria dos métodos apresentados até o momento na literatura baseia-se em um conjunto de critérios, cada um voltado a um atributo isolado da imagem. A qualidade da imagem é considerada adequada se aprovada por todos os critérios individualmente. Desconsidera-se, portanto, o efeito cumulativo de diversos fatores que afetam a qualidade das imagens e, por conseguinte, o desempenho do reconhecimento facial. Essa monografia propõe uma metodologia para o projeto de métricas de qualidade de imagens faciais que expressem num único índice o efeito combinado de diversos fatores que afetam o reconhecimento. Tal índice é dado por uma função de um conjunto de atributos extraídos diretamente da imagem. O presente estudo analisa experimentalmente uma função linear e uma rede neural do tipo back-propagation como alternativas para a estimativa de qualidade a partir dos atributos. Experimentos conduzidos sobre a base de dados IMM para o algoritmo de reconhecimento baseado em padrões binários locais comprovam a o bom desempenho da metodologia. / [en] Face Recognition is the process of identifying people based on facial images. In its most usual form the identification procedure consists of extracting information from an input face image and comparing them to the records of other face images stored in a face data base, and finally indicating the most similar one to the input image. The performance of this process is directly dependent on the input image quality, as well as on the images in the data base. Thus, it is important that the quality of a face image is tested before it is given to the recognition procedure, either as a input image or as a new record in the face database. Most methods proposed thus far based on a set of criteria, each one devoted to an isolated attribute. The image quality is considered adequate if approved by all criteria individually. Thus, the cumulative effect of different factors affecting the image quality is no regarded. This dissertation proposes a methodology for the design of quality metrics of facial images that Express in a single scalar the combined effect of multiple factors affecting the quality. Such score is given by a function of attributes extracted directly from the image. This study investigates a linear and a non-linear approach for quality assessment. Experiments conducted upon the IMM face database for a Local Binary Pattern face recognition algorithm demonstrate the good performance of the proposed methodology.

Tone mapping operators aim to compress high dynamic range (HDR) images to low dynamic range ones so as to visualize HDR images on standard displays. Most existing works were demonstrated on specific examples without being thoroughly tested on well-established and subject-validated image quality assessment models. A recent tone mapped image quality index (TMQI) made the first attempt on objective quality assessment of tone mapped images. TMQI consists of two fundamental building blocks: structural fidelity and statistical naturalness. In this thesis, we propose an enhanced tone mapped image quality index (eTMQI) by 1) constructing an improved nonlinear mapping function to better account for the local contrast visibility of HDR images and 2) developing an image dependent statistical naturalness model to quantify the unnaturalness of tone mapped images based on a subjective study. Experiments show that the modified structural fidelity and statistical naturalness terms in eTMQI better correlate with subjective quality evaluations. Furthermore, we propose an iterative optimization algorithm for tone mapping. The advantages of this algorithm are twofold: 1) eTMQI and TMQI can be compared in a more straightforward way; 2) better quality tone mapped images can be automatically generated by using eTMQI as the optimization goal. Numerical and subjective experiments demonstrate that eTMQI is a superior objective quality assessment metric for tone mapped images and consistently outperforms TMQI.

Halftones are reproductions of images created through the process of halftoning. The goal of halftones is to create a replica of an image which, at a distance, looks nearly identical to the original. Several different methods for producing these halftones are available, three of which are error diffusion, DBS and IMCDP. To check whether a halftone would be perceived as of high quality there are two options: Subjective image quality assessments (IQA’s) and objective image quality (IQ) measurements. As subjective IQA’s often take too much time and resources, objective IQ measurements are preferred. But as there is no standard for which metric should be used when working with halftones, this brings the question of which one to use. For this project both online and on-location subjective testing was performed where observers were tasked with ranking halftoned images based on perceived image quality, the images themselves being chosen specifically to show a wide range of characteristics such as brightness and level of detail. The results of these tests were compiled and then compared to that of eight different objective metrics, the list of which is the following: MSE, PSNR, S-CIELAB, SSIM, BlurMetric, BRISQUE, NIQE and PIQE. The subjective and objective results were compared using Z-scores and showed that SSIM and NIQE were the objective metrics which most closely resembled the subjective results. The online and on-location subjective tests differed greatly for dark colour halftones and colour halftones containing smooth transitions, with a smaller variation for the other categories chosen. What did not change was the clear preference for DBS by both the observers and the objective IQ metrics, making it the better of the three methods tested. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>