<div>First-person videos (FPVs) captured by wearable cameras provide a huge amount of visual data. FPVs have different characteristics compared to broadcast videos and mobile videos. The video quality of FPVs are influenced by motion blur, tilt, rolling shutter and exposure distortions. In this work, we design image and video assessment methods applicable for FPVs. </div><div><br></div><div>Our video quality assessment mainly focuses on three quality problems. The first problem is the video frame artifacts including motion blur, tilt, rolling shutter, that are caused by the heavy and unstructured motion in FPVs. The second problem is the exposure distortions. Videos suffer from exposure distortions when the camera sensor is not exposed to the proper amount of light, which often caused by bad environmental lighting or capture angles. The third problem is the increased blurriness after video stabilization. The stabilized video is perceptually more blurry than its original because the masking effect of motion is no longer present. </div><div><br></div><div>To evaluate video frame artifacts, we introduce a new strategy for image quality estimation, called mutual reference (MR), which uses the information provided by overlapping content to estimate the image quality. The MR strategy is applied to FPVs by partitioning temporally nearby frames with similar content into sets, and estimating their visual quality using their mutual information. We propose one MR quality estimator, Local Visual Information (LVI), that estimates the relative quality between two images which overlap.</div><div><br></div><div>To alleviate exposure distortions, we propose a controllable illumination enhancement method that adjusts the amount of enhancement with a single knob. The knob can be controlled by our proposed over-enhancement measure, Lightness Order Measure (LOM). Since the visual quality is an inverted U-shape function of the amount of enhancement, our design is to control the amount of enhancement so that the image is enhanced to the peak visual quality. </div><div><br></div><div>To estimate the increased blurriness after stabilization, we propose a visibility-inspired temporal pooling (VTP) mechanism. VTP mechanism models the motion masking effect on perceived video blurriness as the influence of the visibility of a frame on the temporal pooling weight of the frame quality score. The measure for visibility is estimated as the proportion of spatial details that is visible for human observers.</div>

We tackle the problems of no-reference/blind image and video quality evaluation. The approach we take is that of modeling the statistical characteristics of natural images and videos, and utilizing deviations from those natural statistics as indicators of perceived quality. We propose a probabilistic model of natural scenes and a probabilistic model of natural videos to drive our image and video quality assessment (I/VQA) algorithms respectively. The VQA problem is considerably different from the IQA problem since it imposes a number of challenges on top of the challenges faced in the IQA problem; namely the challenges arising from the temporal dimension in video that plays an important role in influencing human perception of quality. We compare our IQA approach to the state of the art in blind, reduced reference and full-reference methods, and we show that it is top performing. We compare our VQA approach to the state of the art in reduced and full-reference methods (no blind VQA methods that perform reliably well exist), and show that our algorithm performs as well as the top performing full and reduced reference algorithms in predicting human judgments of quality. / text

We propose a natural scene statistic based quality assessment model Refer- enceless Image Spatial QUality Evaluator (RISQUE) which extracts marginal statistics of local normalized luminance signals and measures 'un-naturalness' of the distorted image based on measured deviation of them. We also model distribution of pairwise products of adjacent normalized luminance signals providing us with orientation distortion information. Although multi-scale, the model is defined in the space domain avoiding costly frequency or wavelet transforms. The frame work is simple, fast, human perception based and shown to perform statistically better than other proposed no reference algorithms and full reference structural similarity index(SSIM). / text

No description available.

In recent years, the rapid development of autonomous vehicles (AVs) has necessitated the need for high-quality perception systems. Perception is a fundamental requirement for AVs, with cameras and LiDARs being commonly used sensors for environmental understanding and localization. However, there is a research gap in assessing the quality of AVs perception systems. To address this gap, this dissertation proposes a novel paradigm for evaluating AVs perception quality by studying the perception quality of cameras and LiDARs sensors. Our proposed paradigm aims to provide a comprehensive assessment of the quality of perception systems used in AVs.To achieve our research goals, we first validate the concept of surrounding environmental complexity through subjective experiments that rate complexity scores. In this study, we propose a neural network to classify complexity. Subsequently, we study image-based perception quality assessment by using image saliency and 2D object detection algorithms to create an image-based quality index. We then develop a neural network model to regress the proposed quality index score. Furthermore, we extend our research to LiDAR-based point cloud quality assessment by using the image-based saliency map as guidance to generate a point cloud quality index score. We then develop a neural network model to regress the score. Finally, we validate the proposed perception quality index with a novel designed AVs perception algorithm. In conclusion, this dissertation makes a significant contribution to the field of AVs perception by proposing a new paradigm for assessing perception quality. Our research findings can be used to improve the overall performance and safety of AVs, which has significant implications for the transportation industry and society as a whole. / Doctor of Philosophy / This dissertation delves into the fundamentals of autonomous vehicles (AVs), which is perception, with the aim of developing a new paradigm for evaluating the quality of perception algorithms. AVs are the dream of humanity, and perception is the fundamental requirement for achieving their full potential. Our research proposes a new approach to assessing the quality of perception algorithms, which can have significant implications for the performance and safety of AVs. By studying the perception algorithm quality, we aim to identify areas for improvement, leading to better AV performance and enhancing user trust. Our findings highlight the importance of perception in the development of AVs and demonstrate the need for continuous evaluation and improvement of the perception algorithms used in AVs.

This article contributes a framework for the quality assessment of imputations within a broader structure to evaluate the quality of register-based data. Four quality-related hyperdimensions examine the data processing from the raw-data level to the final statistics. Our focus lies on the quality assessment of different imputation steps and their influence on overall data quality. We suggest classification rates as a measure of accuracy of imputation and derive several computational approaches. (authors' abstract)

With the rapid development of network visual communication technologies, digital video has become ubiquitous and indispensable in our everyday lives. Video acquisition, communication, and processing systems introduce various types of distortions, which may have major impact on perceived video quality by human observers. Effective and efficient objective video quality assessment (VQA) methods that can predict perceptual video quality are highly desirable in modern visual communication systems for performance evaluation, quality control and resource allocation purposes. Moreover, perceptual VQA measures may also be employed to optimize a wide variety of video processing algorithms and systems for best perceptual quality. This thesis exploits several novel ideas in the areas of video quality assessment and enhancement. Firstly, by considering a video signal as a 3D volume image, we propose a 3D structural similarity (SSIM) based full-reference (FR) VQA approach, which also incorporates local information content and local distortion-based pooling methods. Secondly, a reduced-reference (RR) VQA scheme is developed by tracing the evolvement of local phase structures over time in the complex wavelet domain. Furthermore, we propose a quality-aware video system which combines spatial and temporal quality measures with a robust video watermarking technique, such that RR-VQA can be performed without transmitting RR features via an ancillary lossless channel. Finally, a novel strategy for enhancing video denoising algorithms, namely poly-view fusion, is developed by examining a video sequence as a 3D volume image from multiple (front, side, top) views. This leads to significant and consistent gain in terms of both peak signal-to-noise ratio (PSNR) and SSIM performance, especially at high noise levels.

Distributed compression and quality assessment (QA) are essential ingredients in the design and analysis of networked signal processing systems with voluminous data. Distributed source coding techniques enable the efficient utilization of available resources and are extremely important in a multitude of data intensive applications including image and video. The quality analysis of such systems is also equally important in providing benchmarks on performance leading to improved design and control. This dissertation approaches the complementary problems of distributed compression and quality assessment using information theoretic methods. While such an approach provides intuition on designing practical coding schemes for distributed compression, it directly yields image and video QA algorithms with excellent performance that can be employed in practice. This dissertation considers the information theoretic study of sophisticated problems in distributed compression including, multiterminal multiple description coding, multiterminal source coding through relays and joint source channel coding of correlated sources over wireless channels. Random and/or structured codes are developed and shown to be optimal or near optimal through novel bounds on performance. While lattices play an important role in designing near optimal codes for multiterminal source coding through relays and joint source channel coding over multiple access channels, time sharing random Gaussian codebooks is optimal for a wide range of system parameters in the multiterminal multiple description coding problem. The dissertation also addresses the challenging problem of reduced reference image and video QA. A family of novel reduced reference image and video QA algorithms are developed based on spatial and temporal entropic differences. While the QA algorithms for still images only compute spatial entropic differences, the video QA algorithms compute both spatial and temporal entropic differences and combine them in a perceptually relevant manner. These algorithms attain excellent performances in terms of correlation with human judgments of quality on large QA databases. The framework developed also enables the study of the degradation in performance of QA algorithms from full reference information to almost no information from the reference image or video. / text

This dissertation focuses on the investigation of human perception of stereoscopic 3D image quality and the development of automatic stereoscopic 3D image quality assessment frameworks. In order to assess human perception of visual quality, a human study was conducted and interactions between image quality, depth quality, visual comfort, and 3D viewing quality were inferred. The results indicate that the overall 3D viewing quality can be well predicted from only image quality and depth quality. Between image and depth quality, image quality seems to be the main factor that enables accurate prediction of overall 3D viewing quality. Two other human studies were conducted to study the effect of masking on stereoscopic distortions. Binocular suppression was observed in the stereo images which were distorted by blur, JPEG compression, or JPEG2K compression, however, no such suppression was observed for stereo images distorted by white noise. Further, a facilitation effect was also observed against disparity variation for blur and JPEG2K distorted stereo images while no depth masking effect was observed. Based on these results, I proposed an automatic full-reference (FR) 3D quality assessment framework. In this framework, I used Gabor filterbank responses to model stimulus strength and then synthesize a Cyclopean image from a stereo image pair. Because the quality of this synthesized view is similar to that of a Cyclopean image, which the human visual system recreates from the stereoscopic stimuli, performing the task of 3D quality assessment on synthesized views can deliver better performance. I verified the performance of this FR framework on the LIVE 3D Image Quality Database and the results indicate that applying the proposed framework improves the performance of FR 2D quality assessment algorithms when applied to stereo 3D images. Further, I proposed a no-reference (NR) 3D quality assessment (QA) algorithm based on natural scene statistics in both the spatial and the depth domain. Experiments indicate that the proposed NR algorithm outperforms all 2D FR QA algorithms and most 3D FR QA models in predicting 3D quality of stereo images. Finally, a fourth subjective study was conducted to understand depth quality when stereo content is free from visual discomfort. The result suggests that human perception of depth quality is correlated with the content of the stereo image and the stereoacuity function of human visual system. / text

With the rapid development of network visual communication technologies, digital video has become ubiquitous and indispensable in our everyday lives. Video acquisition, communication, and processing systems introduce various types of distortions, which may have major impact on perceived video quality by human observers. Effective and efficient objective video quality assessment (VQA) methods that can predict perceptual video quality are highly desirable in modern visual communication systems for performance evaluation, quality control and resource allocation purposes. Moreover, perceptual VQA measures may also be employed to optimize a wide variety of video processing algorithms and systems for best perceptual quality. This thesis exploits several novel ideas in the areas of video quality assessment and enhancement. Firstly, by considering a video signal as a 3D volume image, we propose a 3D structural similarity (SSIM) based full-reference (FR) VQA approach, which also incorporates local information content and local distortion-based pooling methods. Secondly, a reduced-reference (RR) VQA scheme is developed by tracing the evolvement of local phase structures over time in the complex wavelet domain. Furthermore, we propose a quality-aware video system which combines spatial and temporal quality measures with a robust video watermarking technique, such that RR-VQA can be performed without transmitting RR features via an ancillary lossless channel. Finally, a novel strategy for enhancing video denoising algorithms, namely poly-view fusion, is developed by examining a video sequence as a 3D volume image from multiple (front, side, top) views. This leads to significant and consistent gain in terms of both peak signal-to-noise ratio (PSNR) and SSIM performance, especially at high noise levels.