This work applies sparse representations and nonlinear image processing to two inverse imaging problems. The first problem involves image restoration, where the aim is to reconstruct an unknown high-quality image from a low-quality observed image. Sparse representations of images have drawn a considerable amount of interest in recent years. The assumption that natural signals, such as images, admit a sparse decomposition over a redundant dictionary leads to efficient algorithms for handling such sources of data. The standard sparse representation, however, does not consider the intrinsic geometric structure present in the data, thereby leading to sub-optimal results. Using the concept that a signal is block sparse in a given basis —i.e., the non-zero elements occur in clusters of varying sizes — we present a novel and efficient algorithm for learning a sparse representation of natural images, called graph regularized block sparse dictionary (GRBSD) learning. We apply the proposed method towards two image restoration applications: 1) single-Image super-resolution, where we propose a local regression model that uses learned dictionaries from the GRBSD algorithm for super-resolving a low-resolution image without any external training images, and 2) image inpainting, where we use GRBSD algorithm to learn a multiscale dictionary to generate visually plausible pixels to fill missing regions in an image. Experimental results validate the performance of the GRBSD learning algorithm for single-image super-resolution and image inpainting applications. The second problem addressed in this work involves image enhancement for detection and segmentation of objects in images. We exploit the concept that even though data from various imaging modalities have high dimensionality, the data is sufficiently well described using low-dimensional geometrical structures. To facilitate the extraction of objects having such structure, we have developed general structure enhancement methods that can be used to detect and segment various curvilinear structures in images across different applications. We use the proposed method to detect and segment objects of different size and shape in three applications: 1) segmentation of lamina cribrosa microstructure in the eye from second-harmonic generation microscopy images, 2) detection and segmentation of primary cilia in confocal microscopy images, and 3) detection and segmentation of vehicles in wide-area aerial imagery. Quantitative and qualitative results show that the proposed methods provide improved detection and segmentation accuracy and computational efficiency compared to other recent algorithms.

With the progress of computer technology, computers are expected to be more intelligent in the interaction with humans, presenting information according to the user's psychological and physiological characteristics. However, computer users with visual problems may encounter difficulties on the perception of icons, menus, and other graphical information displayed on the screen, limiting the efficiency of their interaction with computers. In this dissertation, a personalized and dynamic image precompensation method was developed to improve the visual performance of the computer users with ocular aberrations. The precompensation was applied on the graphical targets before presenting them on the screen, aiming to counteract the visual blurring caused by the ocular aberration of the user's eye. A complete and systematic modeling approach to describe the retinal image formation of the computer user was presented, taking advantage of modeling tools, such as Zernike polynomials, wavefront aberration, Point Spread Function and Modulation Transfer Function. The ocular aberration of the computer user was originally measured by a wavefront aberrometer, as a reference for the precompensation model. The dynamic precompensation was generated based on the resized aberration, with the real-time pupil diameter monitored. The potential visual benefit of the dynamic precompensation method was explored through software simulation, with the aberration data from a real human subject. An "artificial eye'' experiment was conducted by simulating the human eye with a high-definition camera, providing objective evaluation to the image quality after precompensation. In addition, an empirical evaluation with 20 human participants was also designed and implemented, involving image recognition tests performed under a more realistic viewing environment of computer use. The statistical analysis results of the empirical experiment confirmed the effectiveness of the dynamic precompensation method, by showing significant improvement on the recognition accuracy. The merit and necessity of the dynamic precompensation were also substantiated by comparing it with the static precompensation. The visual benefit of the dynamic precompensation was further confirmed by the subjective assessments collected from the evaluation participants.

The multimedia domain is undergoing a rapid development phase with transition in audio, image, and video systems such as VoIP, Telepresence, Live/On-Demand Internet Streaming, SecondLife, and many more. In such a situation, the analysis of multimedia systems, like retrieval, quality evaluation, enhancement, summarization, and re-targeting applications, from various context is becoming critical. Current methods for solving the above-mentioned analysis problems do not consider the existence of humans and their affective characteristics in the design methodology. This contradicts the fact that most of the digital media is consumed only by the human end-users. We believe incorporating human feedback during the design and adaptation stage is key to the building process of multimedia systems. In this regard, we observe that affect is an important indicator of human perception and experience. This can be exploited in various ways for designing effective systems that will adapt more closely to the human response. We advocate an affect-based modeling approach for solving multimedia analysis problems by exploring new directions. In this dissertation, we select two representative multimedia analysis problems, e.g. Quality-of-Experience (QoE) evaluation and Image Enhancement in order to derive solutions based on affect-based modeling techniques. We formulate specific hypothesis for them by correlating system parameters to user's affective response, and investigate their roles under varying conditions for each respective scenario. We conducted extensive user studies based on human-to-human interaction through an audio conferencing system.We also conducted user studies based on affective enhancement of images and evaluated the effectiveness of our proposed approaches. Moving forward, multimedia systems will become more media-rich, interactive, and sophisticated and therefore effective solutions for quality, retrieval, and enhancement will be more challenging. Our work thus represents an important step towards the application of affect-based modeling techniques for the future generation of multimedia systems.

This paper introduced a Computer-Aided Optically Scanned Document Information Extraction System. It could extract information including invoice No., issued date, buyer, etc., from the optically scanned document to meet the demand of customs declaration companies. The system output the structured information to a relational database. In detail, a software architecture for the information extraction of diverse-structure optically scanned document is designed. In this system, the original document is classified firstly. It would put into template-based extraction to improve the extraction performance if its template is pre-defined in the system. Then, a method for image enhancement to improve the image classification is proposed. This method aims to optimize the accuracy of neural network model by extracting the template-related feature and actively removing the unrelated feature. Lastly, the above system is implemented in this paper. This extraction are programed in Python which is a cross-platform languages. This system comprises three parts, classification module, template-based extraction and non-template extraction all of which have APIs and could be ran independently. This feature make this system flexible and easy to customization for the further demand. 445 real-world customs document images were input to evaluate the system. The result revealed that the introduced system ensured the diverse document support with non-template extraction and reached the overall high performance with template-based extraction showing the goal was basically achieved.

Game engines such as Unreal Engine 5 are widely used to create photo-realistic renderings. To run these renderings at high quality without experiencing any performance issues,high-performance hardware is often required. In situations where the hardware is lacking,users may be forced to lower the quality and resolution of renderings to maintain goodperformance. While this may be acceptable in some situations, it limits the benefit that apowerful tool like Unreal Engine 5 can provide. This thesis aims to explore the possibilityof using a Real-ESRGAN, fine-tuned on a custom data set, to increase both the resolutionand quality of screenshots taken in Unreal Engine 5. By doing this, users can lower theresolution and quality of their Unreal Engine 5 rendering while still being able to generatehigh quality screenshots similar to those produced when running the rendering at higherresolution and higher quality. To accomplish this, a custom data set was created by randomizing camera positionsand capturing screenshots in an Unreal Engine 5 rendering. This data set was used to finetune a pre-trained Real-ESRGAN model. The fine-tuned model could then generate imagesfrom low resolution and low quality screenshots taken in Unreal Engine 5. The resultingimages were analyzed and evaluated using both quantitative and qualitative methods.The conclusions drawn from this thesis indicate that images generated using the finetuned weights are of high quality. This conclusion is supported by quantitative measurements, demonstrating that the generated images and the ground truth images are similar.Furthermore, visual inspection conducted by the authors confirms that the generated images are similar to the reference images, despite occasional artifacts.

No description available.

No description available.

No description available.

Yes / Surgical smoke removal algorithms can improve the quality of intra-operative imaging and reduce hazards in image-guided surgery, a highly desirable post-process for many clinical applications. These algorithms also enable effective computer vision tasks for future robotic surgery. In this paper, we present a new unsupervised learning framework for high-quality pixel-wise smoke detection and removal. One of the well recognized grand challenges in using convolutional neural networks (CNNs) for medical image processing is to obtain intra-operative medical imaging datasets for network training and validation, but availability and quality of these datasets are scarce. Our novel training framework does not require ground-truth image pairs. Instead, it learns purely from computer-generated simulation images. This approach opens up new avenues and bridges a substantial gap between conventional non-learning based methods and which requiring prior knowledge gained from extensive training datasets. Inspired by the Generative Adversarial Network (GAN), we have developed a novel generative-collaborative learning scheme that decomposes the de-smoke process into two separate tasks: smoke detection and smoke removal. The detection network is used as prior knowledge, and also as a loss function to maximize its support for training of the smoke removal network. Quantitative and qualitative studies show that the proposed training framework outperforms the state-of-the-art de-smoking approaches including the latest GAN framework (such as PIX2PIX). Although trained on synthetic images, experimental results on clinical images have proved the effectiveness of the proposed network for detecting and removing surgical smoke on both simulated and real-world laparoscopic images. / Research Development Fund Publication Prize Award winner, November 2019.

No description available.