A system is described for identifying obstacle-free landing sites for a vertical-takeoff-and-landing (VTOL) semi-autonomous unmanned aerial vehicle (UAV) from point cloud data obtained from a stereo vision system. The relatively inexpensive, commercially available Bumblebee stereo vision camera was selected for this study. A "point cloud viewer" computer program was written to analyze point cloud data obtained from 2D images transmitted from the UAV to a remote ground station. The program divides the point cloud data into segments, identifies the best-fit plane through the data for each segment, and performs an independent analysis on each segment to assess the feasibility of landing in that area. The program also rapidly presents the stereo vision information and analysis to the remote mission supervisor who can make quick, reliable decisions about where to safely land the UAV. The features of the program and the methods used to identify suitable landing sites are presented in this thesis. Also presented are the results of a user study that compares the abilities of humans and computer-supported point cloud analysis in certain aspects of landing site assessment. The study demonstrates that the computer-supported evaluation of potential landing sites provides an immense benefit to the UAV supervisor. / Master of Science

The field of robotics has advanced to the point where robots are being developed for use in fire environments to perform firefighting tasks. These environments contain varying levels of fire and smoke, both of which obstruct robotic perception sensors. In order to effectively use robots in fire environments, the issue of perception in the presence of smoke and fire needs to be addressed. The goal of this research was to address the problem of perception, specifically rangefinding, in fire smoke environments. A series of tests were performed in fire smoke filled environments to evaluate the performance of different commercial rangefinders and cameras as well as a long-wavelength infrared (LWIR) stereo vision system developed in this research. The smoke was varied from dense, low temperature smoke to light, high temperature smoke for evaluation in a range of conditions. Through small-scale experiments on eleven different sensors, radar and LWIR cameras outperformed other perception sensors within both smoke environments. A LWIR stereo vision system was developed for rangefinding and compared to radar, LIDAR, and visual stereo vision in large-scale testing, demonstrating the ability of LWIR stereo vision to rangefind in dense smoke when LIDAR and visual stereo vision fail. LWIR stereo vision was further developed for improved rangefinding in fire environments. Intensity misalignment between cameras and stereo image filtering were addressed quantitatively. Tests were performed with approximately isothermal scenes and thermally diverse scenes to select subsystem methods. In addition, the effects of image filtering on feature distortion were assessed. Rangefinding improvements were quantified with comparisons to ground truth data. Improved perception in varying levels of clear and smoke conditions was developed through sensor fusion of LWIR stereo vision and a spinning LIDAR. The data were fused in a multi-resolution 3D voxel domain using evidential theory to model occupied and free space states. A heuristic method was presented to separate significantly attenuated LIDAR returns from low-attenuation returns. Sensor models were developed for both return types and LWIR stereo vision. The fusion system was tested in a range of conditions to demonstrate its ability for improved performance over individual sensor use in fire environments. / Ph. D.

Loudspeakers are used professionally and for leisure as a device which presents audio information to a listener. Loudspeakers “color” this information in different ways because of different properties, which they inherit from the decisions made in the design process. This study investigated the audibility of phase distortion in loudspeaker systems in ecologically valid environments using different types of program material and levels of group-delay. 13 subjects participated in a listening test, each performing 48 trials across various conditions. Results revealed significant differences in the ability to differentiate between reference and impaired signals based on program material and impairment level. Notably, participants demonstrated better discrimination for simple transient sounds compared to a mixed music recording. These results suggest that phase distortion may be less audible in mixed music reproduction than in click-like sounds. However, findings indicate a lower audible threshold for phase distortion compared to existing literature for click-like stimuli. Overall, while phase distortion may not always be audible, consideration for it can be relevant for achieving high audio quality in loudspeaker systems. These findings hopefully contribute to the understanding of phase distortion's perceptual effects and its implications for audio engineering and consumer electronics design.

This paper presents a new approach to the feature-matching problem for 3D reconstruction by taking advantage of GPS and IMU data, along with a prior calibrated stereo camera system. It is expected that pose estimates and calibration can be used to increase feature matching speed and accuracy. Given pose estimates of cameras and extracted features from images, the algorithm first enumerates feature matches based on stereo projection constraints in 2D and then backprojects them to 3D. Then, a grid search algorithm over potential camera poses is proposed to match the 3D features and find the largest group of 3D feature matches between pairs of stereo frames. This approach will provide pose accuracy to within the space that each grid region covers. Further refinement of relative camera poses is performed with an iteratively re-weighted least squares (IRLS) method in order to reject outliers in the 3D matches. The algorithm is shown to be capable of running in real-time correctly, where the majority of processing time is taken by feature extraction and description. The method is shown to outperform standard open source software for reconstruction from imagery. / Master of Science

This dissertation proposes a novel approach to achieve circumferential three-dimensional (3D) profiling for dark objects by investigating specular micro-texture photometry. A small patch of a target surface in micro-texture level yields different appearance under different illumination. This photometric property can be used to reconstruct the target surface with pixel-level resolution. However, due to the nature of some material, the surface of whom has stronger specular components than diffuse components, making the usage of general microtexture photometry more difficult. On the other hand, without using micro-texture photometry, the conventional circumferential 3D approaches only utilizes the geometric property of the target surface, compared to which, the proposed is able to reconstruct the target surface with finer detail. The original contributions of this dissertation are threefold. To begin with, the specular component in the micro-texture photometry is investigated to propose the pixel-level 3D profiling. The intensities of the same pixel from different images, which are taken under different lighting conditions are different. The specular components are used to recover the surface normal of the corresponding surface patch of the target surface. Consequently, the proposed specular-photometry-based technique produces pixel-wise measurement on surface normal. Furthermore, the conventional circumferential 3D profiling approach is extended with the proposed specular-photometry-based technique. The result of 3D profiling via the conventional approach is sparse due to its nature. On the other hand, the result of 3D profiling from the integration using the surface normal obtained from the proposed specular-photometry-based technique suffers from accumulative error. A new approach is then proposed to use the result from the conventional approach as global constraint, for the purpose of reducing the accumulative error. The proposed approach is able to achieve pixel-resolution globally bounded profiling because of the dense surface normal measurement from the proposed specular-photometry-based technique and the constraints from the conventional approach. Lastly, a system is developed to apply the proposed circumferential specular-photometry-based 3D profiling approach. The developed system is not only able to acquire data and but also to provide different lighting conditions for both the specular-photometry-based technique and conventional approach using a digital single-lens reflex camera and different lighting devices. With a step motor to rotate the object for three hundred and sixty degrees, the system is able to achieve circumferential scanning / Doctor of Philosophy / This dissertation explains a novel approach to fulfill circumferential 3D profiling with high resolution for dark objects. With the proposed approach, the resolution is able to achieve micro-texture level. The high resolution measurement is achieved by investigating specular micro-texture photometry. As for dark objects, the specular components dominate the reflection. The usage of photometry is explained as follow. A small patch of a target surface yields different appearance under different illumination. For the material of the surface of dark objects, the surface reflects stronger specular components than diffuse components. The proposed approach utilizes this photometric property to recover the small patch's surface normal using its specular components. The recovered surface normal is then used to calculate the surface profile through integration. The conventional circumferential 3D profiling approach, which can only produce low-resolution measurement, is also adopted in the proposed approach to enhance its performance, as the integration method suffers from accumulative error. The result from the conventional approach serves as a global constraint to bound the final profiling result. A system is developed to apply the proposed circumferential specular-photometry-based 3D profiling approach, which is equipped with a step motor to rotate the object for three hundred and sixty degrees for the purpose of circumferential scanning. A series of numerical and experimental studies were conducted to validate the performance of the proposed approach. As it is shown in the result, the proposed approach is able to measure the tire tread with 31µm resolution.

Object tracking and object measurement are two well-established and prominent concepts within the field of computer vision. While the two techniques are fairly robust in images and videos where the object of interest(s) is clear, there is a significant decrease in performance when objects appear obscured due to a number of factors including motion blur, far distance from the camera, and blending with the background. Additionally, most established object detection models focus on detecting as many objects as possible, rather than striving for high accuracy on a few, predetermined objects. One application of computer vision tracking and measurement in imprecise and single-object scenarios is programmatically measuring the distance of a shot put throw in the sport of track and field. Shot put throws in competition are currently measured by human officials, which is both time-consuming and often erroneous. In this work, a computer vision system is developed that automatically tracks the path of a shot put throw through combining a custom-trained YOLO model and path predictor with kinematic formulas and then measures its distance traveled by triangulation using binocular stereo vision. The final distance measurements produce directionally accurate results with an average error of 82% after removing one outlier, an average detection time of 2.9 ms per frame and a total average run time of 4.5 minutes from the time the shot put leaves the thrower's hand. Shortcomings of tracking and measurement in imperfect or singular object settings are addressed and potential improvements are suggested, while also providing the opportunity to increase the accuracy and efficiency of the sporting event. / Master of Science / Object tracking and object measurement are two well-established and prominent concepts within the field of computer vision. While the two techniques are fairly robust in images and videos where the object of interest(s) is clear, there is a significant decrease in performance when objects appear obscured due to a number of factors including motion blur, far distance from the camera, and blending with the background. Additionally, most established object detection models focus on detecting as many objects as possible, rather than striving for high accuracy on a few, predetermined objects. One application of computer vision tracking and measurement in imprecise and single-object scenarios is programmatically measuring the distance of a shot put throw in the sport of track and field. Shot put throws in competition are currently measured by human officials, which is both time-consuming and often erroneous. In this work, a computer vision system is developed that automatically tracks the path of a shot put throw through combining a custom-trained YOLO model and path predictor with kinematic formulas and then measures its distance traveled by triangulation using binocular stereo vision. The final distance measurements produce directionally accurate results with an average error of 82% after removing one outlier, an average detection time of 2.9 ms per frame and a total average run time of 4.5 minutes from the time the shot put leaves the thrower's hand. Shortcomings of tracking and measurement in imperfect or singular object settings are addressed and potential improvements are suggested, while also providing the opportunity to increase the accuracy and efficiency of the sporting event.

Detta arbete ämnar att undersöka hur den upplevda rumsstorleken i digitala spel påverkas vid användningen av binauralt ljud i jämförelse med stereoljud. Denna studie relaterar till tidigare forskning inom ljudteknik för digitala spel. För att undersöka frågeställningen skapades en artefakt i form av en digital miljö där målet är att få en uppfattning av vilket av fyra olika rum som känns störst respektive minst. Artefakten är skapad så att det finns två stereomiljöer och två binaurala miljöer. Arbetet har fokuserat på främst kvalitativa metoder men har även inkluderat ett fåtal kvantitativa frågor. Resultatet av dessa sammanställs och analyseras för att fastställa vilken teknik som fungerar bäst gällande upplevd rumsstorlek samt immersionen därav. / <p>Stavningsvarierad titel: Ljudets påverkan på uppfattning av rumsstorlek i spel</p>

Résumé : Afin de répondre aux besoins de communication et de divertissement, il ne fait aucun doute que la parole et l’audio doivent être encodés sous forme numérique. En qualité CD, cela nécessite un débit numérique de 1411.2 kb/s pour un signal stéréo-phonique. Une telle quantité de données devient rapidement prohibitive pour le stockage de longues durées d’audio ou pour la transmission sur certains réseaux, particulièrement en temps réel (d’où l’adhésion universelle au format MP3). De plus, ces dernières années, la quantité de productions musicales et cinématographiques disponibles en cinq canaux et plus ne cesse d’augmenter. Afin de maintenir le débit numérique à un niveau acceptable pour une application donnée, il est donc naturel pour un codeur audio à bas débit d’exploiter la redondance entre les canaux et la psychoacoustique binaurale. Le codage perceptuel et plus particulièrement le codage paramétrique permet d’atteindre des débits manifestement inférieurs en exploitant les limites de l’audition humaine (étudiées en psychoacoustique). Cette recherche se concentre donc sur le codage paramétrique à bas débit de plus d’un canal audio. // Abstract : In order to fulfill our communications and entertainment needs, there is no doubt that speech and audio must be encoded in digital format. In"CD" quality, this requires a bit-rate of 1411.2 kb/s for a stereo signal. Such a large amount of data quickly becomes prohibitive for long-term storage of audio or for transmitting on some networks, especially in real-time (leading to a universal adhesion to the MP3 format). Moreover, throughout the course of these last years, the number of musical and cinematographic productions available in five channels or more continually increased.In order to maintain an acceptable bit-rate for any given application, it is obvious that a low bit-rate audio coder must exploit the redundancies between audio channels and binaural psychoacoustics. Perceptual audio coding, and more specifically parametric audio coding, offers the possibility of achieving much lower bit-rates by taking into account the limits of human hearing (psychoacoustics). Therefore, this research concentrates on parametric audio coding of more than one audio channel.

Kasmet daugybė žmonių būna sužalojami autoįvykiuose, iš kurių dalis sužalojimų būna rimti arba pasibaigia mirtimi. Dedama vis daugiau pastangų kuriant įvairias sistemas, kurios padėtų mažinti nelaimių skaičių kelyje. Tokios sistemos gebėtų perspėti vairuotojus apie galimus pavojus, atpažindamos eismo dalyvius ir sekdamos jų padėtį kelyje. Eismo dalyvių kelyje atpažinimas iš vaizdo yra pakankamai sudėtinga, daug skaičiavimų reikalaujanti problema. Šiame darbe šiai problemai spręsti pasitelkti stereo vaizdai, nesugretinamumo žemėlapis bei konvoliuciniai neuroniniai tinklai. Konvoliuciniai neuroniniai tinklai reikalauja daug skaičiavimų, todėl jie optimizuoti pasitelkus grafikos procesorių ir OpenCL. Gautas iki 33,4% spartos pagerėjimas lyginant su centriniu procesoriumi. Stereo vaizdai ir nesugretinamumo žemėlapis leidžia atmesti didelius kadro regionus, kurių nereikia klasifikuoti su konvoliuciniu neuroniniu tinklu. Priklausomai nuo scenos vaizde, reikalingų klasifikavimo operacijų skaičius sumažėja vidutiniškai apie 70-95% ir tai leidžia kadrą apdoroti atitinkamai greičiau. / Many people are injured during auto accidents each year, some injures are serious or end in death. Many efforts are being put in developing various systems, which could help to reduce accidents on the road. Such systems could warn drivers of a potential danger, while recognizing on-road vehicles and tracking their position on the road. On-road vehicle recognition on image is a complex and computationally very intensive problem. In this paper, to solve this problem, stereo images, disparity map and convolutional neural networks are used. Convolutional neural networks are very computational intensive, so to optimize it GPU and OpenCL are used. 33.4% speed improvement was achieved compared to the central processor. Stereo images and disparity map allows to discard large areas of the image, which are not needed to be classified using convolutional neural networks. Depending on the scene of the image, the number of the required classification operations decreases on average by 70-95% and this allows to process the image accordingly faster.

Comprendre, analyser et modéliser l'environment 3D à partir d'images provenant de caméras et d'appareils photos est l'un des défis majeurs actuel de recherche en vision par ordinateur. Cette thèse s'interesse à plusieurs aspects géométriques et photometriques liés à la reconstruction de surface à partir de plusieurs caméras calibrées. La reconstruction 3D est vue comme un problème de rendu inverse, et vise à minimiser une fonctionnelle d'énergie afin d'optimiser un maillage triangulaire représentant la surface à reconstruire. L'énergie est définie via un modèle génératif faisant naturellement apparaître des attributs tels que la visibilité ou la photométrie. Ainsi, l'approche présentée peut indifférement s'adapter à divers cas d'application tels que la stéréovision multi-vues, la stéréo photométrique multi-vues ou encore le “shape from shading” multi-vues. Plusieurs approches sont proposées afin de résoudre les problèmes de correspondances de l'apparence pour des scènes non Lambertiennes, dont l'apparence varie en fonction du point de vue. La segmentation, la stéréo photométrique ou encore la réciprocité d'Helmholtz sont des éléments étudiés afin de contraindre la reconstruction. L'exploitation de ces contraintes dans le cadre de reconstruction multi-vues permet de reconstruire des modèles complets 3D avec une meilleure qualité. / Understanding, analyzing and modeling the 3D world from 2D pictures and videos is probably one of the most exciting and challenging problem of computer vision. In this thesis, we address several geometric and photometric aspects to 3D surface reconstruction from multi-view calibrated images. We first formulate multi-view shape reconstruction as an inverse rendering problem. Using generative models, we formulate the problem as an energy minimization method that leads to the non-linear surface optimization of a deformable mesh. A particular attention is addressed to the computation of the discrete gradient flow, which leads to coherent vertices displacements. We particularly focus on models and energy functionals that depend on visibility and photometry. The same framework can then be equally used to perform multi-view stereo, multi-view shape from shading or multi-view photometric stereo. Then, we propose to exploit different additional information to constraint the problem in the non-Lambertian case, where the appearance of the scene depends on the view-point direction. Segmentation for instance can be used to segment surface regions sharing similar appearance or reflectance. Helmholtz reciprocity can also be applied to reconstruct 3D shapes of objects of any arbitrary reflectance properties. By taking multiple image-light pairs around an object, multi-view Helmholtz stereo can be performed. Using this constrained acquisition scenario and our deformable mesh framework, it is possible to reconstruct high quality 3D models.