<p>Bayesian nonparametric models, such as the Gaussian process and the Dirichlet process, have been extensively applied for target kinematics modeling in various applications including environmental monitoring, traffic planning, endangered species tracking, dynamic scene analysis, autonomous robot navigation, and human motion modeling. As shown by these successful applications, Bayesian nonparametric models are able to adjust their complexities adaptively from data as necessary, and are resistant to overfitting or underfitting. However, most existing works assume that the sensor measurements used to learn the Bayesian nonparametric target kinematics models are obtained a priori or that the target kinematics can be measured by the sensor at any given time throughout the task. Little work has been done for controlling the sensor with bounded field of view to obtain measurements of mobile targets that are most informative for reducing the uncertainty of the Bayesian nonparametric models. To present the systematic sensor planning approach to leaning Bayesian nonparametric models, the Gaussian process target kinematics model is introduced at first, which is capable of describing time-invariant spatial phenomena, such as ocean currents, temperature distributions and wind velocity fields. The Dirichlet process-Gaussian process target kinematics model is subsequently discussed for modeling mixture of mobile targets, such as pedestrian motion patterns. </p><p>Novel information theoretic functions are developed for these introduced Bayesian nonparametric target kinematics models to represent the expected utility of measurements as a function of sensor control inputs and random environmental variables. A Gaussian process expected Kullback Leibler divergence is developed as the expectation of the KL divergence between the current (prior) and posterior Gaussian process target kinematics models with respect to the future measurements. Then, this approach is extended to develop a new information value function that can be used to estimate target kinematics described by a Dirichlet process-Gaussian process mixture model. A theorem is proposed that shows the novel information theoretic functions are bounded. Based on this theorem, efficient estimators of the new information theoretic functions are designed, which are proved to be unbiased with the variance of the resultant approximation error decreasing linearly as the number of samples increases. Computational complexities for optimizing the novel information theoretic functions under sensor dynamics constraints are studied, and are proved to be NP-hard. A cumulative lower bound is then proposed to reduce the computational complexity to polynomial time.</p><p>Three sensor planning algorithms are developed according to the assumptions on the target kinematics and the sensor dynamics. For problems where the control space of the sensor is discrete, a greedy algorithm is proposed. The efficiency of the greedy algorithm is demonstrated by a numerical experiment with data of ocean currents obtained by moored buoys. A sweep line algorithm is developed for applications where the sensor control space is continuous and unconstrained. Synthetic simulations as well as physical experiments with ground robots and a surveillance camera are conducted to evaluate the performance of the sweep line algorithm. Moreover, a lexicographic algorithm is designed based on the cumulative lower bound of the novel information theoretic functions, for the scenario where the sensor dynamics are constrained. Numerical experiments with real data collected from indoor pedestrians by a commercial pan-tilt camera are performed to examine the lexicographic algorithm. Results from both the numerical simulations and the physical experiments show that the three sensor planning algorithms proposed in this dissertation based on the novel information theoretic functions are superior at learning the target kinematics with</p><p>little or no prior knowledge</p> / Dissertation

Wide-area camera networks are becoming more and more common. They have widerange of commercial and military applications from video surveillance to smart home and from traffic monitoring to anti-terrorism. The design of such a camera network is a challenging problem due to the complexity of the environment, self and mutual occlusion of moving objects, diverse sensor properties and a myriad of performance metrics for different applications. In this dissertation, we consider two such challenges: camera planing and camera fusion. Camera planning is to determine the optimal number and placement of cameras for a target cost function. Camera fusion describes the task of combining images collected by heterogenous cameras in the network to extract information pertinent to a target application. I tackle the camera planning problem by developing a new unified framework based on binary integer programming (BIP) to relate the network design parameters and the performance goals of a variety of camera network tasks. Most of the BIP formulations are NP hard problems and various approximate algorithms have been proposed in the literature. In this dissertation, I develop a comprehensive framework in comparing the entire spectrum of approximation algorithms from Greedy, Markov Chain Monte Carlo (MCMC) to various relaxation techniques. The key contribution is to provide not only a generic formulation of the camera planning problem but also novel approaches to adapt the formulation to powerful approximation schemes including Simulated Annealing (SA) and Semi-Definite Program (SDP). The accuracy, efficiency and scalability of each technique are analyzed and compared in depth. Extensive experimental results are provided to illustrate the strength and weakness of each method. The second problem of heterogeneous camera fusion is a very complex problem. Information can be fused at different levels from pixel or voxel to semantic objects, with large variation in accuracy, communication and computation costs. My focus is on the geometric transformation of shapes between objects observed at different camera planes. This so-called the geometric fusion approach usually provides the most reliable fusion approach at the expense of high computation and communication costs. To tackle the complexity, a hierarchy of camera models with different levels of complexity was proposed to balance the effectiveness and efficiency of the camera network operation. Then different calibration and registration methods are proposed for each camera model. At last, I provide two specific examples to demonstrate the effectiveness of the model: 1)a fusion system to improve the segmentation of human body in a camera network consisted of thermal and regular visible light cameras and 2) a view dependent rendering system by combining the information from depth and regular cameras to collecting the scene information and generating new views in real time.

The visibility-aware path planner addresses the problem of path planning for target visibility. It computes sequences of motions that afford a line of sight to a stationary visual target for sensors on a robotic platform. The visibility-aware planner uses a model of the visible region, namely, the region of the task space in which a line of sight exists to the target. The planner also takes the orientation of the sensor into account, utilizing a model of the field of view frustum. The planner applies a penalty to paths that cause the sensor to lose target visibility by exiting the visible region or rotating so the target is not in the field of view. The planner applies these penalties to the edges in a probabilistic roadmap, providing weights in the roadmap graph for graph-search based planning algorithms. This thesis presents two variants on the planner. The static multi-query planner precomputes penalties for all roadmap edges and performs a best-path search using Dijkstra's algorithm. The dynamic single-query planner uses an iterative test-and-reject search to find paths of acceptable penalty without the benefit of precomputation. Four experiments are presented which validate the planners and present examples of the path planning for visibility on 6-DOF robot manipulators. The algorithms are statistically tested with multiple queries. Results show that the planner finds paths with significantly lower losses of target visibility than existing shortest-path planners.

The visibility-aware path planner addresses the problem of path planning for target visibility. It computes sequences of motions that afford a line of sight to a stationary visual target for sensors on a robotic platform. The visibility-aware planner uses a model of the visible region, namely, the region of the task space in which a line of sight exists to the target. The planner also takes the orientation of the sensor into account, utilizing a model of the field of view frustum. The planner applies a penalty to paths that cause the sensor to lose target visibility by exiting the visible region or rotating so the target is not in the field of view. The planner applies these penalties to the edges in a probabilistic roadmap, providing weights in the roadmap graph for graph-search based planning algorithms. This thesis presents two variants on the planner. The static multi-query planner precomputes penalties for all roadmap edges and performs a best-path search using Dijkstra's algorithm. The dynamic single-query planner uses an iterative test-and-reject search to find paths of acceptable penalty without the benefit of precomputation. Four experiments are presented which validate the planners and present examples of the path planning for visibility on 6-DOF robot manipulators. The algorithms are statistically tested with multiple queries. Results show that the planner finds paths with significantly lower losses of target visibility than existing shortest-path planners.

The visibility-aware path planner addresses the problem of path planning for target visibility. It computes sequences of motions that afford a line of sight to a stationary visual target for sensors on a robotic platform. The visibility-aware planner uses a model of the visible region, namely, the region of the task space in which a line of sight exists to the target. The planner also takes the orientation of the sensor into account, utilizing a model of the field of view frustum. The planner applies a penalty to paths that cause the sensor to lose target visibility by exiting the visible region or rotating so the target is not in the field of view. The planner applies these penalties to the edges in a probabilistic roadmap, providing weights in the roadmap graph for graph-search based planning algorithms. This thesis presents two variants on the planner. The static multi-query planner precomputes penalties for all roadmap edges and performs a best-path search using Dijkstra's algorithm. The dynamic single-query planner uses an iterative test-and-reject search to find paths of acceptable penalty without the benefit of precomputation. Four experiments are presented which validate the planners and present examples of the path planning for visibility on 6-DOF robot manipulators. The algorithms are statistically tested with multiple queries. Results show that the planner finds paths with significantly lower losses of target visibility than existing shortest-path planners. / Science, Faculty of / Computer Science, Department of / Graduate

Although sensor planning in computer vision has been a subject of research for over two decades, a vast majority of the research seems to concentrate on two particular applications in a rather limited context of laboratory and industrial workbenches, namely 3D object reconstruction and robotic arm manipulation. Recently, increasing interest is engaged in research to come up with solutions that provide wide-area autonomous surveillance systems for object characterization and situation awareness, which involves portable, wireless, and/or Internet connected radar, digital video, and/or infrared sensors. The prominent research problems associated with multisensor integration for wide-area surveillance are modality selection, sensor planning, data fusion, and data exchange (communication) among multiple sensors. Thus, the requirements and constraints to be addressed include far-field view, wide coverage, high resolution, cooperative sensors, adaptive sensing modalities, dynamic objects, and uncontrolled environments. This article summarizes a new survey and analysis conducted in light of these challenging requirements and constraints. It involves techniques and strategies from work done in the areas of sensor fusion, sensor networks, smart sensing, Geographic Information Systems (GIS), photogrammetry, and other intelligent systems where finding optimal solutions to the placement and deployment of multimodal sensors covering a wide area is important. While techniques covered in this survey are applicable to many wide-area environments such as traffic monitoring, airport terminal surveillance, parking lot surveillance, etc., our examples will be drawn mainly from such applications as harbor security and long-range face recognition.

The Next-Best-View (NBV) problem plays an important part in automatic 3D object reconstruction and exploration applications. This thesis presents a novel approach of ray-casting in Occupancy Grid Maps (OGM) in the context of solving the NBV problem in a 3D-exploration setting. The proposed approach utilizes the structure of an octree-based OGM to perform calculations of potential information gain. The computations are significantly faster than current methods, without decreasing mapping quality. Performance, both in terms of mapping quality, coverage and computational complexity, is experimentally verified through a comparison with existing state-of-the-art methods using high-resolution point cloud data generated using time-of-flight laser range scanners. Current methods for viewpoint ranking focus either heavily on mapping performance or computation speed. The results presented in this thesis indicate that the proposed method is able to achieve a mapping performance similar to the performance-oriented approaches while maintaining the same low computation speed as more approximative methods.

This thesis addresses the problem of gas distribution modelling for gas monitoring and gas detection. The presented research is particularly focused on the methods that are suitable for uncontrolled environments. In such environments, gas source locations and the physical properties of the environment, such as humidity and temperature may be unknown or only sparse noisy local measurements are available. Example applications include air pollution monitoring, leakage detection, and search and rescue operations. This thesis addresses how to efficiently obtain and compute predictive models that accurately represent spatio-temporal gas distribution. Most statistical gas distribution modelling methods assume that gas dispersion can be modelled as a time-constant random process. While this assumption may hold in some situations, it is necessary to model variations over time in order to enable applications of gas distribution modelling for a wider range of realistic scenarios. This thesis proposes two time-dependent gas distribution modelling methods. In the first method, a temporal (sub-)sampling strategy is introduced. In the second method, a time-dependent gas distribution modelling approach is presented, which introduces a recency weight that relates measurement to prediction time. These contributions are presented and evaluated as an extension of a previously proposed method called Kernel DM+V using several simulation and real-world experiments. The results of comparing the proposed time-dependent gas distribution modelling approaches to the time-independent version Kernel DM+V indicate a consistent improvement in the prediction of unseen measurements, particularly in dynamic scenarios under the condition that there is a sufficient spatial coverage. Dynamic scenarios are often defined as environments where strong fluctuations and gas plume development are present. For mobile robot olfaction, we are interested in sampling strategies that provide accurate gas distribution models given a small number of samples in a limited time span. Correspondingly, this thesis addresses the problem of selecting the most informative locations to acquire the next samples. As a further contribution, this thesis proposes a novel adaptive sensor planning method. This method is based on a modified artificial potential field, which selects the next sampling location based on the currently predicted gas distribution and the spatial distribution of previously collected samples. In particular, three objectives are used that direct the sensor towards areas of (1) high predictive mean and (2) high predictive variance, while (3) maximising the coverage area. The relative weight of these objectives corresponds to a trade-off between exploration and exploitation in the sampling strategy. This thesis discusses the weights or importance factors and evaluates the performance of the proposed sampling strategy. The results of the simulation experiments indicate an improved quality of the gas distribution models when using the proposed sensor planning method compared to commonly used methods, such as random sampling and sampling along a predefined sweeping trajectory. In this thesis, we show that applying a locality constraint on the proposed sampling method decreases the travelling distance, which makes the proposed sensor planning approach suitable for real-world applications where limited resources and time are available. As a real-world use-case, we applied the proposed sensor planning approach on a micro-drone in outdoor experiments. Finally, this thesis discusses the potential of using gas distribution modelling and sensor planning in large-scale outdoor real-world applications. We integrated the proposed methods in a framework for decision-making in hazardous inncidents where gas leakage is involved and applied the gas distribution modelling in two real-world use-cases. Our investigation indicates that the proposed sensor planning and gas distribution modelling approaches can be used to inform experts both about the gas plume and the distribution of gas in order to improve the assessment of an incident.

This thesis deals with the development of a new methodology for automatic generation of scanning positions based on a computer model of the part for digitization of sheet metal parts. Manufacture and related inspection of sheet metal parts are closely connected to automotive industry. Based on increasing general requirements on accuracy, there is also a requirement for accurate inspection of manufactured parts in serial-line production. Optical 3D scanners and industrial robots are used more often for that purpose. Measuring positions for accurate and fast digitization of a part need to be prepared as the manufacturing of the new part begins. Planning of such positions is done manually by positioning of the industrial robot and saving the positions. The planning of positions proposed by this methodology is done automatically. A methodology of positions planning, their simulation for true visibility of the part elements using reflectance model and a simulation of the positions for robot reachability is presented in this thesis. The entire methodology is implemented as a plug-in for the Rhinoceros software. High reduction of time in positions planning compared to the manual approach was observed in the performed experiments.

The ability to search their environment is one of the most fundamental skills for any living creature. Visual search in particular is abundantly common for almost all animals. This act of searching is generally active in nature, with vision not simply reacting to incoming stimuli but also actively searching the environment for potential stimuli (such as by moving their head or eyes). Automatic visual search, likewise, is a crucial and powerful tool within a wide variety of different fields. However, performing such an active search is a nontrivial issue for many machine learning approaches. The added complexity of choosing which area to observe, as well as the common case of having a camera with adaptive field-of-view capabilities further complicates the problem. Hierarchical Reinforcement Learning have in recent years proven to be a particularly powerful means of solving hard machine learning problems by a divide-and-conquer methodology, where one highly complex task can be broken down into smaller sub-tasks which on their own may be more easily learnable. In this thesis, we present a hierarchical reinforcement learning system for solving a visual search problem in a stationary camera environment with adjustable pan, tilt and field-of-view capabilities. This hierarchical model also incorporates non-reinforcement learning agents in its workflow to better utilize the strengths of different agents and form a more powerful overall model. This model is then compared to a non-hierarchical baseline as well as some learning-free approaches.