Finding lost persons, collecting information in disturbed communities, efficiently traversing urban areas after a blast or similar catastrophic events have motivated researchers to develop intelligent sensor frameworks to aid law enforcement, first responders, and military personnel with situational awareness. This dissertation consists of a two-part framework for providing situational awareness using both acoustic ground sensors and aerial sensing modalities. Ground sensors in the field of data-driven detection and classification approaches typically rely on computationally expensive inputs such as image or video-based methods [6, 91]. However, the information given by an acoustic signal offers several advantages, such as low computational needs and possible classification of occluded events including gunshots or explosions. Once an event is identified, responding to real-time events in urban areas is difficult using an Unmanned Aerial Vehicle (UAV) especially when GPS is unreliable due to coverage blackouts and/or GPS degradation [10]. Furthermore, if it is possible to deploy multiple in-situ static intelligent acoustic autonomous sensors that can identify anomalous sounds given context, then the sensors can communicate with an autonomous UAV that can navigate in a GPS-denied urban environment for investigation of the event; this could offer several advantages for time-critical and precise, localized response information necessary for life-saving decision-making. Thus, in order to implement a complete intelligent sensor framework, the need for both an intelligent static ground acoustic autonomous unattended sensors (AAUS) and improvements to GPS-degraded localization has become apparent for applications such as anomaly detection, public safety, as well as intelligence surveillance and reconnaissance (ISR) operations. Distributed AAUS networks could provide end-users with near real-time actionable information for large urban environments with limited resources. Complete ISR mission profiles require a UAV to fly in GPS challenging or denied environments such as natural or urban canyons, at least in a part of a mission. This dissertation addresses, 1) the development of intelligent sensor framework through the development of a static ground AAUS capable of machine learning for audio feature classification and 2) GPS impaired localization through a formal framework for trajectory-based flight navigation for unmanned aircraft systems (UAS) operating BVLOS in low-altitude urban airspace. Our AAUS sensor method utilizes monophonic sound event detection in which the sensor detects, records, and classifies each event utilizing supervised machine learning techniques [90]. We propose a simulated framework to enhance the performance of localization in GPS-denied environments. We do this by using a new representation of 3D geospatial data using planar features that efficiently capture the amount of information required for sensor-based GPS navigation in obstacle-rich environments. The results from this dissertation would impact both military and civilian areas of research with the ability to react to events and navigate in an urban environment. / Doctor of Philosophy / Emergency scenarios such as missing persons or catastrophic events in urban areas require first responders to gain situational awareness motivating researchers to investigate intelligent sensor frameworks that utilize drones for observation prompting questions such as: How can responders detect and classify acoustic anomalies using unattended sensors? and How do they remotely navigate in GPS-denied urban environments using drones to potentially investigate such an event? This dissertation addresses the first question through the development of intelligent WSN systems that can provide time-critical and precise, localized environmental information necessary for decision-making. At Virginia Tech, we have developed a static ground Acoustic Autonomous Unattended Sensor (AAUS) capable of machine learning for audio feature classification. The prior arts of intelligent AAUS and network architectures do not account for network failure, jamming capabilities, or remote scenarios in which cellular data wifi coverage are unavailable [78, 90]. Lacking a framework for such scenarios illuminates vulnerability in operational integrity for proposed solutions in homeland security applications. We address this through data ferrying, a communication method in which a mobile node, such as a drone, physically carries data as it moves through the environment to communicate with other sensor nodes on the ground. When examining the second question of navigation/investigation, concerns of safety arise in urban areas regarding drones due to GPS signal loss which is one of the first problems that can occur when a drone flies into a city (such as New York City). If this happens, potential crashes, injury and damage to property are imminent because the drone does not know where it is in space. In these GPS-denied situations traditional methods use point clouds (a set of data points in space (X,Y,Z) representing a 3D object [107]) constructed from laser radar scanners (often seen in a Microsoft Xbox Kinect sensor) to find itself. The main drawback from using methods such as these is the accumulation of error and computational complexity of large data-sets such as New York City. An advantage of cities is that they are largely flat; thus, if you can represent a building with a plane instead of 10,000 points, you can greatly reduce your data and improve algorithm performance. This dissertation addresses both the needs of an intelligent sensor framework through the development of a static ground AAUS capable of machine learning for audio feature classification as well as GPS-impaired localization through a formal framework for trajectory-based flight navigation for UAS operating BVLOS in low altitude urban and suburban environments.

<p>Static terrestrial laser scanners have been increasingly used in three-dimensional data acquisition since it can rapidly provide accurate measurements with high resolution. Several scans from multiple viewpoints are necessary to achieve complete coverage of the surveyed objects due to occlusion and large object size. Therefore, in order to reconstruct three-dimensional models of the objects, the task of registration is required to transform several individual scans into a common reference frame. This thesis introduces three alternative approaches for the coarse registration of two adjacent scans, namely, feature-based approach, pseudo-conjugate point-based method, and closed-form solution. In the feature-based approach, linear and planar features in the overlapping area of adjacent scans are selected as registration primitives. The pseudo-conjugate point-based method utilizes non-corresponding points along common linear and planar features to estimate transformation parameters. The pseudo-conjugate point-based method is simpler than the feature-based approach since the partial derivatives are easier to compute. In the closed-form solution, a rotation matrix is first estimated by using a unit quaternion, which is a concise description of the rotation. Afterward, the translation parameters are estimated with non-corresponding points along the linear or planar features by using the pseudo-conjugate point-based method. Alternative approaches for fitting a line or plane to data with errors in three-dimensional space are investigated.</p><p><br></p><p>Experiments are conducted using simulated and real datasets to verify the effectiveness of the introduced registration procedures and feature fitting approaches. The proposed two approaches of line fitting are tested with simulated datasets. The results suggest that these two approaches can produce identical line parameters and variance-covariance matrix. The three registration approaches are tested with both simulated and real datasets. In the simulated datasets, all three registration approaches produced equivalent transformation parameters using linear or planar features. The comparison between the simulated linear and planar features shows that both features can produce equivalent registration results. In the real datasets, the three registration approaches using the linear or planar features also produced equivalent results. In addition, the results using real data indicates that the registration approaches using planar features produced better results than the approaches using linear features. The experiments show that the pseudo-conjugate point-based approach is easier to implement than the feature-based approach. The pseudo-conjugate point-based method and feature-based approach are nonlinear, so an initial guess of transformation parameters is required in these two approaches. Compared to the nonlinear approaches, the closed-form solution is linear and hence it can achieve the registration of two adjacent scans without the requirement of any initial guess for transformation parameters. Therefore, the pseudo-conjugate point-based method and closed-form solution are the preferred approaches for coarse registration using linear or planar features. In real practice, the planar features would have a better preference when compared to linear features since the linear features are derived indirectly by the intersection of neighboring planar features. To get enough lines with different orientations, planes that are far apart from each other have to be extrapolated to derive lines.</p><div><br></div>

Visual Simultaneous Localisation and Mapping (SLAM) is concerned with simultaneously estimating the pose of a camera and a map of the environment from a sequence of images. Traditionally, sparse maps comprising isolated point features have been employed, which facilitate robust localisation but are not well suited to advanced applications. In this thesis, we present map representations that allow a more dense description of the environment. In one approach, planar features are used to represent textured planar surfaces in the scene. This model is applied within a visual SLAM framework based on the Extended Kalman Filter. We presents solutions to several challenges which arise from this approach.

Visual Simultaneous Localisation and Mapping (SLAM) is concerned with simultaneously estimating the pose of a camera and a map of the environment from a sequence of images. Traditionally, sparse maps comprising isolated point features have been employed, which facilitate robust localisation but are not well suited to advanced applications. In this thesis, we present map representations that allow a more dense description of the environment. In one approach, planar features are used to represent textured planar surfaces in the scene. This model is applied within a visual SLAM framework based on the Extended Kalman Filter. We presents solutions to several challenges which arise from this approach.