We are seeing a revolution in computing and artificial intelligence; intelligent machines have become ingrained in and improved every aspect of our lives. Despite the increasing number of intelligent devices and breakthroughs in artificial intelligence, we have yet to achieve truly intelligent environments. Audio is one of the most common sensing and actuation modalities used in intelligent devices. In this thesis, we focus on how we can more robustly integrate audio intelligence into a wide array of resource-constrained platforms that enable more intelligent environments. We present systems and methods for adaptive audio filtering that enables us to more robustly embed acoustic intelligence into a wide range of real time and resource-constrained mobile, embedded, and cyber-physical systems that are adaptable to a wide range of different applications, environments, and scenarios. First, we introduce methods for embedding audio intelligence into wearables, like headsets and helmets, to improve pedestrian safety in urban environments by using sound to detect vehicles, localize vehicles, and alert pedestrians well in advance to give them enough time to avoid a collision. We create a segmented architecture and data processing pipeline that partitions computation between embedded front-end platform and the smartphone platform. The embedded front-end hardware platform consists of a microcontroller and commercial-off-the shelf (COTS) components embedded into a headset and samples audio from an array of four MEMS microphones. Our embedded front-end platform computes a series of spatiotemporal features used to localize vehicles: relative delay, relative power, and zero crossing rate. These features are computed in the embedded front-end headset platform and transmitted wirelessly to the smartphone platform because there is not enough bandwidth to transmit more than two channels of raw audio with low latency using standard wireless communication protocols, like Bluetooth Low-Energy. The smartphone platform runs machine learning algorithms to detect vehicles, localize vehicles, and alert pedestrians. To help reduce power consumption, we integrate an application specific integrated circuit into our embedded front-end platform and create a new localization algorithm called angle via polygonal regression (AvPR) that combines the physics of audio waves, the geometry of a microphone array, and a data driven training and calibration process that enables us to estimate the high resolution direction of the vehicle while being robust to noise resulting from movements in the microphone array as we walk the streets. Second, we explore the challenges in adapting our platforms for pedestrian safety to more general and noisier scenarios, namely construction worker safety sounds of nearby power tools and machinery that are orders of magnitude greater than that of a distant vehicle. We introduce an adaptive noise filtering architecture that allows workers to filter out construction tool sounds and reveal low-energy vehicle sounds to better detect them. Our architecture combines the strengths of both the physics of audio waves and data-driven methods to more robustly filter out construction sounds while being able to run on a resource-limited mobile and embedded platform. In our adaptive filtering architecture, we introduce and incorporate a data-driven filtering algorithm, called probabilistic template matching (PTM), that leverages pre-trained statistical models of construction tools to perform content-based filtering. We demonstrate improvements that our adaptive filtering architecture brings to our audio-based urban safety wearable in real construction site scenarios and against state-of-art audio filtering algorithms, while having a minimal impact on the power consumption and latency of the overall system. We also explore how these methods can be used to improve audio privacy and remove privacy-sensitive speech from applications that have no need to detect and analyze speech. Finally, we introduce a common selective audio filtering platform that builds upon our adaptive filtering architecture for a wide range of real-time mobile, embedded, and cyber-physical applications. Our architecture can account for a wide range of different sounds, model types, and signal representations by integrating an algorithm we present called content-informed beamforming (CIBF). CIBF combines traditional beamforming (spatial filtering using the physics of audio waves) with data driven machine learning sound detectors and models that developers may already create for their own applications to enhance and filter out specified sounds and noises. Alternatively, developers can also select sounds and models from a library we provide. We demonstrate how our selective filtering architecture can improve the detection of specific target sounds and filter out noises in a wide range of application scenarios. Additionally, through two case studies, we demonstrate how our selective filtering architecture can easily integrate into and improve the performance of real mobile and embedded applications over existing state-of-art solutions, while having minimal impact on latency and power consumption. Ultimately, this selective filtering architecture enables developers and engineers to more easily embed robust audio intelligence into common objects found around us and resource-constrained systems to create more intelligent environments.

Approved for public release, distribution is unlimited / Modern technology is making virtual environments a part of daily life. However, some constraints about the usage of virtual environments, such as the need for high performance and well-configured computers, prevent users from accessing virtual environments in some places other than special computer rooms. Mobile devices may be used to solve this limitation in a virtual environment. The remote-control approach to access virtual worlds on the Internet or on a corporate network is a new concept that opens new doors to users. First step of this approach is already in use, such as games implemented for mobile devices using the screen of a mobile device as display, and has given satisfying results for some users. This research will take the user, who not only wants to be mobile but also does not want to sacrifice high resolution textures and complex models, closer to his/her goal. Mobile devices provide mobility to the user, but sacrifice not only the reality of the virtual environments but also screen size, which is very important for visibility of complex virtual environments. The hybrid approach with wireless internet connection by using mobile devices as remote control gives the user the advantages of mobility over desktop PCs. On the other hand, the realism provided by high-quality PCs on the server side exceeds the capabilities of mobile devices. / Lieutenant Junior Grade, Turkish Navy

An energy-aware on-demand Bluetooth scatternet formation and routing protocol taking into account network architecture and traffic pattern is proposed. The scatternet formation protocol is able to cope with multiple sources initiating traffic simultaneously as well as prolong network lifetime. A modified Inquiry scheme using extended ID packet is introduced for fast device discovery and power efficient propagation of route request messages with low delay. A mechanism employing POLL packets in Page processes is proposed to transfer scatternet formation and route reply information without extra overhead. In addition, the energy aware forwarding nodes selection scheme is based on local information and results in more uniform network resource utilization and improved network lifetime. Simulation results show that this protocol can provide scatternet formation with reasonable delay and with good load balance which results in prolonged network lifetime for Bluetooth-based wireless sensor networks. In this research, a metric-based scatternet formation algorithm for the Bluetooth-based sensor motes is presented. It optimizes the Bluetooth network formation from the hop distance and link quality perspectives. In addition, a smart repair mechanism is proposed to deal with link/node failure and recover the network connectivity promptly with low overhead. The experiments with the Intel Mote platform demonstrate the effectiveness of the optimizations. This research also investigates the scalability of ad hoc routing protocols in very large-scale wireless ad hoc networks. A comprehensive simulation study is conducted of the performance of an on-demand routing protocol on a very large-scale, with as many as 50,000 nodes in the network. The scalability analysis is addressed based on various network sizes, node density, traffic load, and mobility. The reasons for packet loss are analyzed and categorized at each network layer. Based on the observations, we observe the effect of the parameter selection and try to exhaust the scalability boundary of the on-demand routing protocol for wireless ad hoc networks.

Indiana University-Purdue University Indianapolis (IUPUI) / Physical activity monitoring using wearable sensors give valuable information about patient's neuro activities. Fall among ages of 60 and older in US is a leading cause for injury-related health issues and present serious concern in the public health care sector. If the emergency treatments are not on time, these injuries may result in disability, paralysis, or even death. In this work, we present an approach that early detect fall occurrences. Low power capacitive accelerometers incorporated with microcontroller processing units were utilized to early detect accurate information about fall events. Decision tree algorithms were implemented to set thresholds for data acquired from accelerometers. Data is then verified against their thresholds and the data acquisition decision unit makes the decision to save patients from fall occurrences. Daily activities are logged on an onboard memory chip with Bluetooth option to transfer the data wirelessly to mobile devices. In this work, a system prototype based on neurosignal activities was built and tested against seven different daily human activities for the sake of differentiating between fall and non-fall detection. The developed system features low power, high speed, and high reliability. Eventually, this study will lead to wearable fall detection system that serves important need within the health care sector. In this work Inter-Integrated Circuit (I2C) protocol is used to communicate between the accelerometers and the embedded control system. The data transfer from the Microcontroller unit to the mobile device or laptop is done using Bluetooth technology.

Indiana University-Purdue University Indianapolis (IUPUI) / With the advances in embedded technology and the omnipresence of smartphones, tracking systems do not need to be confined to a specific tracking environment. By introducing mobile devices into a tracking system, we can leverage their mobility and the availability of multiple sensors such as camera, Wi-Fi, Bluetooth and Inertial sensors. This thesis proposes to improve the existing tracking systems, enhanced Distributed Tracking System (e-DTS 2.0) [19] and enhanced Distributed Object Tracking System (eDOTS)[26], in the form of e-DTS 3.0 and provides an empirical analysis of these improvements. The enhancements proposed are to introduce Android-based mobile devices into the tracking system, to use multiple sensors on the mobile devices such as the camera, the Wi-Fi and Bluetooth sensors and inertial sensors and to utilize possible resources that may be available in the environment to make the tracking opportunistic. This thesis empirically validates the proposed enhancements through the experiments carried out on a prototype of e-DTS 3.0.