Classification of emotions plays a very important role in affective computing and has real-world applications in fields as diverse as entertainment, medical, defense, retail, and education. These applications include video games, virtual reality, pain recognition, lie detection, classification of Autistic Spectrum Disorder (ASD), analysis of stress levels, and determining attention levels. This vast range of applications motivated us to study automatic emotion recognition which can be done by using facial expression, speech, and physiological data. A person’s physiological signals such are heart rate, and blood pressure are deeply linked with their emotional states and can be used to identify a variety of emotions; however, they are less frequently explored for emotion recognition compared to audiovisual signals such as facial expression and voice. In this thesis, we investigate a multimodal approach to emotion recognition using physiological signals by showing how these signals can be combined and used to accurately identify a wide range of emotions such as happiness, sadness, and pain. We use the deep convolutional neural network for our experiments. We also detail comparisons between gender-specific models of emotion. Our investigation makes use of deep convolutional neural networks, which are the latest state of the art in supervised learning, on two publicly available databases, namely DEAP and BP4D+. We achieved an average emotion recognition accuracy of 98.89\% on BP4D+ and on DEAP it is 86.09\% for valence, 90.61\% for arousal, 90.48\% for liking and 90.95\% for dominance. We also compare our results to the current state of the art, showing the superior performance of our method.

Many physical and algorithmic swarms utilize inter-agent communication to achieve advanced swarming behaviors. These swarms are inspired by biological swarms that can be seen throughout nature and include bee swarms, ant colonies, fish schools, and bird flocks. These biological swarms do not utilize inter-agent communication like their physical and algorithmic counterparts. Instead, organisms in nature rely on a local awareness of other swarm members that facilitates proper swarm motion and behavior. This research aims to pursue an effective swarm algorithm using only line-of-sight proximity information and no inter-agent communication. It is expected that the swarm performance will be lower than that of a swarm utilizing inter-agent communication. Various sensors were studied and considered for this project but infrared sensors were ultimately selected. These sensors were then modeled in software using a neural network in order to calculate the minimum number of infrared transmitters and receivers necessary for each agent while still ensuring the proper functionality of the swarm. A physical swarm was designed and constructed using the selected number and type of infrared sensors, DC stepper motors, a 16-bit microprocessor, and additional infrared proximity sensors. The performance of the physical robots was compared to the performance of the simulated robots under similar conditions. It was observed that the physical and simulated swarms performed similarly and that swarm behavior with no inter-agent communication was successfully achieved.

This dissertation addresses the problem of employing 3D depth information on solving a number of traditional challenging computer vision/graphics problems. Humans have the abilities of perceiving the depth information in 3D world, which enable humans to reconstruct layouts, recognize objects and understand the geometric space and semantic meanings of the visual world. Therefore it is significant to explore how the 3D depth information can be utilized by computer vision systems to mimic such abilities of humans. This dissertation aims at employing 3D depth information to solve vision/graphics problems in the following aspects: scene understanding, image enhancements and 3D reconstruction and modeling. In addressing scene understanding problem, we present a framework for semantic segmentation and object recognition on urban video sequence only using dense depth maps recovered from the video. Five view-independent 3D features that vary with object class are extracted from dense depth maps and used for segmenting and recognizing different object classes in street scene images. We demonstrate a scene parsing algorithm that uses only dense 3D depth information to outperform using sparse 3D or 2D appearance features. In addressing image enhancement problem, we present a framework to overcome the imperfections of personal photographs of tourist sites using the rich information provided by large-scale internet photo collections (IPCs). By augmenting personal 2D images with 3D information reconstructed from IPCs, we address a number of traditionally challenging image enhancement techniques and achieve high-quality results using simple and robust algorithms. In addressing 3D reconstruction and modeling problem, we focus on parametric modeling of flower petals, the most distinctive part of a plant. The complex structure, severe occlusions and wide variations make the reconstruction of their 3D models a challenging task. We overcome these challenges by combining data driven modeling techniques with domain knowledge from botany. Taking a 3D point cloud of an input flower scanned from a single view, each segmented petal is fitted with a scale-invariant morphable petal shape model, which is constructed from individually scanned 3D exemplar petals. Novel constraints based on botany studies are incorporated into the fitting process for realistically reconstructing occluded regions and maintaining correct 3D spatial relations. The main contribution of the dissertation is in the intelligent usage of 3D depth information on solving traditional challenging vision/graphics problems. By developing some advanced algorithms either automatically or with minimum user interaction, the goal of this dissertation is to demonstrate that computed 3D depth behind the multiple images contains rich information of the visual world and therefore can be intelligently utilized to recognize/ understand semantic meanings of scenes, efficiently enhance and augment single 2D images, and reconstruct high-quality 3D models.

Quasi-articulated objects, such as human beings, are among the most commonly seen objects in our daily lives. Extensive research have been dedicated to 3D shape reconstruction and motion analysis for this type of objects for decades. A major motivation is their wide applications, such as in entertainment, surveillance and health care. Most of existing studies relied on one or more regular video cameras. In recent years, commodity depth sensors have become more and more widely available. The geometric measurements delivered by the depth sensors provide significantly valuable information for these tasks. In this dissertation, we propose three algorithms for monocular pose estimation and shape reconstruction of quasi-articulated objects using a single commodity depth sensor. These three algorithms achieve shape reconstruction with increasing levels of granularity and personalization. We then further develop a method for highly detailed shape reconstruction based on our pose estimation techniques. Our first algorithm takes advantage of a motion database acquired with an active marker-based motion capture system. This method combines pose detection through nearest neighbor search with pose refinement via non-rigid point cloud registration. It is capable of accommodating different body sizes and achieves more than twice higher accuracy compared to a previous state of the art on a publicly available dataset. The above algorithm performs frame by frame estimation and therefore is less prone to tracking failure. Nonetheless, it does not guarantee temporal consistent of the both the skeletal structure and the shape and could be problematic for some applications. To address this problem, we develop a real-time model-based approach for quasi-articulated pose and 3D shape estimation based on Iterative Closest Point (ICP) principal with several novel constraints that are critical for monocular scenario. In this algorithm, we further propose a novel method for automatic body size estimation that enables its capability to accommodate different subjects. Due to the local search nature, the ICP-based method could be trapped to local minima in the case of some complex and fast motions. To address this issue, we explore the potential of using statistical model for soft point correspondences association. Towards this end, we propose a unified framework based on Gaussian Mixture Model for joint pose and shape estimation of quasi-articulated objects. This method achieves state-of-the-art performance on various publicly available datasets. Based on our pose estimation techniques, we then develop a novel framework that achieves highly detailed shape reconstruction by only requiring the user to move naturally in front of a single depth sensor. Our experiments demonstrate reconstructed shapes with rich geometric details for various subjects with different apparels. Last but not the least, we explore the applicability of our method on two real-world applications. First of all, we combine our ICP-base method with cloth simulation techniques for Virtual Try-on. Our system delivers the first promising 3D-based virtual clothing system. Secondly, we explore the possibility to extend our pose estimation algorithms to assist physical therapist to identify their patients’ movement dysfunctions that are related to injuries. Our preliminary experiments have demonstrated promising results by comparison with the gold standard active marker-based commercial system. Throughout the dissertation, we develop various state-of-the-art algorithms for pose estimation and shape reconstruction of quasi-articulated objects by leveraging the geometric information from depth sensors. We also demonstrate their great potentials for different real-world applications.

A central problem in artificial intelligence is to choose actions to maximize reward in a partially observable, uncertain environment. To do so, we must obtain an accurate environment model, and then plan to maximize reward. However, for complex domains, specifying a model by hand can be a time consuming process. This motivates an alternative approach: learning a model directly from observations. Unfortunately, learning algorithms often recover a model that is too inaccurate to support planning or too large and complex for planning to succeed; or, they require excessive prior domain knowledge or fail to provide guarantees such as statistical consistency. To address this gap, we propose spectral subspace identification algorithms which provably learn compact, accurate, predictive models of partially observable dynamical systems directly from sequences of action-observation pairs. Our research agenda includes several variations of this general approach: spectral methods for classical models like Kalman filters and hidden Markov models, batch algorithms and online algorithms, and kernel-based algorithms for learning models in high- and infinite-dimensional feature spaces. All of these approaches share a common framework: the model’s belief space is represented as predictions of observable quantities and spectral algorithms are applied to learn the model parameters. Unlike the popular EM algorithm, spectral learning algorithms are statistically consistent, computationally efficient, and easy to implement using established matrixalgebra techniques. We evaluate our learning algorithms on a series of prediction and planning tasks involving simulated data and real robotic systems.

Collaborative filtering based recommender systems use information about a user's preferences to make personalized predictions about content, such as topics, people, or products, that they might find relevant. As the volume of accessible information and active users on the Internet continues to grow, it becomes increasingly difficult to compute recommendations quickly and accurately over a large dataset. In this study, we will introduce an algorithmic framework built on top of Apache Spark for parallel computation of the neighborhood-based collaborative filtering problem, which allows the algorithm to scale linearly with a growing number of users. We also investigate several different variants of this technique including user and item-based recommendation approaches, correlation and vector-based similarity calculations, and selective down-sampling of user interactions. Finally, we provide an experimental comparison of these techniques on the MovieLens dataset consisting of 10 million movie ratings.

The World Health Organization claims that there are more than 285 million blind and visually impaired people in the world. In the US, 25 million Americans suffer from total or partial vision loss. As a result of their impairment, they struggle with mobility problems, especially the risk of falling. According to the National Council On Aging, falls are among the primary causes for fatal injury and they are the most common cause of non-fatal trauma-related hospital admissions among older adults. Visibility, an organization that helps visually impaired people, reports that people with visual impairments are twice as likely to fall as their sighted counterparts. The Centers for Disease Control and Prevention reported that 2.5 million American adults were treated for fall-related injuries in 2013, leading to over 800,000 hospitalizations and over 27,000 deaths. The total cost of fall injuries in the United States in 2013 was $31 billion, and the financial total is expected to rise to $67.7 billion by 2020. Reducing the amount of these unexpected hospital visits saves money and expands the quality of life for the affected population. Technology has completely revolutionized how nowadays activities are conducted and how var- ious tasks are accomplished, and mobile devices are at the center of this paradigm shift. According to the Pew Research Center, 64% of American adults own a smartphone currently, and this number is trending upward. Mobile computing devices have evolved to include a plethora of data sensors that can be manipulated to create solutions for humanity, including fall prevention. Fall prevention is an area of research that focuses on strengthening safety in order to prevent falls from occurring. Many fall prevention systems use sensing devices to measure the likelihood of a fall. Sensor data are usually processed using computer vision, data mining, and machine learning techniques. This work pertains to the implementation of a smartphone-based fall prevention system for the elderly and visually impaired. The system consists of two modules: fall prevention and fall detection. Fall prevention is in charge of identifying tripping hazards in the user’s surroundings. Fall detection is in charge of detecting when falls happen and alerting a person of interest. The proposed system is challenged by multiple problems: it has to run in near real time, it has to run efficiently in a smartphone hardware, it has to process structured and unstructured environments, and many more related to image analysis (occlusion, motion blur, computational complexity, etc). The fall prevention module is divided into three parts, floor detection, object-on-floor detection, and distance estimation. The evaluation process of the best approach for floor detection achieved an accuracy of 92%, a precision of 88%, and a recall of 92%. The evaluation process of the best approach for object-on-floor detection achieved an accuracy of 90%, a precision of 56%, and a recall of 78%. The evaluation process of the best approach for distance estimation achieved a MSE error of 0.45 meters. The fall detection module is approached from two perspectives, using inertial measuring units (IMU) embedded in today’s smartphones, and using a 2D camera. The evaluation process of the solution using IMUs achieved an accuracy of 83%, a precision of 89%, and a recall of 58.2%. The evaluation process of the solution that uses a 2D camera achieved an accuracy of 85.37% and a recall of 70.97%.

This thesis is a critique on the ease in which humans tend to anthropomorphize chatbots, assigning human characteristics to entities that fundamentally will never understand the human experience. It will be further exploring these consequences on our society's socio-cultural fabric, representations of the self and identity formation in terms of communication and the essence of humanity.

In the past decade, Human Activity Recognition (HAR) has been an important part of the regular day to day life of many people. Activity recognition has wide applications in the field of health care, remote monitoring of elders, sports, biometric authentication, e-commerce and more. Each HAR application needs a unique approach to provide solutions driven by the context of the problem. In this dissertation, we are primarily discussing two application of HAR in different contexts. First, we design a novel approach for in-home, fine-grained activity recognition using multimodal wearable sensors on multiple body positions, along with very small Bluetooth beacons deployed in the environment. State-of-the-art in-home activity recognition schemes with wearable devices are mostly capable of detecting coarse-grained activities (sitting, standing, walking, or lying down), but cannot distinguish complex activities (sitting on the floor versus on the sofa or bed). Such schemes are not effective for emerging critical healthcare applications – for example, in remote monitoring of patients with Alzheimer's disease, Bulimia, or Anorexia – because they require a more comprehensive, contextual, and fine-grained recognition of complex daily user activities. Second, we introduced Watch-Dog – a self-harm activity recognition engine, which attempts to infer self-harming activities from sensing accelerometer data using wearable sensors worn on a subject's wrist. In the United States, there are more than 35,000 reported suicides with approximately 1,800 of them being psychiatric inpatients every year. Staff perform intermittent or continuous observations in order to prevent such tragedies, but a study of 98 articles over time showed that 20% to 62% of suicides happened while inpatients were on an observation schedule. Reducing the instances of suicides of inpatients is a problem of critical importance to both patients and healthcare providers. Watch-dog uses supervised learning algorithm to model the system which can discriminate the harmful activities from non-harmful activities. The system is not only very accurate but also energy efficient. Apart from these two HAR systems, we also demonstrated the difference in activity pattern between elder and younger age group. For this experiment, we used 5 activities of daily living (ADL). Based on our findings we recommend that a context aware age-specific HAR model would be a better solution than all age-mixed models. Additionally, we find that personalized models for each individual elder person perform better classification than mixed models.

Artificial intelligence is an emerging technology; something far beyond smartphones, cloud integration, or surgical microchip implantation. Utilizing the work of Ray Kurzweil, Nick Bostrom, and Steven Shaviro, this thesis investigates technology and artificial intelligence through the lens of the cinema. It does this by mapping contemporary concepts and the imagined worlds in film as an intersection of reality and fiction that examines issues of individual identity and alienation. I look at a non-linear timeline of films involving machine advancement, machine intelligence, and stages of post-human development; Elysium (2013) and Surrogates (2009) are about technology as an extension of the self, The Terminator franchise (1984-2015), Blade Runner (1982), and Bicentennial Man (1999) portray artificial intelligent androids and cyborgs, Transcendence (2013) is a contemporary depiction of human consciousness fusing with technology, and Chappie and Ex Machina are both released in 2015 are situated in contemporary society with sentient artificial intelligence. Looking at these films portrayals of man’s relationship with machines creates a discourse for contemporary society’s anxiety surrounding technology. I argue that recent film’s depiction of artificial intelligence signals a contemporary change in our perception of technology, urging that we reevaluate the ways that we define our identity.