Master of Science / Department of Computing and Information Sciences / David A. Gustafson / IEDs are a prevailing threat to today’s armed forces and civilians. With some IEDs being well concealed and planted sometimes days or weeks prior to detonation, it is extremely difficult to detect their presence. Remotely triggered IEDs do offer an indirect method of detection as an insurgent must monitor the IED’s kill zone and detonate the device once the intended target is in range. Within the safe confines of a video game we can model the behavior of an insurgent using remotely triggered IEDs. Specifically, we can build a model of the sequence of actions an insurgent goes through immediately prior to detonating an IED. Using this insurgent model, we can recognize the behavior an insurgent would exhibit before detonating an IED. Once the danger level reaches a certain threshold, we can then react by changing our original course to a new one that does not cross the area we believe an IED to be in. We can show proof of concept of this by having human players take on the role of an insurgent in an online video game in which they try to destroy an autonomous agent. Successful tactics used by the autonomous agent should then be good tactics in the real world as well.

Emotion adds an important element to the discussion of how information is conveyed and processed by humans; indeed, it plays an important role in the contextual understanding of messages. This research is centered on investigating relevant features for affect classification, along with modeling the multimodal and multitemporal nature of emotion. The use of formant-based features for affect classification is explored. Since linear predictive coding (LPC) based formant estimators often encounter problems with modeling speech elements, such as nasalized phonemes and give inconsistent results for bandwidth estimation, a robust formant-tracking algorithm was introduced to better model the formant and spectral properties of speech. The algorithm utilizes Gaussian mixtures to estimate spectral parameters and refines the estimates using maximum a posteriori (MAP) adaptation. When the method was used for features extraction applied to emotion classification, the results indicate that an improved formant-tracking method will also provide improved emotion classification accuracy. Spectral features contain rich information about expressivity and emotion. However, most of the recent work in affective computing has not progressed beyond analyzing the mel-frequency cepstral coefficients (MFCC’s) and their derivatives. A novel method for characterizing spectral peaks was introduced. The method uses a multi-resolution sinusoidal transform coding (MRSTC). Because of MRSTC’s high precision in representing spectral features, including preservation of high frequency content not present in the MFCC’s, additional resolving power was demonstrated. Facial expressions were analyzed using 53 motion capture (MoCap) markers. Statistical and regression measures of these markers were used for emotion classification along the voice features. Since different modalities use different sampling frequencies and analysis window lengths, a novel classifier fusion algorithm was introduced. This algorithm is intended to integrate classifiers trained at various analysis lengths, as well as those obtained from other modalities. Classification accuracy was statistically significantly improved using a multimodal-multitemporal approach with the introduced classifier fusion method. A practical application of the techniques for emotion classification was explored using social dyadic plays between a child and an adult. The Multimodal Dyadic Behavior (MMDB) dataset was used to automatically predict young children’s levels of engagement using linguistic and non-linguistic vocal cues along with visual cues, such as direction of a child’s gaze or a child’s gestures. Although this and similar research is limited by inconsistent subjective boundaries, and differing theoretical definitions of emotion, a significant step toward successful emotion classification has been demonstrated; key to the progress has been via novel voice and visual features and a newly developed multimodal-multitemporal approach.

The digital footprints collected from the prevailing sensing systems provide novel ways to perceive an individual's behaviors. Furthermore, large collections of digital footprints from communities bring novel understandings of human behaviors from the community perspective (community behaviors), such as investigating their characteristics and learning the hidden human intelligence. The perception of human behaviors from the sensing digital footprints enables novel applications for the sensing systems. Bases on the digital footprints collected with accelerometer-embedded mobile phones and GPS equipped taxis, in this dissertation we present our work in recognizing individual behaviors, capturing community behaviors and demonstrating the novel services enabled. With the GPS footprints of a taxi, we summarize the individual anomalous passenger delivery behaviors and improve the recognition efficiency of the existing method iBOAT by introducing an inverted index mechanism. Besides, based on the observations in real life, we propose a method to detect the work-shifting events of an individual taxi. With real-life large-scale GPS traces of thousands of taxis, we investigate the anomalous passenger delivery behaviors and work shifting behaviors from the community perspective and exploit taxi serving strategies. We find that most anomaly behaviors are intentional detours and high detour inclination won't make taxis the top players. And the spatial-temporal distribution of work shifting events in the taxi community reveals their influences. While exploiting taxi serving strategies, we propose a novel method to find the initial intentions in passenger finding. Furthermore, we present a smart taxi system as an example to demonstrate the novel applications that are enabled by the perceived individual and community behaviors

Les empreintes digitales recueillies par détection systèmes offrent de nouvelles façons de percevoir les comportements d'un individu. En outre, de grandes collections d'empreintes numériques des communautés apportent de nouvelles compréhensions des comportements humains. La perception des comportements humains à partir des empreintes digitales de détection permet de construire des nouvelles applications sur les systèmes de détection. D’après les empreintes digitales recueillies avec l'accéléromètre embarqué dans les téléphones mobiles et les taxis équipés avec GPS, nous présentons ici notre travail sur la reconnaissance des comportements individuels, la capture des comportements communautaires et la démonstration des nouveaux services activés. En reconnaissant les comportements individuels, nous présentons la reconnaissance des activités physiques d'une personne avec les lectures de l'accéléromètre recueillies à partir des téléphones mobiles mis dans les poches autour de la zone pelvienne. Avec les empreintes GPS d'un taxi, nous résumons les comportements anormaux du transport des passagers pour un individu et améliorons l'efficacité de la reconnaissance de la méthode existante IBOAT. En outre, sur la base des observations dans la vie réelle, nous proposons une méthode pour détecter les événements de changement de service d’un taxi individuel. Avec des traces GPS à grande échelle et à l’aide des milliers de taxis, nous étudions les comportements anormaux pour le transport des passagers et les comportements de changement de travail et exploitons les stratégies de service de taxi. En outre, nous présentons un système intelligent de taxi comme une étude exemplaire des nouvelles applications qui s’appuie sur les comportements perçus individuelles et communautaires / The digital footprints collected from the prevailing sensing systems provide novel ways to perceive an individual's behaviors. Furthermore, large collections of digital footprints from communities bring novel understandings of human behaviors from the community perspective (community behaviors), such as investigating their characteristics and learning the hidden human intelligence. The perception of human behaviors from the sensing digital footprints enables novel applications for the sensing systems. Bases on the digital footprints collected with accelerometer-embedded mobile phones and GPS equipped taxis, in this dissertation we present our work in recognizing individual behaviors, capturing community behaviors and demonstrating the novel services enabled. With the GPS footprints of a taxi, we summarize the individual anomalous passenger delivery behaviors and improve the recognition efficiency of the existing method iBOAT by introducing an inverted index mechanism. Besides, based on the observations in real life, we propose a method to detect the work-shifting events of an individual taxi. With real-life large-scale GPS traces of thousands of taxis, we investigate the anomalous passenger delivery behaviors and work shifting behaviors from the community perspective and exploit taxi serving strategies. We find that most anomaly behaviors are intentional detours and high detour inclination won't make taxis the top players. And the spatial-temporal distribution of work shifting events in the taxi community reveals their influences. While exploiting taxi serving strategies, we propose a novel method to find the initial intentions in passenger finding. Furthermore, we present a smart taxi system as an example to demonstrate the novel applications that are enabled by the perceived individual and community behaviors