Social perception is dynamic and ambiguous. Whilst previous research favoured bottom-up views where observed actions are matched to higher level (or motor) representations, recent accounts suggest top-down processes where prior knowledge guides perception of others’ actions, in a predictive manner. This thesis investigated how person-specific models of others’ typical behaviour in different situations are reactivated when they are re-encountered and predict their actions, using strictly controlled computer-based action identification tasks, event-related potentials (ERPs), as well as recording participants’ actions via motion tracking (using the Microsoft Kinect Sensor). The findings provided evidence that knowledge about seen actor’s typical behaviour is used in action observation. It was found, first, that actions are identified faster when performed by an actor that typically performed these actions compared to another actor who only performed them rarely (Chapters Two and Three). These effects were specific to meaningful actions with objects, not withdrawals from them, and went along with action-related ERP responses (oERN, observer related error negativity). Moreover, they occurred despite current actor identity not being relevant to the task, and were largely independent of the participants’ ability to report the individual’s behaviour. Second, the findings suggested that these predictive person models are embodied such that they influenced the observers own motor systems, even when the relevant actors were not seen acting (Chapter Four). Finally, evidence for theses person-models were found when naturalistic responding was required when participants had to use their feet to ‘block’ an incoming ball (measured by the Microsoft Kinect Sensor), where they made earlier and more pronounced movements when the observed actor behaved according to their usual action patterns (Chapter Five). The findings are discussed with respect to recent predictive coding theories of social perception, and a new model is proposed that integrates the findings.

When interacting, people imitate each other. This tendency is truly ubiquitous and occurs in many different situations and behaviours. But what causes it? Several mechanisms have been proposed to contribute to imitation. In this thesis, I focus on three candidate mechanisms: simulation, temporal adaptation, and the goal to affiliate with others. I start by discussing different imitative behaviours, and reviewing the evidence that imitation might at times emerge spontaneously. I also review the evidence suggesting that the three candidate mechanisms might be involved in such emergent imitation. Then, I present three sets of experiments. In the first set, I investigate the role of simulation in language processing. In three experiments, I test the hypothesis that comprehenders use their language production system to simulate their interlocutor, which in turn facilitates their ability to predict the next word they will see or hear. I manipulate whether participants read the sentences silently or aloud and measure their ability to predict the final word of a sentence. My results demonstrate that prediction is enhanced when people use their production system during reading aloud. This gives some credence to the idea that simulation is routinely engaged in language processing, which in turn opens up a possibility that it may contribute to linguistic imitation. In the second set of experiments, I investigate whether temporal adaptation leads agents to imitate features of their partner’s actions. In three experiments, I test this by manipulating the partner’s response speed and the information about the partner’s actions. I show that agents imitate response speed when they are able to observe the partner. Moreover, they adapt to the specific temporal pattern of their partner’s actions. These findings provide evidence for the engagement of the temporal adaptation mechanism during motor interactions, and for its involvement in imitation. In the third set of experiments, I turn to the hypothesis that people engage in linguistic imitation because they want to harness the social benefits it brings. I investigate a key assumption of this hypothesis: that imitation has positive consequences for the social interaction. In three experiments, I manipulate whether participants’ word choice is imitated or counter-imitated by their conversational partner and measure how it affects the participants’ evaluation of the interaction and the partner, and their willingness to cooperate with the partner. I find evidence that linguistic imitation has positive social consequences. These results are consonant with the claim that imitation is motivated by the goal to affiliate and foster social relations. Taken together, these findings suggest that imitation might occur both in motor actions and language, and that it might have diverse causes. My work on language suggests that the tendency to linguistically imitate others could both result from the simulation mechanism, and be motivated by the goal to affiliate. My work on motor actions shows that automatic temporal adaptation contributes to emergent imitation during interactions. This research is conducive to the greater aim of cross-examining the currently known mechanisms of imitation.

Being able to predict the goal of other people’s actions is an important aspect of our daily lives. This ability allows us to interact timely with others and adjust our behaviour appropriately. The general aim of the present thesis was to explore which processes best explain our ability to predict other people’s action goals during development. There are different theories concerning this ability. Some stress the fact that observation of others actions activate the same areas of the brain involved in our own action production, this way helping us to understand what they are doing. Other theories suggest that we understand actions independently of our own motor proficiency. For example, the ability to predict other peoples’ action goals could be based on visual experience seeing others actions acquired trough time or on the assumption that actions will be performed in a rational way. The studies included in this thesis use eye tracking to study infants’ and adults’ action prediction during observation of goal directed actions. Prediction is operationalized as predictive gaze shifts to the goal of the action. Study I showed that infants are sensitive to the functionality of hand configuration and predict the goal of reaching actions but not moving fists. Fourteen-month-olds also looked earlier to the goal of reaching actions when the goal was to contain rather than displace, indicating that the overarching goal (contain/displace) impact the ability to predict local action goals, in this case the goal of the initial reaching action. Study II demonstrated that 6-month-olds, an age when infants have not yet started placing objects into containers, did not look to the container ahead of time when observing another person placing objects into containers. They did, however, look to the container ahead of time when a ball was moving on its own. The results thus indicate that different processes might be used to predict human actions and other events. Study III showed that 8-month-old infants in China looked to the mouth of an actor eating with chopsticks ahead of time but not when the actor was eating with a spoon. Swedish infants on the other hand looked predictively to the mouth when the actor was eating with a spoon but not with chopsticks. This study demonstrates that prediction of others’ goal directed actions is not simply based on own motor ability (as assumed in Study I and II) but rather on a combination of visual/cultural experience and own motor ability. The results of these studies suggest that both own motor proficiency as well as visual experience with observing similar actions is necessary for our ability to predict other people’s action goals. These results are discusses in the light of a newer account of the mirror neuron system taking both statistical regularities in the environment and own motor capabilities into account.

No description available.

Being able to generate predictions about what is going to happen next while observing other people’s actions plays a crucial role in our daily lives. Different theoretical explanations for the underlying processes of humans’ action prediction abilities have been suggested. Whereas an embodied account posits that predictive gaze relies on embodied simulations in the observer’s motor system, other accounts do not assume a causal role of the motor system for action prediction. The general aim of this thesis was to augment current knowledge about the functional mechanisms behind humans’ action prediction abilities. In particular, the present thesis outlines and tests an embodied account of action prediction. The second aim of this thesis was to extend prior action prediction studies by exploring infants’ online gaze during observation of social interactions. The thesis reports 3 eye-tracking studies that were designed to measure adults’ and infants’ predictive eye movements during observation of different manual and social actions. The first two studies used point-light displays of manual reaching actions as stimuli to isolate human motion information. Additionally, Study II used transcranial magnetic stimulation (TMS) to directly modify motor cortex activity. Study I showed that kinematic information from biological motion can be used to anticipate the goal of other people’s point-light actions and that the presence of biological motion is sufficient for anticipation to occur. Study II demonstrated that TMS-induced temporary lesions in the primary motor cortex selectively affected observers’ gaze latencies. Study III examined 12-month-olds’ online gaze during observation of a give-and-take interaction between two individuals. The third study showed that already at one year of age infants shift their gaze from a passing hand to a receiving hand faster when the receiving hand forms a give-me gesture compared to an inverted hand shape. The reported results from this thesis make two major contributions. First, Studies I and II provide evidence for an embodied account of action prediction by demonstrating a direct connection between anticipatory eye movements and motor cortex activity. These findings support the interpretation that predictive eye movements are driven by a recruitment of the observer’s own motor system. Second, Study III implicates that properties of social action goals influence infants’ online gaze during action observation. It further suggests that at one year of age infants begin to show sensitivity to social goals within the context of give-and-take interactions while observing from a third-party perspective.

In this dissertation, we address the problem of understanding human activities in videos by developing a two-pronged approach: coarse level modeling of scene activities and fine level modeling of individual activities. At the coarse level, where the resolution of the video is low, we rely on person tracks. At the fine level, richer features are available to identify different parts of the human body, therefore we rely on the body joint tracks. There are three main goals of this dissertation: (1) identify unusual activities at the coarse level, (2) recognize different activities at the fine level, and (3) predict the behavior for synthesizing and tracking activities at the fine level. The first goal is addressed by modeling activities at the coarse level through two novel and complementing approaches. The first approach learns the behavior of individuals by capturing the patterns of motion and size of objects in a compact model. Probability density function (pdf) at each pixel is modeled as a multivariate Gaussian Mixture Model (GMM), which is learnt using unsupervised expectation maximization (EM). In contrast, the second approach learns the interaction of object pairs concurrently present in the scene. This can be useful in detecting more complex activities than those modeled by the first approach. We use a 14-dimensional Kernel Density Estimation (KDE) that captures motion and size of concurrently tracked objects. The proposed models have been successfully used to automatically detect activities like unusual person drop-off and pickup, jaywalking, etc. The second and third goals of modeling human activities at the fine level are addressed by employing concepts from theory of chaos and non-linear dynamical systems. We show that the proposed model is useful for recognition and prediction of the underlying dynamics of human activities. We treat the trajectories of human body joints as the observed time series generated from an underlying dynamical system. The observed data is used to reconstruct a phase (or state) space of appropriate dimension by employing the delay-embedding technique. This transformation is performed without assuming an exact model of the underlying dynamics and provides a characteristic representation that will prove to be vital for recognition and prediction tasks. For recognition, properties of phase space are captured in terms of dynamical and metric invariants, which include the Lyapunov exponent, correlation integral, and correlation dimension. A composite feature vector containing these invariants represents the action and will be used for classification. For prediction, kernel regression is used in the phase space to compute predictions with a specified initial condition. This approach has the advantage of modeling dynamics without making any assumptions about the exact form (polynomial, radial basis, etc.) of the mapping function. We demonstrate the utility of these predictions for human activity synthesis and tracking.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / The availability of low cost sensors and mobile devices allowed many advances in research of ubiquitous and pervasive computing area. With the capture of contextual data provided by the sensors attached to these devices it is possible to obtain user state information and the environment, and thus map the relationship between them. One approach to map these relationships are the activities performed by the user, which also are part of the context itself. However, even that human activities could cause injuries, there is not much discussion in the academy of how ubiquitous computing could assess the risk related to them. In this sense, the Activity Project aims to determine the risk situations related to activities performed by people in a context aware environment, through a middleware that considers the risk in the actions that composes an activity and the user performance while performing an activity. This thesis aims to specify the Activity Manager middleware layer proposed for the Activity Project, whose goal is to address issues relating to the prediction of actions and activities and the detection of risk situation in the actions performed by an user. The model developed to address the composition and prediction of activities is based on the Activity Theory, while the risk in actions is determined by changes in the physiological context of the user caused by the actions performed by itself, modeled through the model named Hyperspace Analogous to Context. Tests were conducted and developed models outperformed proposals found for action prediction, with an accuracy of 78.69%, as well as for risk situations detection, with an accuracy of 98.94%, showing the efficiency of the proposed solution. / A popularização de sensores de baixo custo e de dispositivos móveis permitiu diversos avanços nas pesquisas da área de computação ubíqua e pervasiva. Com a captura dos dados contextuais providos pelos sensores acoplados a estes dispositivos é possível obter informações do estado do usuário e do ambiente, e dessa forma mapear a relação entre ambos. Uma das possíveis abordagens para mapear essas relações são as atividades executadas pelo usuário, que inclusive são parte constituinte do próprio contexto. Entretanto, mesmo que as atividades humanas possam causar danos físicos, não há muita discussão na academia de como a computação ubíqua poderia avaliar esse risco relacionado a elas. Neste sentido, o projeto Activity Project objetiva determinar situações de risco no momento da realização de atividades desempenhadas por pessoas em um ambiente sensível ao contexto, através de um middleware sensível ao contexto que considera o risco nas ações que compõe uma atividade e o desempenho do usuário enquanto executa uma atividade. Esta dissertação tem por objetivo especificar a camada Gerência de Atividades do middleware proposto para o Activity Project, cujo objetivo é tratar as questões referentes à predição de ações e atividades e a detecção de situações de risco em ações. O modelo desenvolvido para tratar a composição das atividades e a predição das mesmas baseia-se na Teoria da Atividade, enquanto que o risco em ações é determinado pelas mudanças no contexto fisiológico do usuário, modeladas através do modelo Hiperespaço Análogo ao Contexto. Nos testes realizados os modelos desenvolvidos superaram as propostas encontradas até o momento para a predição de ações com uma a precisão de 78,69%, bem como para a determinação de situações de risco com uma precisão de 98,94%, demonstrando a eficácia da solução proposta.

Air traffic controllers (ATCos) monitor flight operations and resolve predicted aircraft conflicts to ensure safe flights, making them one of the essential human operators in air traffic control systems. Researchers have been studying ATCos with human subjective approaches to understand their tasks and air traffic managing processes. As a result, models were developed to predict ATCo actions. However, there is a gap between our knowledge and the real-world. The developed models have never been validated against the real-world, which creates uncertainties in our understanding of how ATCos detect and resolve predicted aircraft conflicts. Moreover, we do not know how information from air traffic control systems affects their actions. This Ph.D. dissertation work introduces methods to evaluate existing ATCo action prediction models. It develops a prediction model based on flight contextual information (information describing flight operations) to explain the relationship between ATCo actions and information. Unlike conventional approaches, this work takes data-driven approaches that collect large-scale flight tracking data. From the collected real-world data, ATCo actions and corresponding predicted aircraft conflicts were identified by developed algorithms. Comparison methods were developed to measure both qualitative and quantitative differences between solutions from the existing prediction models and ATCo actions on the same aircraft conflicts. The collected data is further utilized to develop an ATCo action prediction model. A hierarchical structure found from analyzing the collected ATCo actions was applied to build a structure for the model. The flight contextual information generated from the collected data was used to predict the actions. Results from this work found that the collected ATCo actions do not show any preferences on the methods to resolve aircraft conflicts. Results found that the evaluated existing prediction model does not reflect the real-world. Also, a large portion of the real conflicts was to be solved by the model both physically and operationally. Lastly, the developed prediction model showed a clear relationship between ATCo actions and applied flight contextual information. These results suggest the following takeaways. First, human actions can be identified from closed-loop data. It could be an alternative approach to collect human subjective data. Second, the importance of evaluating models before implications. Third, potentials to utilize the flight contextual information to conduct high-end prediction models.