Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Civil Engineering, 1979. / MICROFICHE COPY AVAILABLE IN ARCHIVES AND ROTCH. / Includes bibliographies. / by David Damm. / Ph.D.

No description available.

Deep learning has become a widely used tool for inference and prediction in neuroscience research. Despite their differences, most neural network architectures convert raw input data into lower-dimensional vector representations that subsequent network layers can more easily process. Significant advancements have been made in improving latent representations in audiovisual problems. However, human neurophysiological data is often scarcer, noisier, and more challenging to learn from when integrated from multiple sources. The present work integrates neural, physiological, and behavioral data to improve human behavior, preference, and state prediction. Across five studies, we explore (i) how embeddings, or vectorized representations, can be designed to understand the context of input data better, (ii) how the attention mechanism found in transformer models can be adapted to capture crossmodal relationships in an interpretable way, and (iii) how humans make sensorimotor decisions in a realistic scenario with implications for designing automated systems. Part I focuses on improving the context for latent representations in deep neural networks. We achieve this by introducing a hierarchical structure in clinical data to predict cognitive performance in a large, longitudinal cohort study. In a separate study, we present a recurrent neural network that captures non-cognitive pupil dynamics by utilizing visual areas of interest as inputs. In Part II, we employ attention-based approaches for multimodal integration by learning to weigh modalities that differ in the type of information they capture. We show that our crossmodal attention framework can adapt to audiovisual and neurophysiological input data. Part III proposes a novel paradigm to study sensorimotor decision-making in a driving scenario and study brain connectivity in the context of pupil-linked arousal. Our findings reveal that embeddings that capture input data's hierarchical or temporal context consistently yield high performance across different tasks. Moreover, our studies demonstrate the versatility of the attention mechanism, which we show can effectively integrate various modalities such as text descriptions, perceived differences in video clips, and recognized objects. Our multimodal transformer, designed to handle neurophysiological data, improves the prediction of emotional states by integrating brain and autonomic activity. Taken together, our work advances the development of multimodal systems for predicting human behavior, preference, and state across domains.

Virtual worlds and massively-multiplayer online games are rich sources of information about large-scale teams and groups, offering the tantalizing possibility of harvesting data about group formation, social networks, and network evolution. They provide new outlets for human social interaction that differ from both face-to-face interactions and non-physically-embodied social networking tools such as Facebook and Twitter. We aim to study group dynamics in these virtual worlds by collecting and analyzing public conversational patterns of users grouped in close physical proximity. To do this, we created a set of tools for monitoring, partitioning, and analyzing unstructured conversations between changing groups of participants in Second Life, a massively multi-player online user-constructed environment that allows users to construct and inhabit their own 3D world. Although there are some cues in the dialog, determining social interactions from unstructured chat data alone is a difficult problem, since these environments lack many of the cues that facilitate natural language processing in other conversational settings and different types of social media. Public chat data often features players who speak simultaneously, use jargon and emoticons, and only erratically adhere to conversational norms. Humans are adept social animals capable of identifying friendship groups from a combination of linguistic cues and social network patterns. But what is more important, the content of what people say or their history of social interactions? Moreover, is it possible to identify whether iii people are part of a group with changing membership merely from general network properties, such as measures of centrality and latent communities? These are the questions that we aim to answer in this thesis. The contributions of this thesis include: 1) a link prediction algorithm for identifying friendship relationships from unstructured chat data 2) a method for identifying social groups based on the results of community detection and topic analysis. The output of these two algorithms (links and group membership) are useful for studying a variety of research questions about human behavior in virtual worlds. To demonstrate this we have performed a longitudinal analysis of human groups in different regions of the Second Life virtual world. We believe that studies performed with our tools in virtual worlds will be a useful stepping stone toward creating a rich computational model of human group dynamics.

Different types of nonprofit organizations including human service nonprofits like homeless shelters, public broadcasting organizations, and the like thrive on donations. Effective fundraising techniques are essential to a nonprofit’s existence. This research study explored a revised theory of planned behavior to include guilt and convenience in order to understand whether these factors are important in donors’ intentions to give. This study also examined the impact of two different kinds of guilt; anticipated guilt and existential guilt to determine if there was any difference between the types of guilt and the roles that they play as predicting factors in a revised TPB model. This study also explored how human service nonprofits and public broadcasting organizations compare in the factors that help better predict their donating intentions. An online survey was administered to a convenience sample, and hierarchical regression analysis was used to determine significant predicting factors within each revised TPB model. This study confirmed that the standard theory of planned behavior model was a significant predictor of intentions to donate for donors of both human service nonprofits and public broadcasting organizations. However, in both contexts, not all traditional factors of the TPB model contributed to the donation intentions. This study also provides further evidence that guilt can increase the predictive value of the standard TPB model for both types of nonprofits. Anticipated guilt more specifically, was a significant predicting factor for donors’ intentions to give to public broadcasting organizations. In contrast, convenience did not affect the explanatory power of the TPB model in either context. The TPB models for the two nonprofits are compared and theoretical and practical explanations are discussed.

A reinforcement-learning technique for enhancing human behavior models in a context-based learning architecture is presented. Prior to the introduction of this technique, human models built and developed in a Context-Based reasoning framework lacked learning capabilities. As such, their performance and quality of behavior was always limited by what the subject matter expert whose knowledge is modeled was able to articulate or demonstrate. Results from experiments performed show that subject matter experts are prone to making errors and at times they lack information on situations that are inherently necessary for the human models to behave appropriately and optimally in those situations. The benefits of the technique presented is two fold; 1) It shows how human models built in a context-based framework can be modified to correctly reflect the knowledge learnt in a simulator; and 2) It presents a way for subject matter experts to verify and validate the knowledge they share. The results obtained from this research show that behavior models built in a context-based framework can be enhanced by learning and reflecting the constraints in the environment. From the results obtained, it was shown that after the models are enhanced, the agents performed better based on the metrics evaluated. Furthermore, after learning, the agent was shown to recognize unknown situations and behave appropriately in previously unknown situations. The overall performance and quality of behavior of the agent improved significantly.

Some of the most important discoveries in cognitive neuroscience have come from recent innovations in experimental tools. Computational models that simulate human perception of environmental inputs have revealed the internal processes and features by which those inputs are learned and represented by the brain. We advance this line of work across two separate research studies in which we leveraged these models to both generate experimental task stimuli and make predictions about behavioral and neural responses to those stimuli. Chapter 1 details how nine language models were used to generate controversial sentence pairs for which two of the models disagreed about which sentence is more likely to occur. Human judgments about these sentence pairs were collected and compared to model preferences in order to identify model-specific pitfalls and provide a behavioral performance benchmark for future research. We found that transformer models GPT-2, RoBERTa and ELECTRA were most aligned with human judgments. Chapter 2 utilizes the GloVe model of semantic word vectors to generate a set of schematically structured poems comprising ten different topics whose specific temporal order was learned by a group of participants. The GloVe model was then used to investigate learning-induced changes in the spatial geometry of the representations of the topics across the cortex. A Hidden Markov Model was also used to measure neural event segmentation during poem listening. In both analyses we identified a consistent topography of learning-induced changes in the default mode network, which could be partially explained by the models.

Bipolar disorder is characterized by disruptions in mood and affect that occur not only during mood episodes, but during euthymic periods as well. At the same time, sensitivity of the behavioral activation system (BAS) has been implicated in the disorder and is a risk marker for it. Less clear is the relationship between BAS sensitivity and positive affect, particularly lower level facets of positive affect. The aim of the present study was to examine the relationship between positive affect and vulnerability for mania as assessed using BAS sensitivity. Specifically, the link between daily levels and fluctuations of positive affect and baseline BAS sensitivity was examined. Following the hierarchical model of affect, this study also assessed the relationship between BAS sensitivity and the distinct facets of positive affect. Finally, this study examined whether BAS sensitivity moderates associations between daily rewards and positive affect. Undergraduates (N = 265) from a large university in the South were recruited to complete measures of BAS sensitivity, affect, and mood symptoms at baseline. Using ecological momentary assessment (EMA), participants completed daily surveys assessing affect and engagement with rewarding situations. An exploratory factory analysis revealed a four factor structure of positive affect, consisting of Serenity, Joviality, Attentiveness, and Self-Assurance. Greater daily levels of overall positive affect, as well as the lower order facets of Joviality, Self-Assurance, and Attentiveness, were predicted by heightened BAS sensitivity. In contrast, the facet of Serenity demonstrated minimal associations with BAS sensitivity. The study findings support a multi-faceted structure of positive affect and suggest that certain facets may be more closely related to risk for bipolar disorder. Specifically, Joviality and Self-Assurance may represent maladaptive forms of positive affect, whereas Serenity may function as a protective element against bipolar disorder.

No description available.

No description available.