<p>Virtual assistants such as Amazon's Alexa can emulate a variety of emotions in their spoken feedback to user requests. The persuasive impact of these emotional displays depends on the inferences individuals make about these emotional displays. This dissertation investigates a class of inferences –attributions of cause to a virtual assistant's emotional display –which individuals use as social information about computer-interactants. The project hypothesized that three key factors influence individuals' attributions of cause to virtual assistants' emotional displays. The first is general tendencies in peoples' attributions of virtual assistants' emotional displays. Second is the target of a virtual assistant's emotional display –the communicated object of their displayed emotional state. The final component is borrowed from Kelley's covariance theory of attribution –the distinctiveness with which a specific emotional display repeatedly co-occurs with plausible explanations for the emotional display. Four attributional outcomes were predicted based on combinations of emotional display targets and the distinctiveness of the emotional display's covariance with potential causes for the emotional display. Findings suggest that individuals first and foremost attribute virtual assistant's emotional displays to the virtual assistant rather than situational causes, that emotional display targets can influence attributions, and that further work is needed to assess the role of distinctiveness in attributions of virtual assistant's emotional displays.</p>

Past research has demonstrated that emotions can accurately be communicated via touch (e.g., Hertenstein, Keltner, App, Bulleit, and Jaskolka, 2006). In stranger female dyads, physiological linkage plays a role in the mechanism whereby this successful communication occurs, as touch strengthens and lengthens linkage (Kissel, 2020). While touch has a direct impact on physiological processes, viewing touch may have similar effects. The current study explored this possibility in regard to physiological linkage. Hertenstein et al., 2006 demonstrated that participants can correctly decode emotions from observing videos of communication via touch to the forearm and hand. The current study replicated this finding with forty-seven female participants, while also determining the levels of physiological linkage between the "live" observers and the video-recorded participants from Kissel (2020) using dynamic linear time series modeling. Results showed that physiological linkage can occur between "live" and recorded participants. Participants demonstrated longer linkage times with the initial dyad they viewed, but linkage with videoed communicators whose communications were correctly perceived by their fellow videoed receiver had a larger influence on emotion word, valence, intensity, and quadrant detection accuracy. Based on these results, physiological linkage may influence empathic accuracy in virtual settings. / Doctor of Philosophy / A common American English slang expression to state that you relate to someone on a deep personal level is "I feel you." This is a verbal expression of empathy, but what if empathy goes deeper than our thoughts or memories of similar experiences? What if our bodies experience the same emotion as the person with whom we are interacting? This is possible through the phenomenon of "physiological linkage." Physiological linkage occurs when physiological signals, such as heart rate, between interaction partners start to sync up—for example, when one person's heart rate speeds up, so does the heart rate of the person with whom they are interacting. The author's thesis study demonstrated that this linkage can occur when people communicate emotions solely through touch. But what happens if you are watching these emotion communications instead of experiencing them? The current study examined if physiological linkage occurs between people watching a video and the people emoting in the video. The results showed that linkage does occur while watching emotional touch interactions and that this can help the observer understand what these emotions are (even if the observer can see no faces and hear no voices). Touch has many health benefits, so the observation that watching recorded touch interactions can have a similar bodily effect has implications for increasing health and connectedness. This is particularly important given the limited face-to-face and touch interactions, as well as the increase in video call interactions, resulting from the COVID-19 pandemic.

Previous research has demonstrated that communication of emotion via touch is possible and occurs well-above chance levels, though the potential mechanism whereby this occurs has yet to be determined. The current study aimed to determine if physiological linkage, or the synchrony between various physiological signals between two interaction partners, played a role in successful communication of emotion via touch. Dynamic linear times series analysis was used to determine the strength and length of synchrony between the inter-beat intervals of fifty-two stranger female-female dyads (n=104, mean age=19.88) during two rounds of an emotion communication task in which they communicated a randomized list emotions to each other via forearm touch alone without being able to see their interaction partner. Results showed the highest magnitude linkage coefficients and the greatest number of consecutive lagged linked seconds during the “touch alone” communication—demonstrating that touch increases physiological linkage. Stronger and longer physiological linkage across tasks predicted emotion word, valence, intensity, and quadrant (from the circumplex model) detection accuracy. Participants serving as the initial communicator in the first round of emotion communication tended to have a greater influence on the physiology of initial receivers. Overall, greater physiological linkage as the result of touch predicted successful communication of emotion via touch and is therefore likely a portion of the mechanism underlying this phenomenon. / M.S. / People often communicate with their friends, family, and acquaintances using touch—when meeting a loved one after a long time, we might give them a particularly tight hug; to congratulate someone, we give a high five; and even in business settings, handshakes are used as a form of greeting or parting. Touch can also be used to communicate distinct emotions, just like a frown or a stern tone can communicate visually and aurally that someone is angry. However, although past research has demonstrated this communicative ability of touch, it is not yet known how touch is able to communicate emotion. The current study hypothesized that physiological linkage might play a role. Physiological linkage occurs when physiological signals, such as heart rate, between interaction partners starts to sync up—for example, when one person’s heart rate speeds up, so does the heart rate of the person with whom they are interacting. Results showed that greater levels of physiological linkage occurred in response to touch and that these increased levels of physiological linkage predicted people’s ability to successfully determine which emotion was communicated to them via touch to their forearm. All the emotions were communicated via touch alone; participants could not see or hear their interaction partner. This demonstrates how powerful communication via touch can be. Future research should examine how touch and physiological linkage can be incorporated into medical and psychological therapies.

Diese kumulative Dissertation umfasst zwei Veröffentlichungen zu drei Bereichen der Emotionskommunikation. Ziel war es, zu untersuchen, ob OP-Masken die Emotionswahrnehmung, die Affiliation und die emotionale Mimikry (d.h. die automatische, aber zielabhängige Imitation des emotionalen Ausdrucks von Interaktionspartner:innen) reduzieren. In zwei Online-Experimenten (Studie 1: N=200, britische Stichprobe; Studie 2: N=235, deutsche Stichprobe) wurden subjektive Bewertungen und die emotionale Mimikry als Reaktion auf maskierte und unmaskierte Gesichter untersucht. Die wahrgenommene Emotionsintensität und die Genauigkeit der Emotionserkennung dienten als Indikatoren für die Emotionswahrnehmung. Die wahrgenommene zwischenmenschliche Nähe diente als Indikator für die Affiliation. Die emotionale Mimikry wurde mit Hilfe einer Gesichtsaktivitätserkennungstechnologie gemessen. In der ersten Studie sahen erwachsene Proband:innen erwachsene Zielpersonen, die Freude und Trauer ausdrückten, eingebettet in Innen- und Außenszenen. In der zweiten Studie sahen erwachsene Proband:innen Erwachsene und Kinder, die Freude, Trauer oder Ärger ausdrückten. Die Freudemimikry wurde durch Masken reduziert, insbesondere wenn die Zielpersonen Kinder waren. Im Gegensatz dazu war die Trauermimikry bei Kindergesichtern stärker und wurde, wie auch die Ärgermimikry, durch Masken nicht beeinträchtigt. Wir konnten auch zeigen, dass durch Gesichtsmasken verringerte Freudewahrnehmung und Nähe mit einer verringerten Freudemimikry verbunden waren. Die Studien zeigen somit, wie erwartet wurde, eine maskenbedingte Verringerung der Emotionswahrnehmung, der Affiliation, und der emotionalen Mimikry. Insgesamt trägt die vorliegende Arbeit zu unserem Verständnis der sozio-affektiven Auswirkungen der partiellen Gesichtsverdeckung bei und stützt die Theorie der emotionalen Mimikry im sozialen Kontext von Hess und Fischer, nach der Emotionswahrnehmung und Affiliation die emotionale Mimikry beeinflussen. / This cumulative doctoral dissertation encompasses two publications on three domains of emotion communication. The goal of the dissertation was to assess if surgical face masks reduce emotion perception, affiliation, and emotional mimicry (i.e., automatic but goal-dependent imitation of an interaction partner's emotional display). We conducted two online experiments (Study 1: N=200, U.K. sample; Study 2: N=235, German sample) that assessed subjective ratings and emotional mimicry in response to masked and unmasked faces. Perceived emotion intensity and emotion recognition accuracy served as indicators of emotion perception. Perceived interpersonal closeness (via the Inclusion of Other in the Self Scale) served as an indicator of affiliation. Emotional mimicry was measured using facial activity recognition technology. We also took into account socio-spatial context effects. In the first study, adult perceivers saw adult targets who expressed happiness and sadness embedded into indoor and outdoor scenes. In the second study, adult perceivers saw adult and child targets who expressed happiness, sadness, and anger. We found that happiness mimicry was reduced by masks, particularly when expressers were children. In contrast, sadness mimicry was stronger for children and, like anger mimicry, unaffected by masks. We also found that reduced emotion perception and closeness due to masks were associated with reduced happiness mimicry. The studies support evidence from before and during the COVID-19 pandemic for mask-related reductions of emotion perception and affiliation. Moreover, the studies provide unprecedented evidence on reductions of emotional mimicry in response to masked faces and child targets. Overall, the present work contributes to our understanding of the socio-affective effects of partial face occlusion and supports emotional mimicry in social context theory by Hess and Fischer, according to which emotion perception and affiliation influence emotional mimicry.