We constantly perceive and categorize internal signals, like our subjective affective state, and complex social signals, like the faces of the people around us. In this dissertation, I aim to characterize some of the ways in which we perceive and categorize affective and social stimuli, top-down influences on those processes, and individual differences in social & affective perception/categorization. First, in Chapter 2, I apply psychophysical methods to assess how individual differences in trait emotional expressivity arise from observers' subjective emotion reporting thresholds. Next, in Chapter 3, I characterize the perception and categorization of age from adult faces. Finally, in Chapter 4, I investigate whether the act of categorizing one's subjective emotional state changes the affective distance between neural representations of positive and negative affect states.

A long standing debate in psychological science is whether the face signals specific emotions. Basic emotion theory presupposes that there are coordinated facial musculature movements that individuals can identify as relating to a core set of basic emotions. In opposition to this view, the constructionist theory contends that the perception of emotion is a far more intricate process involving semantic knowledge and arousal states. The aim of the current investigation was to explore some of the questions at the crux of this debate. We showed participants video clips of real people in real time, where the face was in motion, much as in everyday life. In study 1 we directly manipulated the effects of context to determine what influences emotion perception – situational information or the face? In support of the basic emotion view, participants identified displays of happiness, anger and sadness irrespective of contextual information provided. Importantly, participants also rated one set of facial movements as more intensely expressing a ‘sad’ face. Study 1 also demonstrated unique context effects in partial support for the constructionist view, suggesting that for some facial expressions, the role of context may be important. In study 2, we explored the possible effects that language has on the perception of emotion. In the absence of linguistic cues, participants used significantly more ‘happy’ and ‘sad’ words to label the basic emotion prototype for happiness and for the ‘sad’ face introduced in study 1. Overall, findings from these studies suggest that although contextual cues may be important for specific scenarios, the face is dominant to the layperson when inferring the emotional state of another.

Across three chapters of this doctoral dissertation, I explore how machine learning algorithms can be used to efficiently acquire personal information and responsibly use it in decision making, in marketing and beyond. In the first chapter, I show that machine learning on consumer facial images can reveal a variety of personal information. I provide evidence that such information can be profitably used by marketers. I also investigate the mechanism behind how facial images reveal personal information. In the second chapter, I propose a new self-supervised deep reinforcement learning approach to question prioritization and questionnaire shortening and show it is competitive against benchmark methods. I use the proposed method to show that typical consumer data sets can be reconstructed well based on relatively small select subsets of their columns. The reconstruction quality grows logarithmically in the relative size of the column subset, implying diminishing returns on measurement. Thus, many long questionnaires could be shortened with minimal information loss, increasing the consumer research efficiency and enabling previously impossible multi-scale omnibus studies. In the third chapter, I present a method to speed up ranking under constraints for live ethical content recommendations by predicting, rather than finding exactly, the solution to the underlying time-intensive optimization problem. The approach enables solving larger-than-previously-reported constrained content-ranking problems in real time, within 50 milliseconds, as required to avoid the perception of latency by the users. The approach could also help speed up general assignment and matching tasks.