Orientadores: Alexandre Xavier Falcão, Anderson de Rezende Rocha / Tese (doutorado) - Universidade Estadual de Campinas, Instituto de Computação / Made available in DSpace on 2018-08-23T15:41:33Z (GMT). No. of bitstreams: 1 Chiachia_Giovani_D.pdf: 4376963 bytes, checksum: 8f7d18d591f2a5d943313d89416f96d4 (MD5) Previous issue date: 2013 / Resumo: Os seres humanos são especialistas natos em reconhecimento de faces, com habilidades que excedem em muito as dos métodos automatizados vigentes, especialmente em cenários não controlados, onde não há a necessidade de colaboração por parte do indivíduo sendo reconhecido. No entanto, uma característica marcante do reconhecimento de face humano é que nós somos substancialmente melhores no reconhecimento de faces familiares, provavelmente porque somos capazes de consolidar uma grande quantidade de experiência prévia com a aparência de certo indivíduo e de fazer uso efetivo dessa experiência para nos ajudar no reconhecimento futuro. De fato, pesquisadores em psicologia têm até mesmo sugeridos que a representação interna que fazemos das faces pode ser parcialmente adaptada ou otimizada para rostos familiares. Enquanto isso, a situação análoga no reconhecimento facial automatizado | onde um grande número de exemplos de treinamento de um indivíduo está disponível | tem sido muito pouco explorada, apesar da crescente relevância dessa abordagem na era das mídias sociais. Inspirados nessas observações, nesta tese propomos uma abordagem em que a representação da face de cada pessoa é explicitamente adaptada e realçada com o intuito de reconhecê-la melhor. Apresentamos uma coleção de métodos de aprendizado que endereça e progressivamente justifica tal abordagem. Ao aprender e operar com representações específicas para face de cada pessoa, nós somos capazes de consistentemente melhorar o poder de reconhecimento dos nossos algoritmos. Em particular, nós obtemos resultados no estado da arte na base de dados PubFig83, uma desafiadora coleção de imagens instituída e tornada pública com o objetivo de promover o estudo do reconhecimento de faces familiares. Nós sugerimos que o aprendizado de representações específicas para face de cada pessoa introduz uma forma intermediária de regularização ao problema de aprendizado, permitindo que os classificadores generalizem melhor através do uso de menos |, porém mais relevantes | características faciais / Abstract: Humans are natural face recognition experts, far outperforming current automated face recognition algorithms, especially in naturalistic, \in-the-wild" settings. However, a striking feature of human face recognition is that we are dramatically better at recognizing highly familiar faces, presumably because we can leverage large amounts of past experience with the appearance of an individual to aid future recognition. Researchers in psychology have even suggested that face representations might be partially tailored or optimized for familiar faces. Meanwhile, the analogous situation in automated face recognition, where a large number of training examples of an individual are available, has been largely underexplored, in spite of the increasing relevance of this setting in the age of social media. Inspired by these observations, we propose to explicitly learn enhanced face representations on a per-individual basis, and we present a collection of methods enabling this approach and progressively justifying our claim. By learning and operating within person-specific representations of faces, we are able to consistently improve performance on both the constrained and the unconstrained face recognition scenarios. In particular, we achieve state-of-the-art performance on the challenging PubFig83 familiar face recognition benchmark. We suggest that such person-specific representations introduce an intermediate form of regularization to the problem, allowing the classifiers to generalize better through the use of fewer | but more relevant | face features / Doutorado / Ciência da Computação / Doutor em Ciência da Computação

Algorithmic management systems organize many different kinds of work across domains, and have increasingly come under academic scrutiny. Under labels including gig work, piecemeal work, and platform labor, these systems have been richly theorized under disciplines including human-computer interaction, sociology, communications, economics, and labor law. When it comes to the relationships between such systems and their workers, current theory frames these interactions on a continuum between organizational control and worker autonomy. This has laid the groundwork for other ways of examining micro-level practices of workers under algorithmic management. As an alternative to the binary of control and autonomy, this dissertation takes its cue from feminist scholars in Science, Technology, and Society (STS) studies. Drawing on frameworks from articulation, repair, and mutual shaping, I examine workers’ interpretations and interactions, to ask how new subjectivities around identity and community emerge from these entanglements. To shed empirical light on these processes, this dissertation employs a mixed-methods research design examining the introduction of facial recognition into the sociotechnical systems of algorithmic management. Data include 22 in-person interviews with workers in New York City and Toronto, a survey of 100 workers in the United States who have been subjected to facial recognition, and analysis of over 2800 comments gathered from an online workers’ forum posted over the course of four years.Facial recognition, like algorithmic management, suffers from a lack of empirical, on-the-ground insights into how workers communicate, negotiate, and strategize around and through them. Interviews with workers reveals that facial recognition evokes polysemia, i.e. a number of distinct, yet interrelated interpretations. I find that for some workers, facial recognition means safety and security. To others it means violation of privacy and accusations of fraud. Some are impressed by the “science-fiction”-like capabilities of the system: “it’s like living in the future.” Others are wary, and science fiction becomes a vehicle to encapsulate their fears: “I’m in the [movie] The Minority Report.” For some the technology is hyper-powerful: “It feels like I’m always being watched,” yet others decry, “it’s an obvious façade.” Following interviews, I build a body of research using empirical methods combined with frameworks drawn from STS and organizational theory to illuminate workers’ perceptions and strategies negotiating their algorithmic managers. I operationalize Julian Orr’s studies of storytelling among Xerox technicians to analyze workers’ information-sharing practices in online forums, to better understand how gig workers, devices, forums, and algorithmic management systems engage in mutual shaping processes. Analysis reveals that opposing interpretations of facial recognition, rather than dissolving into consensus of “shared understanding,” continue to persist. Rather than pursuing and relying on shared understanding of their work to maintain relationships, workers under algorithmic management, communicating in online forums about facial recognition, elide consensus. After forum analysis, I then conduct a survey, to assess workers’ fairness perceptions of facial recognition targeting and verification. The goal of this research is to establish an empirical foundation to determine whether algorithmic fairness perceptions are subject to theories of bounded rationality and decision-making. Finally, for the last two articles, I turn back to the forums, to analyze workers’ experiences negotiating two other processes with threats or ramifications for safety, privacy, and risk. In one article, I focus on their negotiation of threats from scam attackers, and the use the forum itself as a “shared repertoire” of knowledge. In the other I use the forums as evidence to illuminate workers’ experiences and meaning-making around algorithmic risk management under COVID-19. In the conclusion, I engage in theory-building to examine how algorithmic management and its attendant processes demand that information-sharing mechanisms serve novel ends buttressing legitimacy and authenticity, in what I call “para-organizational” work, a world of work where membership and legitimacy are liminal and uncertain. Ultimately, this body of research illuminates mutual shaping processes in which workers’ practices, identity, and community are entangled with technological artifacts and organizational structures. Algorithmic systems of work and participants’ interpretations of, and interactions with, related structures and devices, may be creating a world where sharing information is a process wielded not as a mechanism of learning, but as one of belonging.

Across three chapters of this doctoral dissertation, I explore how machine learning algorithms can be used to efficiently acquire personal information and responsibly use it in decision making, in marketing and beyond. In the first chapter, I show that machine learning on consumer facial images can reveal a variety of personal information. I provide evidence that such information can be profitably used by marketers. I also investigate the mechanism behind how facial images reveal personal information. In the second chapter, I propose a new self-supervised deep reinforcement learning approach to question prioritization and questionnaire shortening and show it is competitive against benchmark methods. I use the proposed method to show that typical consumer data sets can be reconstructed well based on relatively small select subsets of their columns. The reconstruction quality grows logarithmically in the relative size of the column subset, implying diminishing returns on measurement. Thus, many long questionnaires could be shortened with minimal information loss, increasing the consumer research efficiency and enabling previously impossible multi-scale omnibus studies. In the third chapter, I present a method to speed up ranking under constraints for live ethical content recommendations by predicting, rather than finding exactly, the solution to the underlying time-intensive optimization problem. The approach enables solving larger-than-previously-reported constrained content-ranking problems in real time, within 50 milliseconds, as required to avoid the perception of latency by the users. The approach could also help speed up general assignment and matching tasks.

Reconhecimento de padrões em imagens é uma área de grande interesse no mundo científico. Os chamados métodos de extração de características, possuem as habilidades de extrair características das imagens e também de reduzir a dimensionalidade dos dados gerando assim o chamado vetor de características. Considerando uma imagem de consulta, o foco de um sistema de reconhecimento de imagens de faces humanas é pesquisar em um banco de imagens, a imagem mais similar à imagem de consulta, de acordo com um critério dado. Este trabalho de pesquisa foi direcionado para a geração de vetores de características para um sistema de reconhecimento de imagens, considerando bancos de imagens de faces humanas, para propiciar tal tipo de consulta. Um vetor de características é uma representação numérica de uma imagem ou parte dela, descrevendo seus detalhes mais representativos. O vetor de características é um vetor n-dimensional contendo esses valores. Essa nova representação da imagem propicia vantagens ao processo de reconhecimento de imagens, pela redução da dimensionalidade dos dados. Uma abordagem alternativa para caracterizar imagens para um sistema de reconhecimento de imagens de faces humanas é a transformação do domínio. A principal vantagem de uma transformação é a sua efetiva caracterização das propriedades locais da imagem. As wavelets diferenciam-se das tradicionais técnicas de Fourier pela forma de localizar a informação no plano tempo-freqüência; basicamente, têm a capacidade de mudar de uma resolução para outra, o que as fazem especialmente adequadas para análise, representando o sinal em diferentes bandas de freqüências, cada uma com resoluções distintas correspondentes a cada escala. As wavelets foram aplicadas com sucesso na compressão, melhoria, análise, classificação, caracterização e recuperação de imagens. Uma das áreas beneficiadas onde essas propriedades tem encontrado grande relevância é a área de visão computacional, através da representação e descrição de imagens. Este trabalho descreve uma abordagem para o reconhecimento de imagens de faces humanas com a extração de características baseado na decomposição multiresolução de wavelets utilizando os filtros de Haar, Daubechies, Biorthogonal, Reverse Biorthogonal, Symlet, e Coiflet. Foram testadas em conjunto as técnicas PCA (Principal Component Analysis) e IMPCA (Image Principal Component Analysis), sendo que os melhores resultados foram obtidos utilizando a wavelet Biorthogonal com a técnica IMPCA / Image pattern recognition is an interesting area in the scientific world. The features extraction method refers to the ability to extract features from images, reduce the dimensionality and generates the features vector. Given a query image, the goal of a features extraction system is to search the database and return the most similar to the query image according to a given criteria. Our research addresses the generation of features vectors of a recognition image system for human faces databases. A feature vector is a numeric representation of an image or part of it over its representative aspects. The feature vector is a n-dimensional vector organizing such values. This new image representation can be stored into a database and allow a fast image retrieval. An alternative for image characterization for a human face recognition system is the domain transform. The principal advantage of a transform is its effective characterization for their local image properties. In the past few years researches in applied mathematics and signal processing have developed practical wavelet methods for the multi scale representation and analysis of signals. These new tools differ from the traditional Fourier techniques by the way in which they localize the information in the time-frequency plane; in particular, they are capable of trading on type of resolution for the other, which makes them especially suitable for the analysis of non-stationary signals. The wavelet transform is a set basis function that represents signals in different frequency bands, each one with a resolution matching its scale. They have been successfully applied to image compression, enhancement, analysis, classification, characterization and retrieval. One privileged area of application where these properties have been found to be relevant is computer vision, especially human faces imaging. In this work we describe an approach to image recognition for human face databases focused on feature extraction based on multiresolution wavelets decomposition, taking advantage of Biorthogonal, Reverse Biorthogonal, Symlet, Coiflet, Daubechies and Haar. They were tried in joint the techniques together the PCA (Principal Component Analysis) and IMPCA (Image Principal Component Analysis)

This thesis represents Bayesian joint audio-visual tracking for the 3D locations of multiple people and a current speaker in a real conference environment. To achieve this objective, it focuses on several different research interests, such as acoustic-feature detection, visual-feature detection, a non-linear Bayesian framework, data association, and sensor fusion. As acoustic-feature detection, time-delay-of-arrival~(TDOA) estimation is used for multiple source detection. Localization performance using TDOAs is also analyzed according to different configurations of microphones. As a visual-feature detection, Viola-Jones face detection is used to initialize the locations of unknown multiple objects. Then, a corner feature, based on the results from the Viola-Jones face detection, is used for motion detection for robust objects. Simple point-to-line correspondences between multiple cameras using fundamental matrices are used to determine which features are more robust. As a method for data association and sensor fusion, Monte-Carlo JPDAF and a data association with IPPF~(DA-IPPF) are implemented in the framework of particle filtering. Three different tracking scenarios of acoustic source tracking, visual source tracking, and joint acoustic-visual source tracking are represented using the proposed algorithms. Finally the real-time implementation of this joint acoustic-visual tracking system using a PC, four cameras, and six microphones is addressed with two parts of system implementation and real-time processing.

Reconhecimento de padrões em imagens é uma área de grande interesse no mundo científico. Os chamados métodos de extração de características, possuem as habilidades de extrair características das imagens e também de reduzir a dimensionalidade dos dados gerando assim o chamado vetor de características. Considerando uma imagem de consulta, o foco de um sistema de reconhecimento de imagens de faces humanas é pesquisar em um banco de imagens, a imagem mais similar à imagem de consulta, de acordo com um critério dado. Este trabalho de pesquisa foi direcionado para a geração de vetores de características para um sistema de reconhecimento de imagens, considerando bancos de imagens de faces humanas, para propiciar tal tipo de consulta. Um vetor de características é uma representação numérica de uma imagem ou parte dela, descrevendo seus detalhes mais representativos. O vetor de características é um vetor n-dimensional contendo esses valores. Essa nova representação da imagem propicia vantagens ao processo de reconhecimento de imagens, pela redução da dimensionalidade dos dados. Uma abordagem alternativa para caracterizar imagens para um sistema de reconhecimento de imagens de faces humanas é a transformação do domínio. A principal vantagem de uma transformação é a sua efetiva caracterização das propriedades locais da imagem. As wavelets diferenciam-se das tradicionais técnicas de Fourier pela forma de localizar a informação no plano tempo-freqüência; basicamente, têm a capacidade de mudar de uma resolução para outra, o que as fazem especialmente adequadas para análise, representando o sinal em diferentes bandas de freqüências, cada uma com resoluções distintas correspondentes a cada escala. As wavelets foram aplicadas com sucesso na compressão, melhoria, análise, classificação, caracterização e recuperação de imagens. Uma das áreas beneficiadas onde essas propriedades tem encontrado grande relevância é a área de visão computacional, através da representação e descrição de imagens. Este trabalho descreve uma abordagem para o reconhecimento de imagens de faces humanas com a extração de características baseado na decomposição multiresolução de wavelets utilizando os filtros de Haar, Daubechies, Biorthogonal, Reverse Biorthogonal, Symlet, e Coiflet. Foram testadas em conjunto as técnicas PCA (Principal Component Analysis) e IMPCA (Image Principal Component Analysis), sendo que os melhores resultados foram obtidos utilizando a wavelet Biorthogonal com a técnica IMPCA / Image pattern recognition is an interesting area in the scientific world. The features extraction method refers to the ability to extract features from images, reduce the dimensionality and generates the features vector. Given a query image, the goal of a features extraction system is to search the database and return the most similar to the query image according to a given criteria. Our research addresses the generation of features vectors of a recognition image system for human faces databases. A feature vector is a numeric representation of an image or part of it over its representative aspects. The feature vector is a n-dimensional vector organizing such values. This new image representation can be stored into a database and allow a fast image retrieval. An alternative for image characterization for a human face recognition system is the domain transform. The principal advantage of a transform is its effective characterization for their local image properties. In the past few years researches in applied mathematics and signal processing have developed practical wavelet methods for the multi scale representation and analysis of signals. These new tools differ from the traditional Fourier techniques by the way in which they localize the information in the time-frequency plane; in particular, they are capable of trading on type of resolution for the other, which makes them especially suitable for the analysis of non-stationary signals. The wavelet transform is a set basis function that represents signals in different frequency bands, each one with a resolution matching its scale. They have been successfully applied to image compression, enhancement, analysis, classification, characterization and retrieval. One privileged area of application where these properties have been found to be relevant is computer vision, especially human faces imaging. In this work we describe an approach to image recognition for human face databases focused on feature extraction based on multiresolution wavelets decomposition, taking advantage of Biorthogonal, Reverse Biorthogonal, Symlet, Coiflet, Daubechies and Haar. They were tried in joint the techniques together the PCA (Principal Component Analysis) and IMPCA (Image Principal Component Analysis)

This thesis considers the problematic of the object recognition in a digital picture, particularly about the human face recognition and its components. There are described the basics of the computer vision, the object detector Viola-Jones, its computer realization with help of the OpenCV libraries and the test results. This thesis also describes the accurate system of the facial features detection per the algorithm of the Active Shape Models and also related mechanism of the classifier training, including the software implementation.

As the amount of computer mediated information (e.g., emails, documents, multi-media) we need to process grows, our need to rapidly sort, organize and store electronic information likewise increases. In order to store information effectively, we must find ways to sort through it and organize it in a manner that facilitates efficient retrieval. The instantaneous and emergent nature of communications across networks like Twitter makes them suitable for discussing events (e.g., natural disasters) that are amorphous and prone to rapid changes. It can be difficult for an individual human to filter through and organize the large amounts of information that can pass through these types of social networks when events are unfolding rapidly. A common feature of social networks is the images (e.g., human faces, inanimate objects) that are often used by those who send messages across these networks. Humans have a particularly strong ability to recognize and differentiate between human Faces. This effect may also extend to recalling information associated with each human Face. This study investigated the difference between human Face images, non-human Face images and alphanumeric labels as retrieval cues under different levels of Task Load. Participants were required to recall key pieces of event information as they emerged from a Twitter-style message feed during a simulated natural disaster. A counter-balanced within-subjects design was used for this experiment. Participants were exposed to low, medium and high Task Load while responding to five different types of recall cues: (1) Nickname, (2) Non-Face, (3) Non-Face & Nickname, (4) Face and (5) Face & Nickname. The task required participants to organize information regarding emergencies (e.g., car accidents) from a Twitter-style message feed. The messages reported various events such as fires occurring around a fictional city. Each message was associated with a different recall cue type, depending on the experimental condition. Following the task, participants were asked to recall the information associated with one of the cues they worked with during the task. Results indicate that under medium and high Task Load, both Non-Face and Face retrieval cues increased recall performance over Nickname alone with Non-Faces resulting in the highest mean recall scores. When comparing medium to high Task Load: Face & Nickname and Non-Face significantly outperformed the Face condition. The performance in Non-Face & Nickname was significantly better than Face & Nickname. No significant difference was found between Non-Faces and Non-Faces & Nickname. Subjective Task Load scores indicate that participants experienced lower mental workload when using Non-Face cues than using Nickname or Face cues. Generally, these results indicate that under medium and high Task Load levels, images outperformed alphanumeric nicknames, Non-Face images outperformed Face images, and combining alphanumeric nicknames with images may have offered a significant performance advantage only when the image is that of a Face. Both theoretical and practical design implications are provided from these findings.