Large displays and information kiosks are becoming increasingly common installations in public venues to provide an efficient self-serve means for patrons to access information and/or services. They have evolved over a relatively short period of time from non-digital, non-interactive static displays to more elaborate media-rich digital interactive systems. While the content and purposes of kiosks have changed, they are still largely based on the traditional single-user-driven design paradigm despite the fact that people often venture to these venues in small social groups, i.e., with family and/or friends. This often limits how groups collaborate and forces transactions to be serialized. This thesis explores design constraints for interaction by multiple social groups in parallel on shared large vertical displays. To better understand design requirements for these systems, this research is separated into two parts: a preliminary observational field study and a follow-up controlled study. Using an observational field study, fundamental patterns of how people use existing public displays are studied: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. These results are then used to motivate a controlled experiment where two individuals or two pairs of individuals complete tasks concurrently on a low-fidelity large vertical display. Results from the studies demonstrate that vertical surface territories are similar to those found in horizontal tabletops in function, but their definitions and social conventions are different. In addition, the nature of use-while-standing systems results in more complex and dynamic physical territories around the display. We show that the anthropological notion of personal space must be slightly refined for application to vertical displays.

Large displays and information kiosks are becoming increasingly common installations in public venues to provide an efficient self-serve means for patrons to access information and/or services. They have evolved over a relatively short period of time from non-digital, non-interactive static displays to more elaborate media-rich digital interactive systems. While the content and purposes of kiosks have changed, they are still largely based on the traditional single-user-driven design paradigm despite the fact that people often venture to these venues in small social groups, i.e., with family and/or friends. This often limits how groups collaborate and forces transactions to be serialized. This thesis explores design constraints for interaction by multiple social groups in parallel on shared large vertical displays. To better understand design requirements for these systems, this research is separated into two parts: a preliminary observational field study and a follow-up controlled study. Using an observational field study, fundamental patterns of how people use existing public displays are studied: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. These results are then used to motivate a controlled experiment where two individuals or two pairs of individuals complete tasks concurrently on a low-fidelity large vertical display. Results from the studies demonstrate that vertical surface territories are similar to those found in horizontal tabletops in function, but their definitions and social conventions are different. In addition, the nature of use-while-standing systems results in more complex and dynamic physical territories around the display. We show that the anthropological notion of personal space must be slightly refined for application to vertical displays.

Com o constante crescimento tecnológico, é bastante comum deparar-se com um display público em lugares de grande concentração de pessoas, como aeroportos e cinemas. Apesar de possuírem informações úteis, esses displays poderiam ser melhor aproveitados se fossem interativos. Baseando-se em pesquisas sobre a interação com displays grandes e as características próprias de um display colocado em um espaço público, busca-se uma maneira de interação que seja adequada a esse tipo de situação. O presente trabalho introduz um método de interação por gestos sem necessitar que o usuário interagente segure ou tenha nele acoplado qualquer dispositivo ao interagir com um display público. Para realizar as tarefas que deseja, o usuário só precisa posicionar-se frente ao display e interagir com as informações na tela com suas mãos. São suportados gestos para navegação, seleção e manipulação de objetos, bem como para transladar a tela de visualização e ampliá-la ou diminui-la. O sistema proposto é construído de forma que possa funcionar em aplicações diferentes sem um grande custo de implantação. Para isso, é utilizado um sistema do tipo cliente-servidor que integra a aplicação que contém as informações de interesse do usuário e a que interpreta os seus gestos. É utilizado o Microsoft Kinect para a leitura dos movimentos do usuário e um pós-processamento de imagens é realizado de modo a detectar se as mãos do usuário se encontram abertas os fechadas. Após, essa informação é interpretada por uma máquina de estados que identifica o que o usuário está querendo executar na aplicação cliente. Afim de avaliar o quão robusto o sistema se portaria em um ambiente público real, são avaliados critérios que poderiam interferir na tarefa interativa, como a diferença de luminosidade do ambiente e a presença de mais pessoas no mesmo local de interação. Foram desenvolvidas três aplicações a título de estudo de caso e cada uma delas foi avaliada de forma diferente, sendo uma delas utiliza para fins de avaliação formal com usuários. Demonstrados os resultados da avaliação realizada, conclui-se que o sistema, apesar de não se portar corretamente em todas situações, tem potencial de uso desde que sejam contornadas suas deficiências, a maior parte das quais originária das próprias limitações inerentes ao Kinect. O sistema proposto funciona suficientemente bem para seleção e manipulação de objetos grandes e para aplicações baseadas em interação do tipo pan & zoom, como navegação em mapas, por exemplo, e não é influenciado por diferenças de iluminação ou presença de outras pessoas no ambiente. / With the constant technological growth, it is quite common to come across a public display in places with high concentration of people, such as airports and theaters. Although they provide useful information, these displays could be better employed if they were interactive. Based on research on topics of interaction with large displays and the characteristics of a display placed in a public space, a way of interaction that is suitable for this kind of situation is searched. This paper introduces a method of interaction by gestures without requiring that the interacting user take hold or have to him attached any device to interact with a public display. To accomplish the tasks he wants, he needs just to position himself in front of the display and to interact with the information on the screen with his hands. Gestures supported provide navigation, selection and manipulation of objects as well as to pan and zoom at the screen. The proposed system is constructed so that it works in different applications without a large installation cost. In order to achieve this, the system implements a client-server model application that is able to integrate the part that contains the useful information to the user and the one that interprets his gestures. The Microsoft Kinect is used for reading the user’s movements and techniques of image processing are performed to detect if the user’s hands are open or closed. After this information is obtained, it runs through a state machine that identifies what the user is trying to do in the application. In order to evaluate how robust the system is in a real public environment, some criteria that could interfere with the interactive task are evaluated, as the difference in brightness in the environment and the presence of another people in the same place of interaction. Three applications were developed as a case study and each one was evaluated differently, one of them being used for formal user evaluation. Given the results of the performed tasks, it is possible to conclude that the system, although not behaving correctly in all situations, has potential use if its difficulties are circumvented, most of which come from Kinect’s own inherent limitations. The proposed system works well enough for selection and manipulation of large objects and for use in applications based on pan & zoom, like those that supports map navigation, for example, and difference of ilumination or the presence of other persons on the environment does not interfere with the interaction process.

abstract: Advances in technology are fueling a movement toward ubiquity for beyond-the-desktop systems. Novel interaction modalities, such as free space or full body gestures are becoming more common, as demonstrated by the rise of systems such as the Microsoft Kinect. However, much of the interaction design research for such systems is still focused on desktop and touch interactions. Current thinking in free-space gestures are limited in capability and imagination and most gesture studies have not attempted to identify gestures appropriate for public walk-up-and-use applications. A walk-up-and-use display must be discoverable, such that first-time users can use the system without any training, flexible, and not fatiguing, especially in the case of longer-term interactions. One mechanism for defining gesture sets for walk-up-and-use interactions is a participatory design method called gesture elicitation. This method has been used to identify several user-generated gesture sets and shown that user-generated sets are preferred by users over those defined by system designers. However, for these studies to be successfully implemented in walk-up-and-use applications, there is a need to understand which components of these gestures are semantically meaningful (i.e. do users distinguish been using their left and right hand, or are those semantically the same thing?). Thus, defining a standardized gesture vocabulary for coding, characterizing, and evaluating gestures is critical. This dissertation presents three gesture elicitation studies for walk-up-and-use displays that employ a novel gesture elicitation methodology, alongside a novel coding scheme for gesture elicitation data that focuses on features most important to users’ mental models. Generalizable design principles, based on the three studies, are then derived and presented (e.g. changes in speed are meaningful for scroll actions in walk up and use displays but not for paging or selection). The major contributions of this work are: (1) an elicitation methodology that aids users in overcoming biases from existing interaction modalities; (2) a better understanding of the gestural features that matter, e.g. that capture the intent of the gestures; and (3) generalizable design principles for walk-up-and-use public displays. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2019

No description available.

Com o constante crescimento tecnológico, é bastante comum deparar-se com um display público em lugares de grande concentração de pessoas, como aeroportos e cinemas. Apesar de possuírem informações úteis, esses displays poderiam ser melhor aproveitados se fossem interativos. Baseando-se em pesquisas sobre a interação com displays grandes e as características próprias de um display colocado em um espaço público, busca-se uma maneira de interação que seja adequada a esse tipo de situação. O presente trabalho introduz um método de interação por gestos sem necessitar que o usuário interagente segure ou tenha nele acoplado qualquer dispositivo ao interagir com um display público. Para realizar as tarefas que deseja, o usuário só precisa posicionar-se frente ao display e interagir com as informações na tela com suas mãos. São suportados gestos para navegação, seleção e manipulação de objetos, bem como para transladar a tela de visualização e ampliá-la ou diminui-la. O sistema proposto é construído de forma que possa funcionar em aplicações diferentes sem um grande custo de implantação. Para isso, é utilizado um sistema do tipo cliente-servidor que integra a aplicação que contém as informações de interesse do usuário e a que interpreta os seus gestos. É utilizado o Microsoft Kinect para a leitura dos movimentos do usuário e um pós-processamento de imagens é realizado de modo a detectar se as mãos do usuário se encontram abertas os fechadas. Após, essa informação é interpretada por uma máquina de estados que identifica o que o usuário está querendo executar na aplicação cliente. Afim de avaliar o quão robusto o sistema se portaria em um ambiente público real, são avaliados critérios que poderiam interferir na tarefa interativa, como a diferença de luminosidade do ambiente e a presença de mais pessoas no mesmo local de interação. Foram desenvolvidas três aplicações a título de estudo de caso e cada uma delas foi avaliada de forma diferente, sendo uma delas utiliza para fins de avaliação formal com usuários. Demonstrados os resultados da avaliação realizada, conclui-se que o sistema, apesar de não se portar corretamente em todas situações, tem potencial de uso desde que sejam contornadas suas deficiências, a maior parte das quais originária das próprias limitações inerentes ao Kinect. O sistema proposto funciona suficientemente bem para seleção e manipulação de objetos grandes e para aplicações baseadas em interação do tipo pan & zoom, como navegação em mapas, por exemplo, e não é influenciado por diferenças de iluminação ou presença de outras pessoas no ambiente. / With the constant technological growth, it is quite common to come across a public display in places with high concentration of people, such as airports and theaters. Although they provide useful information, these displays could be better employed if they were interactive. Based on research on topics of interaction with large displays and the characteristics of a display placed in a public space, a way of interaction that is suitable for this kind of situation is searched. This paper introduces a method of interaction by gestures without requiring that the interacting user take hold or have to him attached any device to interact with a public display. To accomplish the tasks he wants, he needs just to position himself in front of the display and to interact with the information on the screen with his hands. Gestures supported provide navigation, selection and manipulation of objects as well as to pan and zoom at the screen. The proposed system is constructed so that it works in different applications without a large installation cost. In order to achieve this, the system implements a client-server model application that is able to integrate the part that contains the useful information to the user and the one that interprets his gestures. The Microsoft Kinect is used for reading the user’s movements and techniques of image processing are performed to detect if the user’s hands are open or closed. After this information is obtained, it runs through a state machine that identifies what the user is trying to do in the application. In order to evaluate how robust the system is in a real public environment, some criteria that could interfere with the interactive task are evaluated, as the difference in brightness in the environment and the presence of another people in the same place of interaction. Three applications were developed as a case study and each one was evaluated differently, one of them being used for formal user evaluation. Given the results of the performed tasks, it is possible to conclude that the system, although not behaving correctly in all situations, has potential use if its difficulties are circumvented, most of which come from Kinect’s own inherent limitations. The proposed system works well enough for selection and manipulation of large objects and for use in applications based on pan & zoom, like those that supports map navigation, for example, and difference of ilumination or the presence of other persons on the environment does not interfere with the interaction process.

Com o constante crescimento tecnológico, é bastante comum deparar-se com um display público em lugares de grande concentração de pessoas, como aeroportos e cinemas. Apesar de possuírem informações úteis, esses displays poderiam ser melhor aproveitados se fossem interativos. Baseando-se em pesquisas sobre a interação com displays grandes e as características próprias de um display colocado em um espaço público, busca-se uma maneira de interação que seja adequada a esse tipo de situação. O presente trabalho introduz um método de interação por gestos sem necessitar que o usuário interagente segure ou tenha nele acoplado qualquer dispositivo ao interagir com um display público. Para realizar as tarefas que deseja, o usuário só precisa posicionar-se frente ao display e interagir com as informações na tela com suas mãos. São suportados gestos para navegação, seleção e manipulação de objetos, bem como para transladar a tela de visualização e ampliá-la ou diminui-la. O sistema proposto é construído de forma que possa funcionar em aplicações diferentes sem um grande custo de implantação. Para isso, é utilizado um sistema do tipo cliente-servidor que integra a aplicação que contém as informações de interesse do usuário e a que interpreta os seus gestos. É utilizado o Microsoft Kinect para a leitura dos movimentos do usuário e um pós-processamento de imagens é realizado de modo a detectar se as mãos do usuário se encontram abertas os fechadas. Após, essa informação é interpretada por uma máquina de estados que identifica o que o usuário está querendo executar na aplicação cliente. Afim de avaliar o quão robusto o sistema se portaria em um ambiente público real, são avaliados critérios que poderiam interferir na tarefa interativa, como a diferença de luminosidade do ambiente e a presença de mais pessoas no mesmo local de interação. Foram desenvolvidas três aplicações a título de estudo de caso e cada uma delas foi avaliada de forma diferente, sendo uma delas utiliza para fins de avaliação formal com usuários. Demonstrados os resultados da avaliação realizada, conclui-se que o sistema, apesar de não se portar corretamente em todas situações, tem potencial de uso desde que sejam contornadas suas deficiências, a maior parte das quais originária das próprias limitações inerentes ao Kinect. O sistema proposto funciona suficientemente bem para seleção e manipulação de objetos grandes e para aplicações baseadas em interação do tipo pan & zoom, como navegação em mapas, por exemplo, e não é influenciado por diferenças de iluminação ou presença de outras pessoas no ambiente. / With the constant technological growth, it is quite common to come across a public display in places with high concentration of people, such as airports and theaters. Although they provide useful information, these displays could be better employed if they were interactive. Based on research on topics of interaction with large displays and the characteristics of a display placed in a public space, a way of interaction that is suitable for this kind of situation is searched. This paper introduces a method of interaction by gestures without requiring that the interacting user take hold or have to him attached any device to interact with a public display. To accomplish the tasks he wants, he needs just to position himself in front of the display and to interact with the information on the screen with his hands. Gestures supported provide navigation, selection and manipulation of objects as well as to pan and zoom at the screen. The proposed system is constructed so that it works in different applications without a large installation cost. In order to achieve this, the system implements a client-server model application that is able to integrate the part that contains the useful information to the user and the one that interprets his gestures. The Microsoft Kinect is used for reading the user’s movements and techniques of image processing are performed to detect if the user’s hands are open or closed. After this information is obtained, it runs through a state machine that identifies what the user is trying to do in the application. In order to evaluate how robust the system is in a real public environment, some criteria that could interfere with the interactive task are evaluated, as the difference in brightness in the environment and the presence of another people in the same place of interaction. Three applications were developed as a case study and each one was evaluated differently, one of them being used for formal user evaluation. Given the results of the performed tasks, it is possible to conclude that the system, although not behaving correctly in all situations, has potential use if its difficulties are circumvented, most of which come from Kinect’s own inherent limitations. The proposed system works well enough for selection and manipulation of large objects and for use in applications based on pan & zoom, like those that supports map navigation, for example, and difference of ilumination or the presence of other persons on the environment does not interfere with the interaction process.

Abstract This thesis focuses on a systematic assessment of the feasibility and performance of situated crowdsourcing, as well as a basic understanding of the behaviours of its workers. While these aspects have been extensively studied for online and mobile crowdsourcing, this is not the case for situated crowdsourcing mainly due to its relative novelty. Such an assessment is crucial for the development of the crowdsourcing research agenda, so that task requesters and researchers alike can leverage, whenever appropriate, situated technologies for crowdsourcing efforts with more confidence. The key findings of this thesis illustrate how appropriately designed crowdsourcing tasks can perform well even in a complex deployment setting: situated technologies in public spaces. In the articles presented in this thesis, we empirically demonstrate that situated crowdsourcing performance can compete with other means of collecting crowd contributions. While situated technologies have been reported in the past to suffer from credibility and misappropriation issues, one should not forego the use of these technologies for crowdsourcing purposes assuming that the tasks are not haphazardly designed. The thesis also explores the behaviours of situated crowdsourcing workers through in-situ observations, video analysis and longitudinal individual tracking. Towards the end of the thesis, we revisit the research questions put forth in the thesis, and highlight how they were answered. We then discuss the benefits and drawbacks of situated crowdsourcing, and the differences between using non-personal and personal devices for this purpose. In both cases, the decisions made by the task requesters or researchers will ultimately depend on their goals and the task itself. We conclude the thesis by restating the thesis’ research agenda, reflecting on the challenges and opportunities of situated crowdsourcing, and our future work within this area. / Tiivistelmä Tämä väitöskirja keskittyy paikkasidonnaisen joukkoälyn käyttökelpoisuuden sekä tehokkuuden järjestelmälliseen arviointiin. Väitöskirja pyrkii myös ymmärtämään joukkoälyjärjestelmien työntekijöiden käyttäytymistä ohjaavia tekijöitä alustavalla tasolla. Paikkasidonnaisuus on tekijä, jota useimmiten verkossa tai mobiililaitteissa tehtävässä joukkotyötutkimuksessa ei ole mahdollista ymmärtää perusteellisesti. Paikkasidonnaisuus muodostaa kuitenkin elintärkeän osan joukkoälytutkimuksessa, ja sitä hyväksikäyttämällä sekä joukkotyön teettäjät että joukkoälytutkijat voivat soveltuvissa tilanteissa hyödyntää paikkasidonnaisia teknologioita luotettavamman joukkoälytiedon tuottamiseen. Väitöskirjan keskeisimmät löydökset osoittavat kuinka tarkoituksenmukaisesti toteutetut joukkoälytehtävät tuottavat luotettavaa tietoa, jopa monimutkaisissa käyttöympäristöissä kuten paikkasidonnaisia teknologioita hyödyntävissä julkisissa tiloissa. Väitöskirjan artikkelit osoittavat empiirisesti paikkasidoinnaisen joukkoälyn olevan kilpailukykyinen muiden joukkoälyteknologioiden kanssa, vaikka paikkasidonnaisten teknologioiden on aiemmin osoitettu kärsivän uskottavuuden puutteesta sekä väärinkäytöstä. Tämän väitöskirjan löydökset osoittavat, että oikein suunnitellut joukkoälytehtävät sopivat hyvin käytättäväksi kyseisten teknologioiden kautta. Suorituskyvyn sekä tehokkuuden lisäksi väitöskirjassa esitellään empiirisiin havaintoihin, videoanalyysiin, sekä yksilöiden pitkäkestoiseen tutkimukseen pohjautuvia löydöksiä joukkoälytyöntekijöiden käyttäytymismalleista. Väitöskirjan loppuosa käsittelee henkilökohtaisten ja julkisten laitteiden hyötyjä sekä haittapuolia suhteessa paikkasidonnaisten joukkoälyjärjestelmien käyttöön. Löydökset osoittavat, että kummassakin tapauksessa laitetyypin valintaa ohjaavat joukkoälytyön teettäjien tai tutkijoiden asettamat tavoitteet, sekä kyseessä olevat joukkoälytehtävät. Väitöskirjan päätteeksi palataan asetettuihin tutkimuskysymyksiin sekä vastaaviin löydöksiin, ja pohditaan paikkasidonnaisen joukkoälyn tuomia haasteita, mahdollisuuksia sekä tulevaisuuden visioita.

This thesis sets out to investigate and understand behaviours and interactions between individuals while using a large touch screen to plan a holiday trip. By using this approach, the tool digital visual planning by Yolean is indirectly examined. The tool is examined by looking at how it is used and the environment it is used in. The research question in this research paper is “How is interaction between individuals affected by using large touch screens with a digital visual planning tool in a meeting?”. Itis answered by using a mixed method approach containing grounded theory and design research methodology applied to an observation study. The results suggest that a group working for the same goal tend to get a leader without appointing them directly. This leader is also usually the person who stands in front of the rest and dictates what goes where. If problems arise, a shift in leadership occurs naturally and fluently. The digital visual planning tool helped the participants to easier express themselves and to motivate decisions. By using their whole body, they could more easily communicate. / Denna uppsats ämnar undersöka beteenden och interaktioner mellan individer i ett mötessammanhang användandes av ett digitalt visualiserings verktyg. Verktyget används på en stor pekskärm och är skapat av Yolean. Verktyget undersöks indirekt genom att utgå från hur verktyget används och miljön runt om verktyget. Forskningsfrågan som besvaras i uppsatsen lyder: ” Hur påverkas interaktion mellan individer genom att använda stora pekskärmar med ett digitalt visuellt planeringsverktyg i ett möte?”. Frågan besvaras genom en observationsstudie som utgår från blandade metodiker från både”grounded theory” (teoribildning genom empiri) samt designforskning. Resultatet tyder på att en grupp som arbetar tillsammans för att uppnå samma mål tenderar att få en ledare utan att specifikt tilldela denne rollen. Uppstår problem tenderar ett skifte av ledarskap ske. Skiftet sker naturligt och då utan verbala tilldelningar. Det visuella planeringsverktyget hjälpte deltagarna att uttrycka sina tankar och funderingar. Deltagarna kunde kommunicera med hela kroppen och att peka på specifika objekt som de ville diskutera. Genom denna frihet kunde de mer noggrant kommunicera sina förslag och tankar till resten av gruppen.

This thesis investigates the impact of different factors on public displays that use touch and mid-air gestures. We present a novel application for public displays: MirrorTouch — a game combining touch interaction with mid-air gesturing through depth sense. We studied the impact of these factors through a series field studies. We show that clear affordances are important to communicate both modalities clearly and also how to improve the conversion rate of passers-by. We present our findings of how social situations around such a display varied with the location where the display was placed. We also present quantitative data on group behavior around our public display, such as a measuring of the honeypot effect and how people made transitions between the different modalities. Our results can be of great value to designers and researchers of public displays that want to deploy displays with similar techniques.