Driver distraction is a major contributor to car crashes, and visual distraction caused by using invehicle infotainment systems (IVIS) degrades driving performance and increases crash risk. Air gesture interfaces were developed to mitigate for driver distraction, and using auditory displays showed a decrease in off-road glances and an improved perceived workload. However, the design of auditory displays was not fully investigated. This thesis presents directional research in the design of auditory displays for air-gesture IVIS through two dual-task experiments of driving a simulator and air-gesture menu navigation. Experiment 1 with 32 participants employed a 2x4 mixed-model design, and explored the effect of four auditory display conditions (auditory icon, earcon, spearcon, and no-sound) and two menu-generation interfaces (fixed and adaptive) on driving performance, eye glance behavior, secondary task performance and subjective perception. Each auditory display (within-subjects) was tested with both a fixed and adaptive menu-generation interface (between-subjects). Results from Experiment 1 demonstrated that spearcon provided the least visual distraction, least workload, best system usability and was favored by participants; and that fixed menu generation outperformed adaptive menu generation in driving safety and secondary task performance. Experiment 2 with 24 participants utilized the best interface to emerge from Experiment 1 to further explore the auditory display with the most potential: spearcon. 70% spearcon and 40% spearcon were compared to text-to-speech (TTS) and no audio conditions. Results from Experiment 2 showed that 70% spearcon induced less visual distraction than 40% spearcon, and that 70% spearcon resulted in the most accurate but slowest secondary task selections. Experimental results are discussed in the context of the multiple resource theory and the working memory model, design guidelines are proposed, and future work is discussed. / Master of Science / Driver distraction is a major cause of car accidents, and using in-vehicle infotainment systems (IVIS) while driving can distract drivers and increase the risk of crashes. Air gesture interfaces and auditory displays were created to help reduce driver distraction, and using auditory displays has been shown to decrease the number of times a driver looks away from the road and to improve the perceived workload of the driver. However, the design of auditory displays has not been thoroughly studied. This study examined the design of auditory displays for air gesture IVIS through two experiments in which participants drove a simulator and used air gesture menus while navigating. The first experiment, which included 32 participants, looked at the effect of four different types of auditory displays (auditory icon, earcon, spearcon, and no sound) and two different types of menu-generation interfaces (fixed and adaptive) on driving performance, eye glance behavior, secondary task performance, and subjective perception. The second experiment, which included 24 participants, compared the use of 70% and 40% spearcon displays to text-to-speech and no audio conditions. The results of these experiments showed that using spearcon displays resulted in the least visual distraction and workload, the best system usability, and the most accurate but slowest secondary task selections. These findings are discussed in relation to existing theories of how the brain processes multiple tasks, and design guidelines for auditory displays are proposed for future research.

[ES] La investigación reportada en la presente tesis doctoral se lleva a cabo a través de la metodología de la ciencia del diseño que se centra en la creación y evaluación de artefactos. En esta tesis, el principal artefacto es el novedoso enfoque para diseñar y generar interfaces de usuario utilizando el conocimiento experto. Con el fin de permitir el uso del conocimiento experto, el enfoque propuesto se basa en la reutilización de patrones de diseño que incorporan el conocimiento experto del diseño de la interfaz y proporcionan soluciones reutilizables a diversos problemas de diseño. El objetivo principal de dicho enfoque es abordar el uso de patrones de diseño a fin de garantizar que los conocimientos especializados se integren en el diseño y la generación de interfaces de usuario para aplicaciones móviles y web. Las contribuciones específicas de esta tesis se resumen a continuación: Una primera contribución consiste en el marco AUIDP que se define para apoyar el diseño y la generación de interfaces adaptativas para aplicaciones web y móviles utilizando patrones de diseño HCI. El marco propuesto abarca tanto la etapa de diseño como la de ejecución de dichas interfaces. En el momento del diseño, los modelos de patrones de diseño junto con la interfaz de usuario y el perfil de usuario se definen siguiendo una metodología de desarrollo específica. En tiempo de ejecución, los modelos creados se utilizan para permitir la selección de patrones de diseño de HCI y para permitir la generación de interfaces de usuario a partir de las soluciones de diseño proporcionadas por los patrones de diseño relevantes. La segunda contribución es un método de especificación para establecer un modelo de ontología que convierte la representación tradicional basada en texto en la representación formal del patrón de diseño de HCI. Este método adopta la metodología Neon para lograr la transición de las representaciones informales a las formales. El modelo de ontología creado se llama MIDEP, que es una ontología modular que captura el conocimiento sobre los patrones de diseño, así como la interfaz de usuario y el perfil del usuario. La tercera contribución es el IDEPAR, que es el primer sistema dentro del marco global del AUIDP. Este sistema tiene como objetivo recomendar automáticamente los patrones de diseño más relevantes para un problema de diseño dado. Se basa en un enfoque híbrido que utiliza una combinación mixta de técnicas de recomendación basadas en texto y ontología para producir recomendaciones de patrones de diseño que proporcionan soluciones de diseño apropiadas. La cuarta contribución es un sistema generador de interfaz llamado ICGDEP, que se propone para generar automáticamente el código fuente de la interfaz de usuario para aplicaciones web y móviles. El ICGDEP es el segundo sistema dentro del marco global de AUIDP y se basa en el uso de patrones de diseño de HCI que son recomendados por el sistema IDEPAR. Su objetivo principal es generar automáticamente el código fuente de la interfaz de usuario a partir de las soluciones de diseño proporcionadas por los patrones de diseño. Para lograr esto, el sistema ICGDEP utiliza un método que permite la generación de código fuente de interfaz de usuario para la aplicación de destino. Las contribuciones aportadas en la presente tesis han sido validadas a través de diferentes perspectivas. En primer lugar, la evaluación de la ontología MIDEP desarrollada se realiza utilizando preguntas de competencia, enfoques de evaluación basados en la tecnología y basados en aplicaciones. En segundo lugar, la evaluación del sistema IDEPAR se establece mediante un patrón producido por expertos y un estudio de evaluación centrado en el usuario. Luego, el sistema ICGDEP es evaluado en términos de ser utilizado efectivamente por los desarrolladores, considerando el factor de productividad. Por último, la evaluación del marco mundial de AUIDP se lleva a cabo mediante estudios de casos y estudios de usabilidad. / [CA] La investigació reportada en aquesta tesi doctoral es duu a terme a través de la metodologia de la ciència del disseny que se centra en la creació i avaluació d'artefactes. En aquesta tesi, el principal artefacte és el nou enfocament per dissenyar i generar interfícies d'usuari utilitzant el coneixement expert. Per tal de permetre l'ús del coneixement expert, l'enfocament proposat es basa en la reutilització de patrons de disseny que incorporen el coneixement expert del disseny de la interfície i proporcionen solucions reutilitzables a diversos problemes de disseny. L'objectiu principal d'aquest enfocament és abordar l'ús de patrons de disseny per tal de garantir que els coneixements especialitzats s'integrin en el disseny i la generació d'interfícies d'usuari per a aplicacions mòbils i web. Les contribucions específiques d'aquesta tesi es resumeixen a continuació: Una primera contribució consisteix en el marc AUIDP que es defineix per donar suport al disseny i generació d'interfícies adaptatives per a aplicacions web i mòbils utilitzant patrons de disseny HCI. El marc proposat inclou tant l'etapa de disseny com la d'execució de les interfícies esmentades. En el moment del disseny, els models de patrons de disseny juntament amb la interfície d'usuari i el perfil d'usuari es defineixen seguint una metodologia de desenvolupament específica. En temps d'execució, els models creats s'utilitzen per permetre la selecció de patrons de disseny de HCI i per permetre la generació de interfícies d'usuari a partir de les solucions de disseny proporcionades pels patrons de disseny rellevants. La segona contribució és un mètode d'especificació per establir un model d'ontologia que converteix la representació tradicional basada en text en la representació formal del patró de disseny de HCI. Aquest mètode adopta la metodologia Neon per aconseguir la transició de les representacions informals a les formals. El model d'ontologia creat s'anomena MIDEP, una ontologia modular que captura el coneixement sobre els patrons de disseny, així com la interfície d'usuari i el perfil de l'usuari. La tercera contribució és l'IDEPAR, que és el primer sistema dins del marc global de l'AUIDP. Aquest sistema té com a objectiu recomanar automàticament els patrons de disseny més rellevants per a un problema de disseny donat. Es basa en un enfocament híbrid que utilitza una combinació mixta de tècniques de recomanació basades en text i ontologia per produir recomanacions de patrons de disseny que proporcionen solucions de disseny apropiades. La quarta contribució és un sistema generador d'interfície anomenat ICGDEP, que es proposa per generar automàticament el codi font de la interfície d'usuari per a aplicacions web i mòbils. L'ICGDEP és el segon sistema dins del marc global d'AUIDP i es basa en l'ús de patrons de disseny de HCI que són recomanats pel sistema IDEPAR. El seu objectiu principal és generar automàticament el codi font de la interfície d'usuari a partir de les solucions de disseny proporcionades pels patrons de disseny. Per aconseguir-ho, el sistema ICGDEP utilitza un mètode que permet generar codi font d'interfície d'usuari per a l'aplicació de destinació. Les contribucions aportades a la present tesi han estat validades a través de diferents perspectives. En primer lloc, l'avaluació de l'ontologia MIDEP desenvolupada es fa utilitzant preguntes de competència, enfocaments d'avaluació basats en la tecnologia i basats en aplicacions. En segon lloc, l'avaluació del sistema IDEPAR s'estableix mitjançant un patró produït per experts i un estudi d'avaluació centrat en l'usuari. Després, el sistema ICGDEP és avaluat en termes de ser utilitzat efectivament pels desenvolupadors, considerant el factor de productivitat. Finalment, l'avaluació del marc mundial d'AUIDP es fa mitjançant estudis de casos i estudis d'usabilitat. / [EN] The research reported in the present PhD dissertation is conducted through the design science methodology that focuses on creating and evaluating artifacts. In the current thesis, the main artifact is the novel approach to design and generate user interfaces using expert knowledge. In order to enable the use of expert knowledge, the present approach is devoted to reuse design patterns that incorporate expert knowledge of interface design and provide reusable solutions to various design problems. The main goal of the proposed approach is to address the use of design patterns in order to ensure that expert knowledge is integrated into the design and generation of user interfaces for mobile and Web applications. The specific contributions of this thesis are summarized below: This first contribution is the AUIDP framework that is defined to support the design and generation of adaptive interfaces for Web and mobile applications using HCI design patterns. The proposed framework spans over design-time and run-time. At design-time, models of design patterns along with user interface and user profile are defined following a specific development methodology. At run-time, the created models are used to allow the selection of HCI design patterns and to enable the generation of user interfaces from the design solutions provided by the relevant design patterns. The second contribution is a specification method to establish an ontology model that turns traditional text-based representation into formal HCI design pattern representation. This method adopts the Neon methodology to achieve the transition from informal to formal representations. The created ontology model is named MIDEP, which is a modular ontology that captures knowledge about design patterns as well as the user interface and user's profile. The third contribution is the IDEPAR, which is the first system within the global AUIDP framework. This system aims to automatically recommend the most relevant design patterns for a given design problem. It is based on a hybrid approach that relies on a mixed combination of text-based and ontology-based recommendation techniques to produce design pattern recommendations that provide appropriate design solutions. The fourth contribution is an interface generator system called ICGDEP, which is proposed to automatically generate the user interface source code for Web and mobile applications. The proposed ICGDEP is the second system within the global AUIDP framework and relies on the use of HCI design patterns that are recommended by the IDEPAR system. It mainly aims at automatically generating the user interface source code from the design solutions provided by design patterns. To achieve this, the ICGDEP system is based on a generation method that allows the generation of user interface source code for the target application. The contributions provided in the present thesis have been validated through different perspectives. First, the evaluation of the developed MIDEP ontology is performed using competency questions, technology-based, and application-based evaluation approaches. Second, the evaluation of the IDEPAR system is established through an expert-based gold standard and a user-centric evaluation study. Then, the ICGDEP system is evaluated in terms of being effectively used by developers, considering the productivity factor. Finally, the evaluation of the global AUIDP framework is conducted through case studies and usability studies. / Braham, A. (2022). Towards Designing and Generating User Interfaces by Using Expert Knowledge [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/190920

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / The great advance in the semiconductor industry allowed a increase in the development and marketing of mobile electronic devices. With the expansion of this market, the need for new programming methods and a different view for the development of user interfaces increased. Interfaces that were used before only in desktops and relied on keyboard and mouse interaction are now used in a variety of devices, including cell phones, smartphones and tablets. Often making the use of touch screens as well as by voice commands. Taking into account these aspects of cross-platform and different usability, it becomes apparent the importance of interfaces that adapt "to the environment." With the advent of mobile devices, this particular area became of fundamental importance because this kind of devices has specifics characteristics that are essential to the composition of a satisfactory user interface. So, mobile devices are covering a large variety of features, which makes the interfaces development a very complex task. One way to develop and adapt user interfaces in order to facilitate handling and to reduce stress at the time of use of the device is through the use of user profiles and capabilities of devices. Therefore, that interface is adapted to the user needs and preferences, as well be able to fully adapt to the device features. Considering this assumption, this dissertation aims to present the architecture PIDIM. This architecture goal to assist in the customization and adaptation of user interfaces for mobile devices in pervasive environments. The user interfaces adapted for this process plans to facilitate the use of mobile devices. The proposed approach presents an architecture that uses concepts of Pervasive Computing enabling information access anytime, anyplace, and in any computing device. Besides, it represents data on the user s profile, so that adaptation of the interfaces is entirely focused on the end user. The knowledge representation about the user profile needed for PIDIM architecture modeling is done through ontologies due to the possibility of reuse of stored information. In order to validate and demonstrate the flow of operation of the proposed approach is presented a case study in the literature, which has as scenario the adaptation of user interfaces when it is in motion. / O grande avanço na indústria de semicondutores possibilitou um aumento no desenvolvimento e comercialização de dispositivos eletrônicos móveis. Juntamente com este mercado, cresceu a necessidade de novos métodos de programação e uma visão diferente para criação de interfaces. Interfaces que antes só eram utilizadas em desktops com base de interação teclado e mouse, hoje são utilizadas em diferentes tipos de dispositivos, como celulares, smartphones e tablets, seja utilizadas em telas sensíveis ao toque como também por comando de voz. Levando em conta estes aspectos de multiplataforma e diferentes usabilidades, torna-se visível a importância de interfaces que se adaptem "ao meio". Com o aparecimento dos dispositivos móveis, a área em questão passou a ser de fundamental importância, pois estes dispositivos possuem características particulares fundamentais para a composição de uma interface satisfatória ao usuário. Os dispositivos móveis estão abrangendo uma diversidade grande de características, o que torna o desenvolvimento de uma interface um processo complexo. Uma das formas de desenvolver e adaptar interfaces de usuário de forma a facilitar o manuseio e diminuir o estresse no momento da utilização do dispositivo é através do uso de perfis de usuários e capacidades de dispositivos, fazendo com que a interface se adapte às necessidades e preferências do usuário e consiga se adaptar totalmente às funcionalidades do dispositivo. Considerando isto, este trabalho tem como objetivo apresentar a arquitetura PIDIM, a fim de ajudar na personalização e adaptação de interfaces de usuário para dispositivos móveis em ambientes pervasivos. As interfaces de usuários adaptadas por este processo da arquitetura PIDIM visam facilitar a utilização de dispositivos móveis. A abordagem proposta apresenta uma arquitetura que utiliza conceitos de Computação Pervasiva possibilitando acesso à informação a qualquer hora, lugar, e dispositivo computacional, além de representar dados relativos ao perfil de usuários, para que a adaptação das interfaces seja totalmente focada no usuário final. A representação do conhecimento sobre o perfil do usuário necessário para a modelagem da arquitetura PIDIM é feita através de ontologias devido a possibilidade de reuso das informações armazenadas. A fim de validar e demonstrar o fluxo de funcionamento da abordagem proposta, é apresentado um estudo de caso, encontrado na literatura, o qual possui como cenário a adaptação de interfaces de usuários quando o mesmo se encontra em movimento.