This thesis studies the phenomenon of human-system interaction in smart homes as a practice of Internet of Things. The research was conducted through interviews, workshops and observations and followed an interpretive research paradigm of phenomenologically-situated paradigm of HCI and a qualitative research approach. The theories of Phenomenology and Postphenomenology were used to interpret the experiences, beliefs and views of the participants. The empirical findings were processed and a thematic analysis was followed in order to identify the main themes that emerged out of the interviews, workshops and observations. The discussion of the findings showed that the research questions were answered to the grade that the participants of the research design, use and interact with smart homes in a multiple and complex way.

The current research is directed at providing validated guidelines to direct the integration of audio into human-system interfaces. This work first discusses the utility of integrating audio to support multimodal human-information processing. Next, an auditory interactive computing paradigm utilizing Speech, Earcons, Auditory icons, and Spatial audio (SEAS) cues is proposed and guidelines for the integration of SEAS cues into multimodal systems are presented. Finally, the results of two studies are presented that evaluate the utility of using SEAS cues, developed following the proposed guidelines, in relieving perceptual and attention processing bottlenecks when conducting Unmanned Air Vehicle (UAV) control tasks. The results demonstrate that SEAS cues significantly enhance human performance on UAV control tasks, particularly response accuracy and reaction time on a secondary monitoring task. The results suggest that SEAS cues may be effective in overcoming perceptual and attentional bottlenecks, with the advantages being most revealing during high workload conditions. The theories and principles provided in this paper should be of interest to audio system designers and anyone involved in the design of multimodal human-computer systems.

The Architecture, Engineering, and Construction (AEC) industry is currently challenged by low productivity trends and labor shortages. Efforts in academia and industry alike invested in developing solutions to this pressing issue. The majority of such efforts moved towards modernization of the industry, making use of digitalization approaches such as cyber-physical systems (CPS). In this direction, various research works have developed methods to capture information from construction environments and elements and provide monitoring capabilities to measure construction productivity at multiple levels. At the root of construction productivity, the productivity at the worker level is deemed critical. As a result, previous works explored monitoring the productivity of construction workers and resources to address the industry's productivity problems. However, productivity trends are not promising and show a need to more rigorously address productivity issues. Labor shortages also exacerbated the need for increasing the productivity of the current labor workers. Active means to address productivity have been explored as a solution in recent years. As a result, previous research took advantage of CPS and developed systems that sense construction workers' actions and environment and enable interaction with workers to render productivity improvements. One viable solution to this problem is providing on-demand activity-related information to the workers while at work, to decrease the need for manually seeking information from different sources, including supervisors, thereby improving their productivity. Especially, construction workers whose activities involve visual and manual limitations need to receive more attention, as seeking information can jeopardize their safety. Multiple labor trades such as plumbing, steel work, or carpenters are considered within this worker classification. These workers rely on knowledge gathered from the construction project documentation and databases, but have difficulties accessing this information while doing their work. Research works have explored the use of knowledge retrieval systems to give access to construction project data sources to construction workers through multiple methods, including information booths, mobile devices, and augmented reality (AR). However, these solutions do not address the need of this category of workers in receiving on-demand activity related information during their work, without negatively impacting their safety. This research focuses on voice, as an effective modality most appropriate for construction workers whose activities impose visual and manual limit actions. to this end, first, a voice-based solution is developed that supports workers' productivity through providing access to project knowledge available in Building Information Modeling (BIM) data sources. The effect of the selected modality on these workers' productivity is then evaluated using multiple user studies. The work presented in this dissertation is structured as follows: First, in chapter 2, a literature review was conducted to identify means to support construction workers and how integration with BIM has been done in previous research. This chapter identified challenges in incorporating human factors in previous systems and opportunities for seamless integration of workers into BIM practices. In chapter 3, voice-based assistance was explored as the most appropriate means to provide knowledge to workers while performing their activities. As such, Chapter 3 presents the first prototype of a voice-based intelligent virtual agent, aka VIVA, and focuses on evaluating the human factors and testing performance of voice as a modality for worker support. VIVA was tested using a user study involving a simulated construction scenario and the results of the performance achieved through VIVA were compared with the baseline currently used in construction projects for receiving activity-related information, i.e., blueprints. Results from this assessment evidenced productivity performance improvements of users using VIVA over the baseline. Finally, chapter 4 presents an updated version of VIVA that provides automatic real-time link to BIM project data and provides knowledge to the workers through voice. This system was developed based on web platforms, allowing easier development and deployment and access to more devices for future deployment. This study contributes to the productivity improvements in the AEC industry by empowering construction workers through providing on-demand access to project information. This is done through voice as a method that does not jeopardize workers' safety or interrupt their activities. This research contributes to the body of knowledge by developing an in-depth study of the effect of voice-based support systems on worker productivity, enabling real-time BIM-worker integration, and developing a working worker-level productivity support solution for construction workers whose activities limit them in manually accessing project knowledge. / Doctor of Philosophy / The Architecture, Engineering, and Construction (AEC) industry is currently challenged by low productivity trends and labor shortages. At the root of productivity, the improving productivity of construction workers is of critical essence. Therefore, academia and industry alike have shown great interest in research to develop solutions addressing construction worker productivity. For this purpose, monitoring systems for construction worker support have been developed, but productivity trends do not seem to improve, while labor shortages have increased productivity concerns. Other approaches to address productivity improvements have explored active means for productivity support. These include monitoring systems that also interact with the user. Construction workers performing activities that require allocating immense attention while using both hands, e.g. plumbers, steel workers, carpenters, have not been the focus of previous research because of the challenges of their conditions and needs. The activities performed by these workers require access to construction project data and documentation. Still, it is difficult for these workers to access information from the documents while doing their work. Therefore, previous researchers have explored methodologies to bring project data and documentation to the field but providing workers on-demand access to this data and documents have not been thoroughly studied. This research focuses on identifying the most appropriate method to provide workers access to information during activities that require more visual and manual attention. Worker support is provided by developing a solution that provides workers access to knowledge during their activities without being disruptive. The study then evaluated the effect of providing non-disruptive access to information sources enabled through the developed solution on the productivity for workers. First, in chapter 2, this study reviews the literature on approaches to connect construction project databases, a.k.a. Building Information Modeling (BIM), and workers. This review identified system types, integration approaches, and future research trends for linking BIM sources and with workers. In addition, this chapter's outcomes highlight system interoperability challenges and challenges in developing interactive systems involving humans. In chapter 3, a voice-based support system was developed as the most appropriate method for worker support during work activities that limit visual and manual worker capabilities. Then, the performance benefits of using a voice-based support system for construction workers was evaluated through a user study involving simulated construction activities. Finally, in chapter 4, this study provided a new integration method to connect BIM and workers in real-time. This system allows workers to interact with information from BIM through voice. The system was developed based on web platforms, allowing easier development and deployment and access to more devices for future deployment. This study contributes to the productivity improvements in the AEC industry by empowering construction workers through providing on-demand access to project information. This is done through voice as a method that does not jeopardize workers' attention or interrupt their activities.

A framework was developed called the Extended Hedonomic Hierarchy (EHH) that provides a basis for evaluating pleasurable human-system experience. Results from a number of experiments within this framework that evaluated specific dimensions of the framework are reported. The 'Exposure' component of the EHH framework and hedonics of the system were investigated to see how changes would affect other dimensions, such as the occurrence of flow, the mode of interaction, and the needs of the user. Simulations and video games were used to investigate how repeated exposure affects flow, interaction mode, and the user needs. The Kansei Engineering method was used to measure user needs and investigate the effect of different hedonic properties of the system on user needs and flow. Findings reveal that: (a) pleasurable human-system experience increases linearly with repeated exposure to the technology of interest; (b) an habituation effect of flow mediated by day; (c) motivation to satisfy human need for technology is hierarchically structured and contributes to pleasurable human-system experience; (d) interactivity is hierarchically structured and seamless mode of interaction is a behavioral outcome of pleasurable human-system experience; (e) there are individual differences among users that affect the likelihood of experiencing pleasurable human-system interaction; (f) performance is positively correlated to flow and (g) the method of kansei engineering provides data from which informed decisions about design can be made and empirical research can be conducted. Suggestions for (a) making Hedonomics a reality in industry, the workplace, and in the field of Human Factors, (b) future research directions for Hedonomics, and (c) principles and guidelines for the practice of Hedonomics are discussed.

Current studies done using a learning test for children have problems as they only make evaluations of Physically and Neurologically Impaired (PNI) children who can succeed in the test and can be considered as a PASS/FAIL test. This pilot study takes a holistic view of cognitive testing of PNI children using a user-test-device triad model and provides a framework using non-PNI children and adults as controls. Comparisons using adapted off-the-shelf novel interfaces to the computer, in particular, an Electroencephalograph (EEG) head-set, an eye-tracker and a head-tracker and a common mouse were carried out. In addition, two novel multi-modal technologies were developed based on the use of brain-waves and eye-tracking as well as head-tracking technologies to support the study. The devices were used on three tests with increasing cognitive complexity. A self-developed measure based on success streaks (consecutive outcomes) was introduced to improve evaluations of PNI children. A theoretical model regarding a fit of ability to devices was initially setup and finally modified to fit the view of the empirical model that emerged from the outcomes of the study. Results suggest that while multi-modal technologies can address weaknesses of the individual component modes, a compromise is made between the user’s ability for multi-tasking between the modes and the benefits of a multi-modal device but the sample size is very small. Results also show children failing a test with a mouse but passing it subsequently when direct communication is used suggesting that a device can affect a test for children who are of a developing age. This study provides a framework for a more meaningful conversation between educational psychologists as well as other professionals and PNI parents because it provides more discrimination of outcomes in cognitive tests for PNI children. The framework provides a vehicle that addresses scientifically the concerns of parents and schools.