Digital media technologies have been extensively utilized in children's daily lives and many researchers, educators, caregivers, and developers have been interested in finding ways to utilize these technologies in educational settings to facilitate early cognitive development. Among a wide range of media technologies, the accessibility of voice assistants and smart speakers powered by Artificial Intelligence (AI) has notably increased. However, there is a paucity of knowledge about how this advanced technology can be used to teach young children important mathematical concepts during shared book reading. The current study aimed to examine whether and under what circumstances shared book reading with an AI voice agent would enhance preschool-aged children's learning of mathematical language, a critical domain-specific language highly associated with early numeracy skills and vocabulary development. Sixty-six participants who were recruited for home-visit and school-visit sessions were randomly assigned to one of three reading conditions to read a storybook with the AI voice agent three times: math storybook reading with dialogic questions, math storybook reading without dialogic questions, and non-math storybook reading with dialogic questions. The findings indicate that shared math storybook reading supports children's target mathematical language learning differently based on their initial understanding of numeracy skills. Children with higher levels of numeracy skills demonstrated greater benefits from simply listening to the story, whereas children with lower levels of numeracy skills showed a tendency to learn better when hearing questions and feedback from the AI voice agent. This study provides implications for the use of advanced technology involving social interaction to support children's learning of key mathematical language that can benefit from repeated reading. / Doctor of Philosophy / Digital media technologies have been widely used in children's daily lives and many researchers, educators, caregivers, and developers have been interested in finding ways to utilize these technologies in educational environments to support children's early cognitive development. Among a wide range of media technologies, more and more families with young children have access to smart speakers using voice assistant technology where users can talk to and give commands verbally. However, we do not know much about how this advanced technology can be used to teach young children important mathematical concepts during everyday activities. The goal of this study is to look at whether and in what condition shared book reading with an AI voice agent would support preschool-aged children's learning of mathematical language such as fewer, fewest, and a little bit which is critical in developing numeracy skills and vocabulary. Sixty-six participants who were recruited for home-visit and school-visit sessions were randomly assigned to one of the three reading conditions to read a storybook with the AI voice agent three times: math storybook reading with dialogic questions, math storybook reading without dialogic questions, and non-math storybook reading with dialogic questions. The findings suggest that shared math storybook reading supports children's target mathematical language learning differently based on their initial understanding of numeracy skills. Children with higher levels of numeracy skills benefited more from simply listening to the story, whereas children with lower levels of numeracy skills showed a tendency to learn better when hearing questions and feedback from the AI voice agent. This study provides implications for the use of advanced technology involving social interaction to support children's learning of mathematical language that can benefit from repeated reading.

No description available.

Smart speakers, such as the Amazon Echo or Google Home, have become ubiquitous in our daily lives due to their convenience, which offers interactive actions through the use of simple voice commands. These devices allow users to issue a wide range of commands for a variety of services. Users can ask in natural language questions about the weather, stock market, online shopping orders, and other general information. These devices can also be used to control lights, and heating systems, and set timers and alarms in the smart home. However, as smart speaker systems become more prevalent, new security and privacy, usability, and context awareness concerns will need to be explored and addressed. In this dissertation, we carry out the effort to understand and mitigate privacy leaks from third-party applications, improve usability testing using interactability metrics, and improve context-awareness in a multi-occupant home using background sounds. We first study the privacy risks resulting from smart speaker apps developed by third-party developers. Having a device permanently on and always listening led to concerns over user privacy. In addition, the use of the third-party app on smart speaker platforms introduces arguably more serious privacy risks than using only the platform's built-in apps, due to the open nature of the app marketplaces. We explore how an adversary can efficiently create a valid smart speaker app to eavesdrop on users. We developed three different strategies for implementing a malicious app. To mitigate this threat, we propose a strategy for users to limit the success of this adversary. We designed a measurement app to look at the effect of various environmental factors in the home impacting what the third party can hear, and therefore provide users with a recommendation to place their smart speaker in locations that limit the success of this adversary. Next, we propose the idea of an interactability score to quantify how well a smart speaker app can accept potentially different ways a user may express their commands. However, voice-generated input data creates many unpredictable test cases since there are many different ways of how someone will express the same intention. In addition, each third-party developer could implement their own voice commands, making it difficult for users to remember what commands a particular app can process. The architecture of current smart speaker apps further complicates the testing process since the app is hosted on the smart speaker platform as a black-box. Therefore, we develop a testing framework to automatically and systematically evaluate the interactability of the smart speaker applications. It measures how well an app has been implemented to accept different kinds of user interaction. We also focus on improving context-awareness access control for smart speakers. The convenience of these devices is tempered by the possibility of performing unintended or intended actions. At home, the device is usually placed in a fixed location and accessed by multiple people with complex relationships between them, and these complex relationships can lead to complex access control requirements, where the context factors and interpersonal relationships should play a significant role. We design a system to be run on a smart speaker that makes use of the sounds in the home to estimate the current state of the house, e.g. number of occupants, activities being engaged, social relation of occupants, etc. This context information is used to decide whether to execute the command, prompt for confirmation or reject the command entirely. We also designed a simple pictorial configuration utility to help non-expert users configure their access rules. / Computer and Information Science

No description available.

Voice User Interfaces (VUI) are rapidly gaining popularity, revolutionizing user interaction with technology through the widespread adoption in devices such as desktop computers, smartphones, and smart home assistants, thanks to significant advancements in voice recognition and processing technologies. Over a hundred million users now utilize these devices daily, and smart home assistants have been sold in massive numbers, owing to their ease and convenience in controlling a diverse range of smart devices within the home IoT environment through the power of voice, such as controlling lights, heating systems, and setting timers and alarms. VUI enables users to interact with IoT technology and issue a wide range of commands across various services using their voice, bypassing traditional input methods like keyboards or touchscreens. With ease, users can inquire in natural language about the weather, stock market, and online shopping and access various other types of general information.However, as VUI becomes more integrated into our daily lives, it brings to the forefront issues related to security, privacy, and usability. Concerns such as the unauthorized collection of user data, the potential for recording private conversations, and challenges in accurately recognizing and executing commands across diverse accents, leading to misinterpretations and unintended actions, underscore the need for more robust methods to test and evaluate VUI services. In this dissertation, we delve into voice interface testing, evaluation for privacy and security associated with VUI applications, assessment of the proficiency of VUI in handling diverse accents, and investigation into access control in multi-user environments. We first study the privacy violations of the VUI ecosystem. We introduced the definition of the VUI ecosystem, where users must connect the voice apps to corresponding services and mobile apps to function properly. The ecosystem can also involve multiple voice apps developed by the same third-party developers. We explore the prevalence of voice apps with corresponding services in the VUI ecosystem, assessing the landscape of privacy compliance among Alexa voice apps and their companion services. We developed a testing framework for this ecosystem. We present the first study conducted on the Alexa ecosystem, specifically focusing on voice apps with account linking. Our designed framework analyzes both the privacy policies of these voice apps and their companion services or the privacy policies of multiple voice apps published by the same developers. Using machine learning techniques, the framework automatically extracts data types related to data collection and sharing from these privacy policies, allowing for a comprehensive comparison. Next, researchers studied the voice apps' behavior to conduct privacy violation assessments. An interaction approach with voice apps is needed to extract the behavior where pre-defined utterances are input into the simulator to simulate user interaction. The set of pre-defined utterances is extracted from the skill's web page on the skill store. However, the accuracy of the testing analysis depends on the quality of the extracted utterances. An utterance or interaction that was not captured by the extraction process will not be detected, leading to inaccurate privacy assessment. Therefore, we revisited the utterance extraction techniques used by prior works to study the skill's behavior for privacy violations. We focused on analyzing the effectiveness and limitations of existing utterance extraction techniques. We proposed a new technique that improved prior work extraction techniques by utilizing the union of these techniques and human interaction. Our proposed technique makes use of a small set of human interactions to record all missing utterances, then expands that to test a more extensive set of voice apps. We also conducted testing on VUI with various accents to study by designing a testing framework that can evaluate VUI on different accents to assess how well VUI implemented in smart speakers caters to a diverse population. Recruiting individuals with different accents and instructing them to interact with the smart speaker while adhering to specific scripts is difficult. Thus, we proposed a framework known as AudioAcc, which facilitates evaluating VUI performance across diverse accents using YouTube videos. Our framework uses a filtering algorithm to ensure that the extracted spoken words used in constructing these composite commands closely resemble natural speech patterns. Our framework is scalable; we conducted an extensive examination of the VUI performance across a wide range of accents, encompassing both professional and amateur speakers. Additionally, we introduced a new metric called Consistency of Results (COR) to complement the standard Word Error Rate (WER) metric employed for assessing ASR systems. This metric enables developers to investigate and rewrite skill code based on the consistency of results, enhancing overall WER performance. Moreover, we looked into a special case related to the access control of VUI in multi-user environments. We proposed a framework for automated testing to explore the access control weaknesses to determine whether the accessible data is of consequence. We used the framework to assess the effectiveness of voice access control mechanisms within multi-user environments. Thus, we show that the convenience of using voice systems poses privacy risks as the user's sensitive data becomes accessible. We identify two significant flaws within the access control mechanisms proposed by the voice system, which can exploit the user's private data. These findings underscore the need for enhanced privacy safeguards and improved access control systems within online shopping. We also offer recommendations to mitigate risks associated with unauthorized access, shedding light on securing the user's private data within the voice systems. / Computer and Information Science

Smart speakers are becoming increasingly ubiquitous. Previous research has studied the security of these devices; however, only some studies have employed a penetration testing methodology. Moreover, most studies have only investigated models by well-known brands such as the Amazon or Google. Therefore, there is a research gap of penetration tests on less popular smart speaker models. This study aims to address this gap by conducting a penetration test on the less popular JBL Link Music with firmware version 23063250. The results show that the speaker is subject to several security threats and is vulnerable to some attacks. The Bluetooth Low Energy implementation is vulnerable to passive eavesdropping. Additionally, the speaker is vulnerable to an 802.11 denial of service attack, and a boot log containing sensitive information can be accessed through a serial communication interface. It is concluded that the speaker is, in some aspects, insecure. / Smarta högtalare blir alltmer närvarande. Tidigare forskning har undersökt säkerheten kring dessa, dock har endast några använt en penetrerings testnings metolologi. Därutover har de flesta studier endast studerat modeller av välkända varumärken som Google eller Amazon. Därmed finns en vetenskaplig kunskapslucka kring penetrationstester av mindre populära modeller. Denna studie syftar till att bemöta denna lucka genom att utföra ett penetrationstest av den mindre populära JBL Link Music med mjukvaruversion 23063250. Resultaten visar att högtalaren är utsatt för flera säkerhetshot och är sårbar för några attacket. Bluetooth Low Energy implementationen är sårbar för passiv avlyssning. Därutöver är högtalaren sårbar för en 802.11 denial of service attack och en boot logg innehållande känslig information kan nås genom ett seriellt kommunikations gränssnitt. Slutsatsen dras att högtalaren, i vissa aspekter, är osäker.

The online retail environment is expanding, enhancing the possibilities for customers to shop online. On the one hand, a proliferation of online channels establishes a multichannel online retailing landscape, which offers customers more alternatives in terms of where to shop online. On the other hand, a change in the user interaction mode of existing customer touchpoints, from graphics to voice, creates new voice dialog interfaces, which enhance the way with regard to how customers can shop online. In this context, this publication-based dissertation aims to generate theoretical and practical contributions on these two most recent developments in online retailing, i.e., multichannel online retailing and voice dialog interfaces, to improve marketing mix decision-making.