Improvements in autonomous systems technology and a growing demand within military operations are spurring a revolution in Human-Robot Interaction (HRI). These mixed-initiative human-robot teams are enabled by Multi-Modal Communication (MMC), which supports redundancy and levels of communication that are more robust than single mode interaction. (Bischoff & Graefe, 2002; Partan & Marler, 1999). Tactile communication via vibrotactile displays is an emerging technology, potentially beneficial to advancing HRI. Incorporation of tactile displays within MMC requires developing messages equivalent in communication power to speech and visual signals used in the military. Toward that end, two experiments were performed to investigate the feasibility of a tactile language using a lexicon of standardized tactons (tactile icons) within a sentence structure for communication of messages for robot to human communication. Experiment one evaluated tactons from the literature with standardized parameters grouped into categories (directional, dynamic, and static) based on the nature and meaning of the patterns to inform design of a tactile syntax. Findings of this experiment revealed directional tactons showed better performance than non-directional tactons, therefore syntax for experiment two composed of a non-directional and a directional tacton was more likely to show performance better than chance. Experiment two tested the syntax structure of equally performing tactons identified from experiment one, revealing participants’ ability to interpret tactile sentences better than chance with or without the presence of an independent work imperative task. This finding advanced the state of the art in tactile displays from one to two word phrases facilitating inclusion of the tactile modality within MMC for HRI

There are two theoretical approaches to the nature of attentional resources. One proposes a single, flexible pool of cognitive resources; the other poses there are multiple resources. This study was designed to systematically examine whether there is evidence for multiple resource theory using a counting task consisting of visual, auditory, and tactile signals using two experiments. The goal of the first experiment was the validation of a multi-modal secondary loading task. Thirty-two participants performed nine variations of a multi-modal counting task incorporating three modalities and three demand levels. Performance and subjective ratings of workload were measured for each of the nine conditions of the within-subjects design. Significant differences were found on the basis of task demand level, irrespective of modality. Moreover, the perceived workload associated with the tasks differed by task demand level and not by modality. These results suggest the counting task is a valid means of imposing task demands across multiple modalities. The second experiment used the same counting task as a secondary load to a primary visual monitoring task, the system monitoring component of the Multi-Attribute Task Battery (MATB). The experimental conditions consisted of performing the system monitoring task alone as a reference and performing system monitoring combined with visual, auditory, or tactile counting. Thirty-one participants were exposed to all four experimental conditions in a within-subjects design. Performance on the primary and secondary tasks was measured, and subjective workload was assessed for each condition. Participants were instructed to maintain performance on the primary task, irrespective of condition, which they did so effectively. Secondary task performance for the visual-auditory and visual-tactile conditions was significantly better than for the visual-visual dual task condition. Subjective workload ratings were also consistent with the performance measures. These results clearly indicate that there is less interference for cross-modal tasks than for intramodal tasks. These results add evidence to multiple resource theory. Finally, these results have practical implications that include human performance assessment for display and alarm development, assessment of attentional reserve capacity for adaptive automation systems, and training.

No description available.

Although the instructional designer plays a role in the professional development of instructors in higher education, their perspectives are not explored as frequently compared to instructors and students in multi-modal research. Instructional designers need to keep up with the emerging technologies and changing educational landscape in order to equip instructors with the skills and knowledge they need to create meaningful learning experiences. One way to help instructional designers be able to help guide and support instructors is by creating guiding frameworks that help a designer think through the process and considerations when creating learning environments such as multi-modal. Using a design and development study, a guiding framework was constructed and operationalized using the design phase of the ADDIE model. / Doctor of Philosophy / Although the instructional designer plays a role in the professional development of instructors in higher education, their perspectives are not explored as frequently compared to instructors and students in multi-modal research. The term multi-modal refers to learning environments that take place across multiple modes. For instance, blended or hybrid learning environments. Instructional designers need to keep up with the emerging technologies and changing educational landscape in order to equip instructors with the skills and knowledge they need to create meaningful learning experiences. One way to help instructional designers be able to help guide and support instructors is by creating guiding frameworks that help a designer think through the process and considerations when creating learning environments such as multi-modal. This study conducted a design and development study which involved an analysis of literature and other resources to inform the development of the guiding framework. This study sought to address the practical needs of instructional designers in higher education through the creation of a framework that helps ensure they address key aspects in the design of multi-modal learning environments (i.e., blended or hybrid). The framework is grounded in theory and informed by an analysis of the literature. In order to validate the framework, an expert review panel of three was recruited to provide their insight and expertise. The feedback data was analyzed and the framework was enhanced based on the feedback. the framework helps contribute to the field by providing well-informed guidance in how to effectively design multi-modal environments. When instructional designers can effectively do this, they improve their practice which translates into the curriculum as well as the instructors who work closely with instructional designers. Overall, this can improve student outcomes.

This research examines a possible relationship between intentional memory and possible phonologic cues in the human voice. Specifically, if someone has told us something in the past, does hearing that same voice at the time of recall affect our ability to remember what was said? Also, if voice cues do affect memory, is the effect voice-specific? Since most standardized assessments of student learning and tests of human memory rest their conclusions about human learning solely on non-aural tests, it is necessary to determine if student performance changes with test modality. Via a computer program, ninety-five adults each listened to a male voice read a one-minute story and were then randomly assigned to take one of three different tests consisting of multiple-choice and fill-in-the-blank items. In the first test, the male voice from the story read all questions and possible answers. The second test used a different male voice to read while the third test was text-only. All tests contained identical content and gave single-modality cues only, text or speech. Results show no significant difference in long-term recall or recognition with respect to test-modality. Further research in this area is encouraged to determine if conclusions are generalizable to wider populations and hold for longer memory intervals. / Ph. D.

Mathematics instruction and discourse typically involve two modes of communication: speech and graphical presentation. For the communication to remain situated, dynamic synchrony must be maintained between the speech and dynamic focus in the graphics. Sighted students use vision for two purposes: access to graphical material and awareness of embodied behavior. This embodiment awareness keeps communication situated with visual material and speech. Our goal is to assist students who are blind or visually impaired (SBVI) in the access to such instruction/communication. We employ the typical approach of sensory replacement for the missing visual sense. Haptic fingertip reading can replace visual material. We want to make the SBVI aware of the deictic gestures performed by the teacher over the graphic in conjunction with speech. We employ a haptic glove interface to facilitate this embodiment awareness. In this research, we address issues from the conception through the design, implementation, evaluation to the effective and successful use of our Haptic Deictic System (HDS) in inclusive classrooms. / Ph. D.

>Magister Scientiae - MSc / The recent years have seen an increase in the number of users accessing online services using communication devices such as computers, mobile phones and cards based credentials such as credit cards. This has prompted most governments and business organizations to change the way they do business and manage their identity information. The coming of the online services has however made most Internet users vulnerable to identity fraud and theft. This has resulted in a subsequent increase in the number of reported cases of identity theft and fraud, which is on the increase and costing the global industry excessive amounts. Today with more powerful and effective technologies such as artificial intelligence, wireless communication, mobile storage devices and biometrics, it should be possible to come up with a more effective multi-modal authentication system to help reduce the cases of identity fraud and theft. A multi-modal digital identity management system IS proposed as a solution for managing digital identity information in an effort to reduce the cases of identity fraud and theft seen on most online services today. The proposed system thus uses technologies such as artificial intelligence and biometrics on the current unsecured networks to maintain the security and privacy of users and service providers in a transparent, reliable and efficient way. In order to be authenticated in the proposed multi-modal authentication system, a user is required to submit more than one credential attribute. An artificial intelligent technology is used to implement a technique of information fusion to combine the user's credential attributes for optimum recognition. The information fusion engine is then used to implement the required multi-modal authentication system.

Modern traffic signal control systems have not changed significantly in the past 40-50 years. The most widely applied traffic signal control systems are still time-of-day, coordinated-actuated system, since many existing advanced adaptive signal control systems are too complicated and fathomless for most of people. Recent advances in communications standards and technologies provide the basis for significant improvements in traffic signal control capabilities. In the United States, the IntelliDriveSM program (originally called Vehicle Infrastructure Integration - VII) has identified 5.9GHz Digital Short Range Communications (DSRC) as the primary communications mode for vehicle-to-vehicle (v2v) and vehicle-to-infrastructure (v2i) safety based applications, denoted as v2x. The ability for vehicles and the infrastructure to communication information is a significant advance over the current system capability of point presence and passage detection that is used in traffic control systems. Given enriched data from IntelliDriveSM, the problem of traffic control can be solved in an innovative data-driven and mathematical way to produce robust and optimal outputs.In this doctoral research, three different problems within a v2x environment- "enhanced pseudo-lane-level vehicle positioning", "robust coordinated-actuated multiple priority control", and "multimodal platoon-based arterial traffic signal control", are addressed with statistical techniques and mathematical programming.First, a pseudo-lane-level GPS positioning system is proposed based on an IntelliDriveSM v2x environment. GPS errors can be categorized into common-mode errors and noncommon-mode errors, where common-mode errors can be mitigated by differential GPS (DGPS) but noncommon-mode cannot. Common-mode GPS errors are cancelled using differential corrections broadcast from the road-side equipment (RSE). With v2i communication, a high fidelity roadway layout map (called MAP in the SAE J2735 standard) and satellite pseudo-range corrections are broadcast by the RSE. To enhance and correct lane level positioning of a vehicle, a statistical process control approach is used to detect significant vehicle driving events such as turning at an intersection or lane-changing. Whenever a turn event is detected, a mathematical program is solved to estimate and update the GPS noncommon-mode errors. Overall the GPS errors are reduced by corrections to both common-mode and noncommon-mode errors.Second, an analytical mathematical model, a mixed-integer linear program (MILP), is developed to provide robust real-time multiple priority control, assuming penetration of IntelliDriveSM is limited to emergency vehicles and transit vehicles. This is believed to be the first mathematical formulation which accommodates advanced features of modern traffic controllers, such as green extension and vehicle actuations, to provide flexibility in implementation of optimal signal plans. Signal coordination between adjacent signals is addressed by virtual coordination requests which behave significantly different than the current coordination control in a coordinated-actuated controller. The proposed new coordination method can handle both priority and coordination together to reduce and balance delays for buses and automobiles with real-time optimized solutions.The robust multiple priority control problem was simplified as a polynomial cut problem with some reasonable assumptions and applied on a real-world intersection at Southern Ave. & 67 Ave. in Phoenix, AZ on February 22, 2010 and March 10, 2010. The roadside equipment (RSE) was installed in the traffic signal control cabinet and connected with a live traffic signal controller via Ethernet. With the support of Maricopa County's Regional Emergency Action Coordinating (REACT) team, three REACT vehicles were equipped with onboard equipments (OBE). Different priority scenarios were tested including concurrent requests, conflicting requests, and mixed requests. The experiments showed that the traffic controller was able to perform desirably under each scenario.Finally, a unified platoon-based mathematical formulation called PAMSCOD is presented to perform online arterial (network) traffic signal control while considering multiple travel modes in the IntelliDriveSM environment with high market penetration, including passenger vehicles. First, a hierarchical platoon recognition algorithm is proposed to identify platoons in real-time. This algorithm can output the number of platoons approaching each intersection. Second, a mixed-integer linear program (MILP) is solved to determine the future optimal signal plans based on the real-time platoon data (and the platoon request for service) and current traffic controller status. Deviating from the traditional common network cycle length, PAMSCOD aims to provide multi-modal dynamical progression (MDP) on the arterial based on the real-time platoon information. The integer feasible solution region is enhanced in order to reduce the solution times by assuming a first-come, first-serve discipline for the platoon requests on the same approach. Microscopic online simulation in VISSIM shows that PAMSCOD can easily handle two traffic modes including buses and automobiles jointly and significantly reduce delays for both modes, compared with SYNCHRO optimized plans.

This thesis addresses the problem of generating smooth and efficiently executable locomotion trajectories for legged robots under contact constraints. In addition, we want the trajectories to have the property that small changes in the foot position generate small changes in the joint target path. The first part of this thesis explores methods to select poses for a legged robot that maximises the workspace reachability while maintaining stability and contact constraints. It also explores methods to select configurations based on a reduced-dimensional search of the configuration space. The second part analyses time scaling strategy which tries to minimize the execution time while obeying the velocity and acceleration constraints. These two parts effectively result in smooth feasible trajectories for legged robots. Experiments on the RoboSimian robot demonstrate the effectiveness and scalability of the strategies described for walking and climbing on a rock climbing wall.

abstract: This study explores the online recruitment and mobilization of followers in a social movement. In this study, I identify and analyze how certain narratives were produced, distributed and recirculated online by a social movement organization that depicted players in the movement in ways that engaged followers in actions of advocacy and support. Also, I examine how particular narratives were taken up, negotiated, amplified, and distributed by online supporters who eventually become co-tellers of the narrative and ultimately advocates on behalf of the social movement. By examining a selection of media statements, open letters, protest speeches, blogs, videos and pictures, I show how online practices might contribute to inspiring and mobilizing action or responses from a large number of followers. Data include selected excerpts from an online social movement that began in Norway in 2015 and later gathered momentum and strength outside of Norway and Europe. This multi-modal analysis of digital practices demonstrates how collaboratively produced narratives (e.g., of suffering, sorrow, persecution or resilience) emerge and gain traction in the digital space, the relationship between the temporal and spatial dimensions of narrative, and the role of collective memory in building a sense of community and shared identity. Demonstrating the dialogic and interactional dimensions of meaning-making processes, this case study informs how we might theorize and understand the role of identity and narrative in the emergence and amplification of social movements. / Dissertation/Thesis / Doctoral Dissertation English 2019