Improvements in autonomous systems technology and a growing demand within military operations are spurring a revolution in Human-Robot Interaction (HRI). These mixed-initiative human-robot teams are enabled by Multi-Modal Communication (MMC), which supports redundancy and levels of communication that are more robust than single mode interaction. (Bischoff & Graefe, 2002; Partan & Marler, 1999). Tactile communication via vibrotactile displays is an emerging technology, potentially beneficial to advancing HRI. Incorporation of tactile displays within MMC requires developing messages equivalent in communication power to speech and visual signals used in the military. Toward that end, two experiments were performed to investigate the feasibility of a tactile language using a lexicon of standardized tactons (tactile icons) within a sentence structure for communication of messages for robot to human communication. Experiment one evaluated tactons from the literature with standardized parameters grouped into categories (directional, dynamic, and static) based on the nature and meaning of the patterns to inform design of a tactile syntax. Findings of this experiment revealed directional tactons showed better performance than non-directional tactons, therefore syntax for experiment two composed of a non-directional and a directional tacton was more likely to show performance better than chance. Experiment two tested the syntax structure of equally performing tactons identified from experiment one, revealing participants’ ability to interpret tactile sentences better than chance with or without the presence of an independent work imperative task. This finding advanced the state of the art in tactile displays from one to two word phrases facilitating inclusion of the tactile modality within MMC for HRI

2023 / Difficulties in communication during social interactions are a feature of the diagnosis of autism spectrum disorder (ASD). Little is known about the communication abilities of minimally-verbal (MV) individuals, which comprises one-third of autistic individuals. Natural language sampling (NLS) provides an optimal approach to capture communication abilities in this population. The goal of this project was to use NLS methods to examine multimodal communication: speech, gesture, gesture-speech combinations, and augmentative and alternative communication in MV autistic children and adolescents during social interactions.Study 1 examined how and why MV autistic children and adolescents (N=50; 6-21 years; 12 females) used their communication abilities during social interactions. Unexpectedly, adolescents produced more gesture-speech combinations and communicative functions but did not differ in gesture frequency. I explored their range of communicative functions across different communication modalities. As hypothesized, they produced more requests using gesture but used speech more often to disagree/agree/acknowledge and to respond to questions. I investigated the concurrent relation between gestural communication and spoken language. As hypothesized, fewer speech utterances were related to more gesturing after accounting for age, nonverbal IQ, and receptive communication ability. Study 2 explored how MV autistic children’s (N=47; 48-95 months; 10 females) modality and form of communication influenced parent responsivity. As expected, parents provided semantically-relevant (contingent) responses across children’s communication modalities. Unexpectedly, the precision of the child’s form (e.g., approximated gesture) did not influence parent contingent responding. I investigated the parent’s modality of response following the child’s modality of communication. Across children’s modalities of communication (e.g., gesture), parents responded most frequently using speech. Study 3 evaluated the predictive role of gestural communication in later spoken language ability in MV autistic children (N=50; 54-105 months; 10 females) receiving intervention. Unexpectedly, frequency of deictic gestures did not predict speech outcomes. As hypothesized, supplementary combinations predicted speech outcomes. Findings from this project underscore the importance of 1) examining multiple modalities and forms of communication to obtain a comprehensive understanding of communication skills; 2) inclusion of interactionist models to capture children’s input on parent responses in shaping language learning; and 3) the predictive role of gesture-speech combinations in facilitating spoken language outcomes.

<p>Providing carrier grade characteristics for Internet Protocol (IP) communication applications is a significant problem for IP application providers in order to offer integrated services that span IP&nbsp / and telecommunication networks. This thesis addresses the provision of life-cycle management, which is only one carrier grade characteristic, for a SoftBridge application, which is an example of IP communication applications. A SoftBridge provides semi-synchronous multi-modal IP-based communication. The work related to IP-Telecommunication integrated services and the SoftBridge is analyzed with respect to life-cycle management in a literature review. It is suggested to use an Application Server in a Next Generation Network (NGN) to provide life-cyclemanagement functionality for IP-Telecommunication applications. In this thesis, the Application Server is represented by a JAIN Service Logic Execution Environment(JSLEE), in which&nbsp / a SoftBridge application can be deployed, activated, deactivated, uninstalled and upgraded online.Two methodologies are applied in this research: exploratory prototyping, which evolves the development of a SoftBridge application, and empirical comparison, which is concerned with the empirical evaluation of a SoftBridge application in terms of carriergrade capabilities. A SoftBridge application called SIMBA&nbsp / provides a Deaf Telephony service similar to aprevious Deaf Telephony SoftBridge, However, SIMBA&rsquo / s SoftBridge design and implementation are unique to this thesis. In order to test the life-cycle&nbsp / management ability of SIMBA, an empirical evaluation is carried out including the experiments oflife-cycle management and call-processing performance. The final experimental results of the evaluation show that a JSLEE is able to provide life-cycle management for SIMBA without causing a significant decrease in performance. In conclusion, the life-cycle management can be provided&nbsp / or a SoftBridge application by using an Application Server such as a JSLEE. Futhermore, the results indicate that&nbsp / approach of using Application Server (JSLEE) integration should be&nbsp / sufficiently general to provide life cycle management, and indeed other carrier grade capabilities, for other IP communication applications. This allows IP communication applications to be&nbsp / &nbsp / &nbsp / integrated into an NGN.</p>

<p>Providing carrier grade characteristics for Internet Protocol (IP) communication applications is a significant problem for IP application providers in order to offer integrated services that span IP&nbsp / and telecommunication networks. This thesis addresses the provision of life-cycle management, which is only one carrier grade characteristic, for a SoftBridge application, which is an example of IP communication applications. A SoftBridge provides semi-synchronous multi-modal IP-based communication. The work related to IP-Telecommunication integrated services and the SoftBridge is analyzed with respect to life-cycle management in a literature review. It is suggested to use an Application Server in a Next Generation Network (NGN) to provide life-cyclemanagement functionality for IP-Telecommunication applications. In this thesis, the Application Server is represented by a JAIN Service Logic Execution Environment(JSLEE), in which&nbsp / a SoftBridge application can be deployed, activated, deactivated, uninstalled and upgraded online.Two methodologies are applied in this research: exploratory prototyping, which evolves the development of a SoftBridge application, and empirical comparison, which is concerned with the empirical evaluation of a SoftBridge application in terms of carriergrade capabilities. A SoftBridge application called SIMBA&nbsp / provides a Deaf Telephony service similar to aprevious Deaf Telephony SoftBridge, However, SIMBA&rsquo / s SoftBridge design and implementation are unique to this thesis. In order to test the life-cycle&nbsp / management ability of SIMBA, an empirical evaluation is carried out including the experiments oflife-cycle management and call-processing performance. The final experimental results of the evaluation show that a JSLEE is able to provide life-cycle management for SIMBA without causing a significant decrease in performance. In conclusion, the life-cycle management can be provided&nbsp / or a SoftBridge application by using an Application Server such as a JSLEE. Futhermore, the results indicate that&nbsp / approach of using Application Server (JSLEE) integration should be&nbsp / sufficiently general to provide life cycle management, and indeed other carrier grade capabilities, for other IP communication applications. This allows IP communication applications to be&nbsp / &nbsp / &nbsp / integrated into an NGN.</p>