This thesis work examines how people accomplish annotation tasks (i.e., labelling data based on content) while working with an artificial intelligence (AI) system. When people and AI systems work together to accomplish a task, this is referred to as human-AI teaming. This study reports on the results of an interview and observation study of 15 volunteers from the Washington DC area as the volunteers annotated Twitter messages (tweets) about the COVID-19 pandemic. During the interviews, researchers observed the volunteers as they annotated tweets, noting any needs, frustrations, or confusion that the volunteers expressed about the task itself or when working with the AI. This research provides the following contributions: 1) an examination of annotation work in a human-AI teaming context; 2) the HATA (human-AI teaming annotation) framework with five key factors that affect the way people annotate while working with AI systems--background, task interpretation, training, fatigue, and the annotation system; 3) a set of questions that will help guide users of the HATA framework as they create or assess their own human-AI annotation teams; 4) design recommendations that will give future researchers, designers, and developers guidance for how to create a better environment for annotators to work with AI; and 5) HATA framework implications when it is put into practice.

abstract: Human-agent teams (HATs) are expected to play a larger role in future command and control systems where resilience is critical for team effectiveness. The question of how HATs interact to be effective in both normal and unexpected situations is worthy of further examination. Exploratory behaviors are one that way adaptive systems discover opportunities to expand and refine their performance. In this study, team interaction exploration is examined in a HAT composed of a human navigator, human photographer, and a synthetic pilot while they perform a remotely-piloted aerial reconnaissance task. Failures in automation and the synthetic pilot’s autonomy were injected throughout ten missions as roadblocks. Teams were clustered by performance into high-, middle-, and low-performing groups. It was hypothesized that high-performing teams would exchange more text-messages containing unique content or sender-recipient combinations than middle- and low-performing teams, and that teams would exchange less unique messages over time. The results indicate that high-performing teams had more unique team interactions than middle-performing teams. Additionally, teams generally had more exploratory team interactions in the first session of missions than the second session. Implications and suggestions for future work are discussed. / Dissertation/Thesis / Masters Thesis Human Systems Engineering 2019

No description available.

There are about 16,000 middle schools in the United States; the half million teachers who teach in them affect the academic and emotional lives of about a hundred students while working cooperatively with an array of adult personalities and endeavoring to cover the curriculum. Although research has been conducted on many components of the middle school, an in-depth look at teachers’ experiences with the concept is missing. The purposes of this study were to explore three ideas: the complexities of the work experiences of teachers who participate on interdisciplinary teams in urban middle schools, the possible interactions of the structures and principles of the middle school philosophy with their work lives, and how the reality of interdisciplinary teams connect to the ideals in the middle school and organizational theory literature. I conducted three in-depth phenomenological interviews with 15 urban middle school teachers. Teachers shared their teaching experiences, life histories (to put their experiences in context), and how teaching fits in with their lives. Subject matters, ethnicities, ages, and years experience varied. They came from 9 schools in 5 school districts in the Northeast. I include a brief history of how the junior high morphed into the middle school. The “ideal” practices, programs, and philosophy of the middle school and teaming (as defined by middle school and organizational theory literature) are explained and then contrasted with the realities. Results indicate that the “ideal” characteristics, as described in the literature, do not exist in all urban middle schools. Teachers lamented their absence and described their frustrations with student behavior, colleagues, administrators, and state testing. They also shared the joy they find in seeing their students progress, giving back to the community, and making a difference in students’ lives. I propose that these rewards make up for the incredible difficulties they face daily. I conclude that teachers need team planning time to implement the middle school characteristics and overcome the difficulties of teaching urban students, which include transience, absenteeism, poverty, lack of familial support, and a belief that being smart is “lame.” I also propose increasing community involvement and providing alternative schools.

No description available.

"This case study examined the implementation of the Professional Learning Communities (PLCs) at a high school that was labeled as a turnaround school (overall academic achievement proficiency below 55%) in 2006-2007. The foundation of PLCs is built on a shared vision and mission. The PLC principles of learning for all students, a collaborative culture, and a focus on results were the components most identified in the literature review, which helped shape the design of the methodology. Under the leadership of a new principal and with the implementation and focus on PLCs, Ocean Breeze High School's (OBHS) academic achievement proficiency grew from 48.6% to 87.5% within a three-year period. Based on this phenomenal progress, this case study investigated the relationship between PLC characteristics and instructional practices at this school. The data sources included fourteen interviews with eleven teachers and three administrators, observations of two PLC meetings, fourteen classroom observations, and a document review. The data from the interviews were transcribed and themes were matched to reflect common perceptions from teachers and administrators on PLC practices. PLC meetings were observed to identify collaborative interactions between team members. A document review included any documents or artifacts that the school has used to implement PLC principles or similar practices. Classroom observations were conducted to compare if instructional practices correlate with interview responses, PLC meetings, and the documents reviewed. This study found that there is a relationship between PLCs and instructional practices. The findings included: (a) teachers ensure that all students learn by developing common instructional guides that support the state curriculum; (b) teachers meet to discuss the curriculum, pedagogical strategies, and assessments in PLC meetings; and (c) teachers and administrators analyze data to make instructional decisions to enhance teaching and learning. / Ed. D.

The study is a descriptive analysis using a case-study methodology that identifies the critical elements (methods, tools, processes, personnel, and practices) of adversary analysis identified as a red team and red-teaming. A red team is the adversary element of the analytic method of red-teaming. The study incorporates interview data with organization leadership, subject matter experts, and red-team developers from Department of Defense (DoD), Intelligence Community (IC), and Federally Funded Research and Development Centers (FFRDC) organizations. The study also includes red-team governance documents, red-team briefings, and discussions to first identify the concepts, analyze the critical design elements of the concept(s), and develop a fundamental taxonomy or classification of red-team approaches based on these artifacts. The study compares and contrasts four red teams that utilize groups of adversary subject-matter experts for common themes, differences, and best practices. The data collection builds on grounded theory—i.e., identification of the methods, tools, processes, and personnel as the organizations understand and develop their red teams as part of their red-teaming analyses to address gaps in understanding possible adversaries. The four organizations studied are the U.S. Army, Training and Doctrine Command, University of Foreign Military and Cultural Studies; a Department of Defense unified combatant command; the U.S. Naval War College (NWC) and its red-team detachment; and a Sandia National Laboratories (SNL) Homeland Security and Defense, National Infrastructure Simulation and Analysis Center (NISAC). Two basic types of red teams are identified from the data with a hybrid between the two among the variations of the red-teaming concept. Some of the other findings from the four red teams include a need to develop common terms and standards; a need to explain the benefits of alternative analysis to decisionmakers; a need to develop trend analyses on types of red teams requested by sponsors; a need to design methods to capture non-state actors; a need to include more coalition and foreign partners; and a need to immerse red teams more fully into the culture to be understood. / Ph. D.

The wireless technology of today provides combat systems with the potential to communicate mission critical data to every asset involved in the operation. In such a dynamic environment, the network must be able maintain communication by adapting to subsystems moving relative to each other. A theoretical and experimental foundation is developed that allows an autonomous ground vehicle to serve as an adaptive communication node in a larger network. The vehicle may perform other functions, but its primary role is to constantly reposition itself to maintain optimal link quality for network communication. Experimentation with existing wireless network hardware and software led to the development, implementation, and analysis of two main concepts that provided a signal optimization solution. The first attracts the communication ground vehicle to the network subsystems with weaker links using a vector summation of the signal-to-noise ratio and network subsystem position. This concept continuously generates a desired waypoint for repositioning the ground vehicle. The second concept uses a-priori GIS data to evaluate the desired vehicle waypoint determined by the vector sum. The GIS data is used primarily for evaluating the viewshed, or line-of-sight, between two network subsystems using elevation data. However, infrastructure and ground cover data are also considered in navigation planning. Both concepts prove to be powerful tools for effective autonomous repositioning for maximizing the communication link quality. / Master of Science

Autonomous robots have proven to be useful for search and rescue (SAR) by being deployed in emergency situations and removing the need for direct human presence. However, there remains a need for effective communication between the robot and human operator to cooperate as a team. This thesis investigates the question of when to explain an autonomous agent’s behaviour in the setting of human-robot teaming. A game environment is developed to conduct a virtual SAR experiment to test the effect of explanation timings between two conditions: (1) always explaining - the robot provides explanations whenever possible, and (2) anticipatory explaining - the robot determines when to explain based on the context. When to explain is determined through the construction of argumentation frameworks modelling the context. The effect of anticipatory information pushing is tested on two metrics: team performance and explanation experience. Results indicate anticipatory explaining does not have a significant effect on team performance and participants’ explanation satisfaction. Additionally, participant feedback shows they prefer to be in control instead of cooperating as a team. These findings underline the importance of studying explanation presentation in high-demanding environments and indicate a need for interdisciplinary discussion on the design of human-robot teaming.

No description available.