Clemson University
Studies on the Effects of Operator Fatigue on Performance, Trust, and Workload Demand in Human-In-The-Loop Ai-Enabled Drone Systems
Abstract
dc:description.abstractCivil infrastructure must be regularly inspected to ensure its safety and reliability. Traditionally, risk engineers performed these inspections manually and in person. However, with the introduction of drones equipped with artificial intelligence (AI), inspections of civil infrastructure can now be conducted remotely. These AI-enabled drones capture and analyze images, helping engineers monitor large and complex structures from a distance. However, working with integrated systems consisting of multiple drones, especially those with varying levels of reliability, can be challenging. Operators must divide their attention, determine how much to trust each drone, and manage mental workload, which can affect their performance. Another key factor often overlooked is mental fatigue, which can impair human performance. This dissertation examines how the human operators’ trust in the AI-enabled integrated drone system and their workload and fatigue affect their performance when the system may consist of unreliable drones. An inspection drone is defined as a single infrastructure inspection drone equipped with its own AI system to assist with defect detection. The integrated inspection system is defined as a system consisting of two individual inspection drones operating together. The four studies conducted as part of this dissertation explore how different combinations of drone reliability levels and mental fatigue influence human operators’ trust, task workload, and performance. Specifically, the first study examined and explained that the human operators' trust decreased and the task workload increased when the integrated inspection systems' reliability decreased. The second study explained that when the operator was fatigued, their trust decreased, and workload increased. The third study showed that, similar to studies one and two, operators’ trust goes down as the AI-enabled integrated drone’s reliability goes down. For Studies 1 and 3, integrated drones' reliability levels also led to operators’ poorer performance, which was measured by the number of errors operators made. Human performance decreases as reliability goes down, which means operators make more errors. For study 2, there is no statistical significance for error rate for reliability. None of the studies found a statistically significant error rate for fatigued vs non-fatigued conditions. Using wearable technologies, including a smartwatch and an eye-tracker, this study collected real-time physiological data to train machine learning models capable of classifying operator fatigue. The analysis revealed that parameters derived from physiological indicators [electrodermal activity (EDA), skin temperature, and pupil diameter] significantly differentiated between fatigued and non-fatigued human operators. These findings contribute to the growing body of research supporting the integration of adaptive AI systems that can dynamically respond to human mental states, particularly in the context of infrastructure inspection using AI-enabled drones.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy (PhD)
- Level thesis:degree_level
- Dissertation
- Discipline thesis:degree_discipline
- Civil Engineering
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Lopes, Snowil
- Contributors dc:contributor
-
- Dr. Kapil Chalil Madathil
- Dr. Da Li
- Dr. Patrick Rosopa
- Dr. Mik Carbajales-Dale
Subjects
dc:subject × 6Identifiers
dc:identifier.*- Repository record dc:identifier
- https://open.clemson.edu/all_dissertations/4092