No description available.

La validité des simulateurs de conduite est une question essentielle pour étayer et valoriser les travaux visant à mieux comprendre les comportements de conduite. Alors que les dimensions subjective et comportementale de la validité ont été régulièrement étudiées, elles font face à certaines limites. A l’opposé, rares sont les études qui se sont intéressées à la validité psychologique des simulateurs. Cette dimension compare l'implication des mécanismes qui sous-tendent les comportements entre conduite réelle et virtuelle. De plus, très peu d’études ont confronté les différentes dimensions de la validité, alors que cela pourrait contribuer au développement d’une métrologie des environnements virtuels. C’est précisément à ce niveau que se situe ce travail de thèse, au cours duquel des mesures classiques de la validité comportementale (vitesse, contrôle latéral) ont été confrontées à des mesures de la validité psychologique examinée sous l’angle de la charge mentale en prenant notamment comme indicateur les clignements oculaires. L’objectif principal était de déterminer si le niveau de charge mentale diffère entre conduite sur route et conduite sur simulateur dans des cas où les comportements observés ne permettent pas de faire de distinction.Pour répondre à cette question, ce travail de thèse a été organisé autour de trois expériences réalisées sur simulateur de conduite et d’une étude sur route réelle. Les deux premières expériences visaient à mieux cerner l’effet de certains facteurs inhérents à la conduite sur route ouverte (trafic, environnement) sur les comportements de conduite et la charge mentale. La troisième expérience était quant à elle destinée à comparer la charge mentale induite par la conduite réelle et la conduite sur simulateur « bas coût », en confrontant cette mesure de la validité psychologique aux mesures comportementales classiques. Enfin, la quatrième expérience a porté sur les effets du retour d’effort du volant sur cette même charge mentale.Les principaux résultats ont mis en évidence un niveau de charge supérieur en conduite simulée par comparaison à la conduite en situation réelle, alors que certaines mesures comportementales (vitesse) n’étaient pas différentes. Cette charge accrue sur simulateur ne semble cependant pas trouver son origine dans le retour d’effort au volant, les modalités de retour testées n’ayant pas eu d’effet sur les indicateurs de la charge. Au final, ce travail confirme que la prise en compte de la validité psychologique, examinée ici sous l’angle de la charge mentale, présente un intérêt dans une démarche d’évaluation, dans le but de mieux cerner le positionnement des utilisateurs face à un dispositif de réalité virtuelle. Il ouvre ainsi des perspectives pour améliorer la validité des simulateurs de conduite. / The validity of driving simulators is an essential subject to support and highlight the works aiming to understand driving behaviours more thoroughly. While the subjective and behavioural dimensions of that validity have often been studied, they encounter a few limits. On the other hand, studies about simulators’ validity are fairly rare. This dimension compares the implication of mechanisms inherent in behaviours between real and virtual driving. Furthermore, very few studies have considered the various dimensions of validity at once, whereas it could contribute to the development of a metrology for virtual environments. This thesis is precisely about this, with confrontations between classical measurements of behavioural validity (speed, lateral control) and measurements of psychological validity, examined from the viewpoint of mental workload - using indicators such as eye blinking. The main objective was to ascertain whether the mental workload levels vary between road driving and simulated driving when the observed behaviours aren’t relevant to make a clear difference.To answer this question, this thesis has been organised around three experiences carried out on driving simulators and a comparison between actual road driving and low-cost simulated driving. Two of those experiences were conducted to finely analyse the effect of some factors inherent to open road driving (traffic, environment) on driving behaviours and mental workload. The third experience was made to compare the mental workload induced by real driving and low-cost simulator driving, by putting this psychological validity measurement against classical behavioural measurements. Lastly, the fourth experience focused on the effects of the driving wheel’s force-feedback on this same mental workload.The main results revealed a higher workload level on simulated driving than on real driving, while some behavioural measurements (speed) were not different. This increased workload with the simulator does not seem to stem from the wheel’s force-feedback, since the tested feedback methods did not have a repercussion on the workload levels. In the end, this work confirms that taking psychological validity - examined here from a mental workload viewpoint - does have an interest within an evaluation process to analyse in a finer fashion the mental state of users when faced with a virtual reality system. It creates prospects to improve the validity of driving simulators.

<p>The advancement of robotic-assisted surgery (RAS) has revolutionized the field by enabling surgeons to perform intricate procedures with enhanced precision, improved depth perception, and more precise control. Despite these advancements, current RAS systems still rely on teleoperation, where surgeons control the robots remotely. The complexity of the master-slave control mechanism, along with the technical challenges involved, can impose significant mental workloads on surgeons. As excessive mental workload (MWL) can adversely affect performance and increase the likelihood of errors, addressing operator mental overload has become crucial for successful operation in RAS. To tackle this problem, there has been increased interest in developing robots that can provide operators with varying levels of assistance based on their MWL (i.e., adaptive system) during task execution. However, the research in this area is notably limited, primarily due to two key factors: the absence of a real-time MWL assessment framework and the lack of effective intervention strategies to mitigate MWL in RAS.</p> <p>This Ph.D. dissertation aims to fill these gaps by designing the adaptive system in RAS and exploring its impact on surgical task performance. The dissertation comprises three studies. The first study demonstrated the feasibility of the adaptive system in RAS by introducing an MWL-triggered semi-autonomous suction tool as a proof-of-concept. Building upon the insights gained from the first study, the second study focused on enhancing the adaptive system's adaptability to more complex RAS tasks. In particular, the second study proposed a task-independent MWL model that had potential to be applied to various RAS tasks. Additionally, more intelligent interventions were investigated. Furthermore, the third study aimed to investigate the benefits of adaptive system in RAS training by introducing a personalized and adaptive training program based on human MWL profile. The findings of this dissertation revealed evidence supporting the effectiveness of the adaptive system in moderating subjects’ MWL, and its potential in enhancing task performance in RAS. This dissertation highlights the potential of incorporating adaptive systems into future RAS platforms, so that to provide valuable support and assistance to surgeons during critical moments and facilitate surgical training by identifying and addressing the specific needs of surgeons.</p>

付記する学位プログラム名: デザイン学大学院連携プログラム / 京都大学 / 新制・課程博士 / 博士(工学) / 甲第24607号 / 工博第5113号 / 新制||工||1978(附属図書館) / 京都大学大学院工学研究科機械理工学専攻 / (主査)教授 椹木 哲夫, 教授 小森 雅晴, 教授 泉井 一浩 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

Robots, including UAVs, have found increasing use in helping humans with dangerous and difficult tasks. The number of robots in use is increasing and is likely to continue increasing in the future. As the number of robots increases, human operators will need to coordinate and control the actions of large teams of robots. While multi-robot supervisory control has been widely studied, it requires that an operator divide his or her attention between robots. Consequently, the use of multi-robot supervisory control is limited by the number of robots that a human or team of humans can reasonably control. Swarm robotics -- large numbers of low-cost robots displaying collective behaviors -- offers an alternative approach by providing the operator with a small set of inputs and parameters that alter the behavior of a large number of autonomous or semi-autonomous robots. Researchers have asserted that this approach is more scalable and offers greater promise for managing huge numbers of robots. The emerging field of Human-Swarm Interaction (HSI) deals with the effective management of swarms by human operators. In this thesis we offer foundational work on the effect of HSI (a) on the individual robots, (b) on the group as a whole, and (c) on the workload of the human operator. We (1) show that existing general swarm algorithms are feasible on existing robots and can display collective behaviors as shown in simulations in the literature, (2) analyze the effect of interaction style and neighborhood type on the swarm's topology, (3) demonstrate that operator workload stays stable as the size of the swarm increases, but (4) find that operator workload is influenced by the interaction style. We also present considerations for swarm deployment on real robots.

Augmented Reality (AR) head-mounted display (HMD) provides users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. The information delivered via an interactive user interface provides an innovative method to display complex building instructions, which is more intuitive and accessible compared with traditional paper documentations. However, there are still challenges hindering the practical usage of this technology at the construction jobsite. As a technical restriction, current AR HMD products have a limited field of view (FOV) compared to the human vision range. It leads to an uncertainty of how the obstructed view of display will affect construction workers' perception of hazards in their surrounding area. Similarly, the information displayed to workers requires rigorous testing and evaluation to make sure that it does not lead to information overload. Therefore, it is essential to comprehensively evaluate the impacts of using AR HMD from both perspectives of task performance and cognitive performance. This dissertation aims to bridge the gap in understanding the cognitive impacts of using AR HMD in construction assembly tasks. Specifically, it focuses on answering the following two questions: (1) How are task performance and cognitive skills affected by AR displays under complex working conditions? (2) How are moment-to-moment changes of mental workload captured and evaluated during construction assembly tasks? To answer these questions, this dissertation proposed two experiments. The first study tests two AR displays (conformal and tag-along) and paper instruction under complex working conditions, involving different framing scales and interference settings. Subjective responses are collected and analyzed to evaluate overall mental workload and situation awareness. The second study focuses on exploring an electroencephalogram (EEG) based approach for moment-to-moment capture and evaluation of mental workload. It uncovers the cognitive change on the time domain and provides room for further quantitative analyzing on mental workload. Especially, two frameworks of mental workload prediction are proposed by using (1) Long Short-Term Memory (LSTM) and (2) one-dimensional Convolutional Neural Network (1D CNN)-LSTM for forecasting EEG signal and, classifying task conditions and mental workload levels respectively. The approaches are tested to be effective and reliable for predicting and recognizing subjects' mental workload during assembly. In brief, this research contributes to the existing knowledge with an assessment of AR HMD use in construction assembly, including task performance evaluation and both subjective and physiological measurements for cognitive skills. / Doctor of Philosophy / Augmented Reality (AR) is an emerging technology that bridges the gap between virtual creatures and physical world with an immersive display experience. Today, head-mounted display (HMD) is well developed to meet the demands for portable AR devices. It provides interactive and intuitive display of 2D graphical information to make it easier to understand for users. Therefore, AR display has been studied in the past few years for a more simplified and productive construction assembly process. However, given the premise that construction is a high-risk industry, introducing such display technology to the jobsite needs to be carefully tested. One obstacle in current AR HMD products is the restriction of field of view (FOV), which may block users' view in presenting large-scale 3D objects. In construction assembly, workers need to deal with tasks in different scopes, such as wood framing for a residential house. Consequently, it is necessary to study how such technical challenge will impact workers' performance under different task conditions. Another concern comes from the mental perspective. Although AR display may bring convenience in acquiring effective information, it is difficult to measure if this generates excessive mental burden to users. Especially for construction workers, whether the overlaid display will cause distraction and information overload is crucial for protecting workers from hazards. To address the problems, this dissertation explores the gap in previous literature, where mental workload is not well studied for using AR HMD in construction assembly. Two experiments are conducted to comprehensively evaluate the impacts of AR displays on both assembly performance and users' mental status. The outcomes bring implications to theoretical and practical aspects. First, it compares two AR displays (2D tag-along image and 3D conformal model) with traditional paper documentation for assembly performance (efficiency and accuracy) and users' cognitive skills (mental workload and situation awareness). The findings revealed the impact of FOV restriction and provided a strategic solution to selecting display method for different task conditions. Second, it proposes a physiological approach to calculate mental workload from analyzing the features from brain waves. It uncovered the latent mental changes during the assembly. Furthermore, two deep learning approaches are applied to predict and classify mental workload. The prediction model depicted the trend of mental workload in eighteen seconds based on an eighty-four-second training set, while the classifier recognized two task conditions with different mental workload levels with an accuracy of 93.6%. The results have promising potential for future research in detecting and preventing abnormality in workers' mental status. In addition, it is generalizable to apply in other construction tasks and AR applications.

Assessing the pilot’s cognitive state is of increasing importance in aviation, especially for the development of adaptive assistance systems. For this purpose, the assessment of mental workload (MWL) is of special interest as an indication when and how to adapt the automation to fit the pilot’s current needs. Thus, there is a need to assess the pilot continuously, objectively and non-intrusively. Neurophysiological measurements like electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are promising candidates for such an assessment. Yet, there is evidence that EEG- and fNIRS-based MWL measures are susceptible to influences from other concepts like mental fatigue (MF), and decrease in accuracy when MWL and MF confound. Still, there are only few studies targeting this problem, and no systematic investigation into this problem has taken place. Thus, the validity of neurophysiological MWL measures is not clear yet. In order to undertake such a systematic investigation, I conducted three studies: one experiment in which I investigated the effects of increasing MWL on cortical activation when MF is controlled for; a second experiment in which I examined the effects of increasing MF on cortical activation when MWL is controlled for; and a further comparative analysis of the gathered data. In order to induce MWL and MF in a controllable and comparable fashion, I conceived and used a simplified simulated flight task with an incorporated adapted n-back and monitoring task. I used a concurrent EEG-fNIRS measurement to gain neurophysiological data, and collected performance data and self-reported MWL and MF. In the first study (N = 35), I induce different four levels of MWL by increasing the difficulty of the n-back task, and controlled for MF by means of randomization and a short task duration (≤ 45 minutes). Higher task difficulty elicited higher subjective MWL ratings, declining performance, increased frontal theta band power and decreased frontal deoxyhaemoglobin (HbR) concentration. Furthermore, fNIRS proved more sensitive to tasks with low difficulty, and EEG to tasks with high difficulty. Only the combination of both methods was able to discriminate all four induced MWL levels. Thus, frontal theta band power and HbR were sensitive to changing MWL. In the second study (N = 31), I. I induced MF by means of time on task. Thus, I prolonged the task duration to approx. 90 minutes, and controlled for MWL by using a low but constant task difficulty derived from the first experiment. Over the course of the experiment, the participants’ subjective MF increased linearly, but their performance remained stable. In the EEG data, there was an early increase and levelling in parietal alpha band power and a slower, but steady increase in frontal theta band power. The fNIRS data did not show a consistent trend in any direction with increasing MF. Thus, only parietal alpha and frontal theta band power were sensitive to changing MF. In the third study, I investigated the validity of two EEG indices commonly used for MWL assessment, the Task Load Index (TLI) and the Engagement Index (EI). I computed the indices from the data of the two experiments, and compared the results between the datasets, and to single band powers. The TLI increased with increasing MWL, but was less sensitive than theta band power alone, and varied slightly with increasing MF. The EI did not vary with MWL, and was not sensitive to gradually increasing MF. Thus, neither index could be considered a valid MWL measure. In sum, neurophysiological measures can be used to assess changes in MWL. Yet, frontal HbR was the only measure sensitive to MWL that did not also vary with MF, and further research is needed to conclude if this finding holds true under different task characteristics. Thus, the tested EEG and fNIRS measures are only valid indications of MWL when confounding effects of MF are explicitly controlled for. I discuss further influences on the tested EEG and fNIRS measures, possible combinations with other data sources, and practical challenges for a neurophysiological MWL assessment. I conclude that neurophysiological measures should be used carefully outside the laboratory, as their validity will likely suffer in realistic settings. When their limitations are understood and respected, they can help to understand the cognitive processes involved in MWL, and can be a valuable addition to an MWL assessment.

Effective mental workload measurement is critical because mental workload significantly affects human performance. A non-invasive and objective workload measurement tool is needed to overcome limitations of current mental workload measures. Further, training/learning increases mental workload during skill or knowledge acquisition, followed by a decreased mental workload, though sufficient training times are unknown. The objectives of this study were to: (1) investigate the efficacy of using thermography as a non-contact physiological measure to quantify mental workload, (2) quantify and describe the relationship between mental workload and learning/training, and, (3) introduce a method to determine a sufficient training time and an optimal human performance level for a novel task by using thermography. Three studies were conducted to address these objectives. The first study investigated the efficacy of using thermography to quantity the relationship between mental workload and facial temperature changes while learning an alpha-numeric task. Thermography measured and quantified the mental workload level successfully. Strong and significant correlations were found among thermography, performance, and subjective workload measures (MCH and SWAT ratings). The second study investigated the utility of using a psychophysical approach to determine workload levels that maximize performance on a cognitive task. The second study consisted of an adjustment session (participants adjusted their own workload levels) and work session (participants worked at the chosen workload level). Participants were found to fall into two performance groups (low and high performers by accuracy rate) and results were significantly different. Thermography demonstrated whether both group found their optimal workload level. The last study investigated efficacy of using thermography to quantify mental workload level in a complex training/learning environment. Experienced drivers’ performance data was used as criteria to indicate whether novice drivers mastered the driving skills. Strong and significant correlations were found among thermography, subjective workload measures, and performance measures in novice drivers. This study verified that thermography is a reliable and valid way to measure workload as a non-invasive and objective method. Also, thermography provided more practical results than subjective workload measures for simple and complex cognitive tasks. Thermography showed the capability to identify a sufficient training time for simple or complex cognitive tasks.

No description available.

No description available.