At least 2.6 billion people all over the world suffer from nearsightedness, among whom 312 million are under 19 years old. Just like other vision problems, uncorrected nearsightedness brings inconvenience to many human daily activities including walking. However, the influence of nearsightedness on gait patterns and gaze behaviours remains barely discovered. This project aimed to study the influence of nearsightedness on human visuomotor coordination in different environmental settings. An integrated system combining motion capture and eye-tracking was implemented for measuring gait and gaze simultaneously. Twelve participants were recruited to perform a protocol consisting of walking tasks in various visual and environmental conditions. Nine of the participants were eligible for data analysis. Gaze time distribution and gait cycle parameters were compared between participant groups (five with normal vision, four nearsighted), and among different walking tasks. Results revealed that comparing with the control group, the nearsighted participants made shorter and slower steps, as well as spent more time looking at the walking path. The walking path also affected the gait and gaze behaviours, with shorter step length and longer step time observed when the participants were walking uphill, while increased gaze attention was paid downhill. The practicality of combining gait analysis with eye-tracking was proved in this project, laying a foundation for future studies of visuomotor coordination.

Recent technological developments have ushered in an exciting era for collaborative robots (cobots), which can operate in close proximity with humans, sharing and supporting task goals. While there is increasing research on the biomechanical and ergonomic consequences of using cobots, there is relatively little work on the potential motor-cognitive demand associated with these devices. These cognitive demands primarily stem from the need to form accurate internal (mental) models of robot behavior, while also dealing with the intrinsic motor-cognitive demands of physical co-manipulation tasks, and visually monitoring the environment to ensure safe operation. The primary aim of this work was to investigate the viability of eye-tracking measures for characterizing mental workload during the use of cobots, while accounting for the potential effects of learning, task-type, expertise, and age-differences. While eye-tracking is gaining traction in surgical/rehabilitation robotics domains, systematic investigations of eye tracking for studying interactions with industrial cobots are currently lacking. We conducted three studies in which participants of different ages and expertise levels learned to perform upper- and lower-limb tasks using a dual-armed cobot and a whole-body powered exoskeleton respectively, over multiple trials. Robot-control difficulty was manipulated by changing the joint impedance on one of the robot arms (for the dual-armed cobot). The first study demonstrated that when individuals were learning to interact with a dual-armed cobot to perform an upper-limb co-manipulation task simulated in a virtual reality (VR) environment, pupil dilation (PD) and stationary gaze entropy (SGE) were the most sensitive and reliable measures of mental workload. A combination of eye-tracking measures predicted performance with greater accuracy than experimental task variables. Measures of visual attentional focus were more sensitive to task difficulty manipulations than typical eye-tracking workload measures, and PD was most sensitive to changes in workload over learning. The second study showed that compared to walking freely, walking while using a complex whole-body powered exoskeleton: a) increased PD of novices but not experts, b) led to reduced SGE in both groups and c) led to greater downward focused gaze (on the walking path) in experts compared to novices. In the third study using an upper-limb co-manipulation task similar to Study 1, we found that the PD of younger adults reduced at a faster rate over learning, compared to that of older adults, and older adults showed a significantly greater drop in gaze transition entropy with an increase in task difficulty, compared to younger adults. Also, PD was sensitive to learning and robot-difficulty but not environmental-complexity (collisions with objects in the task environment), and gaze-behavior measures were generally more sensitive to environmental-complexity. This research is the first to conduct a comprehensive analysis of mental workload in physical human-robot interaction using eye-tracking measures. PD was consistently found to show larger effects over learning, compared to task difficulty. Gaze-behavior measures quantifying visual attention towards environmental areas of interest were found to show relatively large effects of task difficulty and should continue to be explored in future research. While walking in a powered exoskeleton, both novices and experts exhibited compensatory gaze strategies. This finding highlights potentially persistent effects of using cobots on visual attention, with potential implications to safety and situational awareness. Older adults were found to apply greater mental effort (indicated by sustained PD) and followed more constrained gaze patterns in order to maintain similar levels of performance to younger adults. Perceived workload measures could not capture these age-differences, thus highlighting the advantages of eye-tracking measures. Lastly, the differential sensitivity of pupillary- and gaze behavior metrics to different types of task demands highlights the need for future research to employ both kinds of measures for evaluating pHRI. Important questions for future research are the potential sensitivity of eye-tracking workload measures over long-term adaptations to cobots, and the potential generalizability of eye-tracking measures to real-world (non-VR) tasks. / Doctor of Philosophy / Collaborative robots (cobots) are an exciting and novel technology that may be used to assist human workers in manual industrial work, reduce physical demand, and potentially enable older adults to re-enter the workforce. However, relatively little is known about the potential cognitive demands that cobots may impose on the human user. Although intended to assist humans, some cobots have been found to be difficult to use, because of the time and effort that is needed to learn their control dynamics (i.e. to learn how to physically control them to perform a complex manual task). Thus, it is important to better understand the potential mental demand/workload that a human operator may experience, while using a cobot, and how this demand may vary over time and learning to use the cobot. Eye-tracking is a promising technique to measure a cobot-operators' mental workload, since it can provide various measures that correlate with the involuntary physiological response to mental workload (e.g. pupil dilation - PD), as well as voluntary gaze strategies (e.g. the durations and patterns of where people look) in order to perform a physical/motor task. Eye-tracking measures may be used to continuously and precisely evaluate whether a cobot imposes excessive workload on the human operator, and if high workload is observed, the cobot may be programmed to adapt its behavior to reduce workload. Although eye-tracking is gaining traction in surgical/rehabilitation robotics domains, systematic investigations of eye tracking for studying interactions with industrial cobots are currently lacking. We designed three studies in which we investigated 1) the ability of eye-tracking measures to measure changes in mental workload while participants learned to use a cobot under different difficulty-levels 2) the changes in pupil diameter and gaze behavior when participants walked while wearing a whole-body powered exoskeleton as opposed to walking freely, and potential differences between novice- and expert exoskeleton-users 3) the differences in mental workload and visual attention between younger and older adults while learning to use a cobot. The first and third studies used virtual reality (VR) to simulate the task environment, to allow for precise control over the presentation of stimuli. In study 1, we found that in higher difficulty-levels, participants' pupils were significantly more dilated, i.e., participants experienced higher mental workload, than in lower-difficulty levels. Also, PD gradually reduced as participants learned to better perform the task. In difficult task-conditions, participants gazed more frequently at the robot, and showed higher randomness (entropy) in their gaze patterns. The proportion of gaze falling on certain objects was at least as sensitive an indicator of task-difficulty, as PD and gaze entropy. In study 2, we found that walking in a whole-body exoskeleton was cognitively demanding, but only for novice participants. However, both novice and expert participants showed changes in their gaze patterns while walking in the exoskeleton – both groups lowered their gaze and focused on the walking path to a greater extent, compared to walking freely. Lastly, in study 3, we also found that older adults applied greater mental effort for maintaining similar levels of performance as younger adults. Older adults also exhibited more repetitive scanning patterns compared to younger adults, when task difficulty increased. This may have been due to potential reduction in the capacity to control attention with age. Our work demonstrates that eye-tracking measures are sensitive and reliable metrics of workload, and that different metrics are sensitive to different sources of workload. Specifically, PD was sensitive to robot-difficulty, and measures of visual attention were generally more sensitive to the complexity of the task environment. Important questions for future research are the potential changes in eye-tracking workload measures over longer time periods of learning to use cobots, and how these results generalize to real-world tasks that are not performed in virtual reality.

De nombreuses situations modifient les coordinations sensorimotrices (e.g., croissance, pathologie, interfaces optiques ou mécaniques). Le système nerveux doit alors s’adapter afin de préserver la précision de ses interactions avec l’environnement. Si l’adaptation visuomotrice à une altération de la direction visuelle par prismes latéraux a été largement étudiée, les mécanismes de l’adaptation visuomotrice à une altération des distances apparentes sont en revanche peu connus. Ce travail regroupe une série d’études de psychophysique explorant les mécanismes de l’adaptation visuomotrice à une altération des distances apparentes, lors d’une exposition à court terme dans l’espace de préhension. L’altération des distances apparentes a été réalisée par modification de la vergence à l’aide de prismes à bases externes. La manipulation des retours visuels a permis de mettre en évidence trois niveaux d’adaptation. Le premier niveau concerne la modification des distances perçues liée à l’augmentation de la vergence tonique. Cette modification résulte de la potentiation musculaire extra-oculaire (EMP) induite par une convergence soutenue. Le second niveau implique la recalibration du signal altéré de distance dérivé de la vergence par les signaux proprioceptivo-moteurs du membre supérieur exposé. Le troisième niveau résulte d’une réorganisation des commandes motrices du membre exposé. Aucune adaptation proprioceptive du membre exposé n’a été retrouvée. La nature des composantes adaptatives à une altération des distances apparentes diffère de celle classiquement décrite pour l’altération de la direction visuelle impliquant essentiellement des composantes proprioceptive et motrice. La contribution des composantes adaptatives sensorielles est déterminée par la précision respective de la localisation spatiale fondée sur la vision et sur la proprioception, qui diffère en latéral et en profondeur. D’autre part, les aspects géométriques de la perception des distances fondée sur la vergence ont été explorés par comparaison de l’adaptation visuelle aux prismes à bases externes (augmentant la demande en vergence d’un angle constant pour toutes les distances de fixation) et au téléstéréoscope (multipliant la demande en vergence pour toutes les distances de fixation). Quel que soit le dispositif optique utilisé, l’adaptation visuelle a consisté en un effet consécutif ne dépendant pas de la distance d’observation, contrairement aux résultats prédits sur la base du signal de vergence. / There are numerous situations in which sensorimotor coordination is altered (e.g., growth, pathology, optical or mechanical interfaces). In such situations, the nervous system must adapt so that the organism continues to interact successfully with the environment. While visuomotor adaptation to visual direction alteration by lateral prisms has been widely studied, the mechanisms underlying visuomotor adaptation to alteration of apparent distance remain poorly known. We performed a series of psychophysical studies to explore the various components of adaptation to alteration of apparent distance. Base-out prisms were used to alter apparent distance by modifying vergence demand. By manipulating visual feedback, we were able to demonstrate three adaptation levels. The first level corresponds to changes in perceived distance related to tonic vergence. These changes result from eye muscle potentiation (EMP) induced by sustained vergence. The second level involves a recalibration of the altered distance signal derived from vergence by limb proprioceptive-motor signals. The third level results from a reorganization of motor commands of the upper limb used. No limb proprioceptive component was identified. The nature of adaptive components to apparent distance alteration differs from that described for visual direction alteration, which involves essentially proprioceptive and motor components. This difference can be attributed to differences in accuracy between proprioception and vision for localization in depth or in lateral directions. The geometrical aspects of distance perception based on vergence were also explored by comparing visual adaptation to base-out prisms (introducing an offset in vergence demand) and to a telestereoscope (multiplying vergence demand for all fixation distances). Regardless of which optic device was used, the recalibration of the relationship between the vergence signal and perceived distance consisted in a constant bias over distances.

La naissance prématurée se caractérise par des expériences multisensorielles et sensorimotrices atypiques lors d’une période du développement très sensible à ces dernières (Koenig-Zores & Kuhn, 2016). Les recherches ont mis en évidence un impact significatif de la prématurité sur le développement des enfants (Marret et al., 2015) induisant notamment des troubles perceptivo-moteurs avec ou sans troubles neurologiques associés (De Rose et al., 2013). Cependant, si beaucoup d’études portent sur la motricité des nouveau-nés et nourrissons nés prématurés, peu de données existent sur le développement perceptivo-moteur des enfants nés prématurés sans diagnostic de trouble neurodéveloppemental. La question principale abordée dans cette thèse est de savoir si la prématurité n’induit pas des signatures motrices spécifiques, même en absence de ces troubles. Dans l’objectif de répondre à cette question, différents tests standardisés, et trois tâches expérimentales impliquant la perception des équivalences intermodalitaires, la coordination visuomotrice et le contrôle postural, ont été proposés à 48 enfants nés à terme et 32 enfants nés grands prématurés sans diagnostic de trouble neurodéveloppemental et tous âgés de 8 ans. Les résultats ont mis en évidence dans les deux groupes des habiletés comparables de perception des équivalences intermodalitaires dans les modalités haptique et visuelle. Cependant, ils ont aussi révélé une efficience moindre de la coordination visuomotrice et du contrôle postural chez les enfants nés prématurés en comparaison des enfants nés à terme. Ces différences suggèrent des processus perceptivo-moteurs différents chez les enfants nés grands prématurés sans diagnostic de trouble neurodéveloppemental âgés de 8 ans. Ces résultats offrent ainsi de nouvelles perspectives cliniques, notamment d’évaluation plus fine des habiletés perceptivo-motrices, ainsi que de nouveaux questionnements de recherche sur les trajectoires développementales de ces mêmes habiletés et des réseaux neuronaux sous-jacents chez les enfants nés prématurés. / Preterm birth is associated with atypical multisensory and sensorimotor experiences during a period of development sensitive to these experiences (Koenig-Zores & Kuhn, 2016). The studies showed a significant impact of prematurity on children development (Marret et al., 2015), such as perceptual-motor disorders, with or without associated neurological disorders (De Rose et al., 2013). However, even though there are many studies on motor abilities of preterm newborns and preterm infants, little data exists on perceptual-motor development of preterm children without neurodevelopmental disorders. The main question addressed in this thesis is to know whether prematurity does induce specific motor signatures, even in the absence of neurodevelopmental disorders. To answer this question, different standardized tests and three research tasks involving perception of intersensory equivalences, visuomotor coordination, and postural control, have been proposed to 48 full-term children and 32 very preterm children without any diagnosis of neurodevelopmental disorders, all aged 8 years old. The results revealed comparable perception abilities of intersensory equivalences in haptic and visual modalities in the preterm and the full-term groups. However, they also showed a lower efficiency of visuomotor coordination and postural control in the preterm children compared to the full-term children. These observations suggest different perceptual-motor processes in very preterm children without diagnosis of neurodevelopmental disorders at the age of 8. The results offer new clinical opportunities for a more precise evaluation of perceptual-motor abilities, and new research questions about developmental trajectories of perceptual-motor abilities and underlying neuronal networks, in preterm children.