Flying fruit flies (Drosophila melanogaster) locate a concealed appetitive odour source most accurately in environments containing vertical visual contrasts (Frye et al, 2003). To investigate how visuomotor and olfactory responses interact to cause this phenomenon, I implement a tracking system capable of recording flies’ flight trajectories in three dimensions. I examine free-flight behaviour in three different visual environments, with and without food odour present. While odour localisation is facilitated by a random chequerboard pattern compared to a horizontally striped one, a single vertical landmark also facilitates odour localisation, but only if the odour source is situated close to the landmark. I implement a closed-loop systems-level model of visuomotor control consisting of three parallel subsystems which use wide-field optic flow cues to control flight behaviour. These are: an optomotor response to stabilise the model fly’s yaw orientation; a collision avoidance system to initiate rapid turns (saccades) away from looming obstacles; and a speed regulation system. This model reproduces in simulation many of the behaviours I observe in flies, including distinctive visually mediated ‘rebound’ turns following saccades. Using recordings of real odour plumes, I simulate the presence of an odorant in the arena, and investigate ways in which the olfactory input could modulate visuomotor control. In accordance with the principle of Occam’s razor, I identify the simplest mechanism of crossmodal integration that reproduces the observed pattern of visual effects on the odour localisation behaviour of flies. The resulting model uses the change in odour intensity to regulate the sensitivity of collision avoidance, resulting in visually mediated chemokinesis. Additionally, it is necessary to amplify the optomotor response whenever odour is present, increasing the model fly’s tendency to steer towards features of the visual environment. This could be viewed as a change in behavioural context brought about by the possibility of feeding. A novel heterogeneous visual environment is used to validate the model. While its predictions are largely borne out by experimental data, it fails to account for a pronounced odour-dependent attraction to regions of exclusively vertical contrast. I conclude that visual and olfactory responses of Drosophila are not independent, but that relatively simple interaction between these modalities can account for the observed visual dependence of odour source localisation.

The research reported in the present explored the interaction between perception and action, focusing on how this might occur under memory conditions. This was done in line with the proposals of grounded cognition and situated action, in which action and perception are tightly linked, and able to influence one another in order to aid the performance of a task. Following this idea of a bi-directional loop between the two processes, studies were conducted which focused on each side of this relationship, in conditions where memory would be necessary. The first experiments investigated how the perception of objects and the memory of those perceived objects could influence the production of actions. Later studies examined how the preparation and performance of actions could affect the perception of a scene, and subsequent recall of the objects presented. Throughout these studies, object properties (e.g., shape, colour, position) were used as a means to either manipulate or measure the effect of the tasks. The findings of the studies suggested that weighting an off-line memory representation by means of the task setting was possible, but that this was not an automatic occurrence. Based on the results obtained, it seemed that there were conditions which would affect whether memories could be tailored to the current demands of the tasks, and that these conditions were linked to the realism of the situation. Factors such as the task complexity, the potential for object interaction and the immersive environment were all suggested as possible contributors to the construction and use of weighted representations. Overall, the studies conducted suggest that memory can play a role in guiding action, as on-line perception does, so long as the situation makes it clear that this is necessary. If such weightings are useful, then the memory will be constructed accordingly. However, if the situation is such that there is no clear task, then the memory representations will remain unaffected and unprepared for one specific action, or not be used to aid action. Memory can be seen as serving action, but our memory systems are flexible, allowing us to cope with the demands and restrictions of particular situations.

Two experiments were conducted in an attempt to further the understanding of how previously identified intrinsic constraints and perceptual factors interact in influencing the learning and performance of various bimanual coordination patterns. The purpose of Experiment 1 was to determine the influence of Lissajous feedback on 1:1 bimanual coordination patterns (0°, 90°, 180° phase lags) when the movement amplitudes of the two limbs were different. Participants coordinated rhythmic movements of their forearms while being provided separate feedback for each limb (no- Lissajous group) or integrated feedback (Lissajous group). Data from Experiment 1 supports the notion that the lead-lag relationship as well as amplitude assimilation between limbs observed in the literature can be partially attributed to the visualperceptual factors present in the testing environment. When participants are provided integrated feedback in the form of Lissajous plots and templates much of the lead-lag and amplitude assimilation effects were eliminated and relative phase error and variability were also greatly reduced after only 3 min of practice under each condition. Results from recent experiments suggest that when the salient visual information (Lissajous feedback) is removed, performance in bimanual coordination tasks rapidly deteriorates. The purpose of Experiment 2 was to determine if reducing the frequency of feedback presentation will decrease the reliance on the feedback and will facilitate the development of an internal representation that will improve performance when visual feedback is removed. Participants receiving reduced frequency feedback presentation were able to perform a delayed retention test with the feedback removed as well as the test with feedback present. Data from Experiment 2 demonstrates that salient extrinsic Lissajous feedback can effectively be combined with reduced frequency feedback presentation in a way that performance levels, when tested without the availability of feedback, match those obtained when tested in the presence of Lissajous feedback. Taken together the present experiments add to the growing literature that supports the notion that salient perceptual information can override some aspects of the system's intrinsic dynamics typically linked to motor output control. The strong tendencies toward the intrinsic dynamics found in numerous previous bimanual movement studies and the difficulties in producing various coordination patterns may actually represent detrimental effects attributable to the perceptual information available in the environment and the attentional focus participants adopt. Given external integrated salient visual information participants can essentially tune-in and learn difficult bimanual coordination patterns with relatively little practice.

The way we act upon the world influences our visual perception of it. For instance, previous work has found visual perceptual enhancement at the targets of upcoming saccades (e.g. Deubel & Schneider, 1996) and pointing movements (e.g. Deubel, Schneider, & Paprotta, 1998). Visual perceptual enhancement has also been found along the trajectories of manual movements (Festman, Adam, Pratt, & Fischer, 2013a, 2013b), but the area surrounding static hands have also been found to receive perceptual enhancement (Reed, Grubb, & Steele, 2006). The initial question addressed by the present thesis was whether the preparation of a manual movement would also induce perceptual enhancement at the effector location (i.e. the movement’s start point). In other words, do people not only attend where they are going, but also where they are coming from? To address this question, the novel aspect of our task was that participants not only had to select the movement target, but also the moving hand. Across the eight experiments of the present thesis we applied variations on a popular experimental task, asking our participants to conduct pointing movements and studying how this influenced their allocation of visuospatial attention. This was measured by recording whether they could successfully identify a discrimination target (DT), with the discrimination rates at different locations taken to index the amount of attention allocated there. Our first four experiments found evidence for enhancement at the starting point of a movement, but this effect was inconsistent and appeared to compete with other mechanisms for orienting attention. For example, our first experiment found enhancement only at the target location, which may have been induced by having used predictable locations for the DT, whereas Experiment 4 found only an enhancement at both hands, static and responding, which might have been due to the ability to plan the movements in advance. Since in each trial in Experiment 4 participants had to execute one movement out of only two possibilities, this may have allowed them to pre-program both movements before each trial and execute them from memory. In Experiment 5 we increased the number of potential movement targets in order to increase the difficulty of target selection and reduce movement predictability, while also lowering the DT presentation times. Under this more challenging paradigm we found perceptual enhancement only at the movement target, but also that the perceptual task was too difficult for half of our participants. Did we fail to induce enhancement at multiple locations because of the specific task our participants were executing, or due to a general inability to do so within this more challenging version of the experimental paradigm? To address this question we decided to test whether we could still induce perceptual enhancement at two locations within these experimental parameters, but attempting to replicate the work of Baldauf, Wolf and Deubel (2006). They had reported that carrying out pointing sequences resulted in parallel allocation of attention to all movement targets before movement onset. We repeatedly failed to find any enhancement at any location, even when we increased the presentation times to durations used in the earlier experiments. In our final experiment we mounted a more direct replication of Baldauf et al (2006), and we also conducted a preliminary calibration stage in which we attempted to adjust the DT’s presentation time to each participant’s level of ability by assessing their perceptual performance. This calibration was only successful in a third of our participants, with the majority still finding the perceptual task too challenging within the range of exposure times used. Furthermore, even amongst those participants selected for their good perceptual performance in the calibration task, we found visual enhancement only at the first movement target during the two-step pointing sequence. This calls into question not only the general replicability of the work of Baldauf et al. (2006), but also Deubel et al. (1998). On the whole, our findings suggest that although the pattern of attentional allocation is influenced by action planning, including the starting point of a movement, this is but one of many competing factors. Furthermore they call into question the general replicability of previous high-profile results, and call for a greater acknowledgement and investigation of the possible role of extensive practice in yielding some of the results found in the literature. The thesis concludes with suggestions for future work.

La théorie dominante suggérant une séparation entre une vision consciente pour la perception et une vision non consciente pour l'action au sein du système visuel fut particulièrement discutée au cours des dernières décennies. La thèse défendue ici se positionne dans le cadre d'une approche alternative reposant sur une évaluation conjointe des réponses motrice et perceptive en présence de stimuli au seuil perceptif. Les travaux réalisés dans ce contexte ont initialement porté sur les temps de réaction et ont contribué au développement d'un modèle psychophysique postulant que les réponses perceptives et motrices dépendraient d'un signal sensoriel unique mais de seuils de décision spécifiques. Les trois études réalisées au cours de cette thèse ont permis de tester ces propositions à partir de travaux portant sur le contrôle moteur manuel. Tout en confirmant une forte association entre les traitements visuels pour la perception et pour l'action, ces travaux ont permis de mettre en évidence un rôle important des conditions expérimentales (par exemple le contraste des stimuli et la présence ou non d'un masque visuel) dans les résultats obtenus. Les résultats se sont révélés compatibles avec les modèles neurophysiologiques du masquage visuel qui postulent que la réponse neurale à une stimulation visuelle est constituée d'une vague d'activation feedforward associée à la présence physique de la stimulation et de boucles de rétroaction liées à la perception consciente de celle-ci. Nos travaux ont également permis d'adapter au contrôle moteur manuel le modèle de décision à signal unique initialement développé dans le cadre des études portant sur les temps de réaction. / The dominant position of a separation between a conscious vision for perception and an unconscious vision for action within the visual system has been particularly discussed in the last decades. The present dissertation is to be placed in the context of an alternative approach consisting in evaluating jointly both perceptual and motor responses in the presence of near-treshold visual stimuli. Previous work within this framework, which have mainly concerned reaction times, have contributed to develop a psychophysical model in which perceptual and motor decision are taken relatively to the same single incoming signal but are based on different tresholds. The three studies conducting during this PhD aimed to testing these proposals in experiments involving manual motor control. While confirming the strong link between perceptual and motor processing within the visual system, the results obtained in these studies underlined the importance of experimental parameters such as the contrast of the stimuli and the presence of visual masks. These results are congruent with neurophysiological models of visual masking, which postulate that the neural response to a visual stimulus is composed of a transient feedforward sweep of activation related to the presence of as stimulus and recurrent feedback loops linked to the conscious perception of this stimulus. Our work also led to the adaptation to manual motor control of the single signal decisional model initially developed on the basis of reaction time studies.

Ce travail est issu de plusieurs constats. Premièrement, il est admis aujourd’hui que la capacité à percevoir et à retirer les informations visuelles les plus pertinentes d’une action adverse est un facteur déterminant de la performance sportive et ce d'autant plus si l'action est incertaine et faussée. Le second constat, découlant du premier, est que la tromperie est mise en jeu dans bon nombre d'interactions humaines, mais qu'elle est particulièrement déterminante et prépondérante dans lecadre des stratégies sportives. Enfin, le dernier constat est que l'exploration de la performance sportive, pour des raisons méthodologiques ou théoriques, ne permet pas dans certains cas d'expliquer pourquoi un expert est meilleur qu'un novice.L’objectif de notre travail est de comprendre ces déterminants (perceptivo-moteurs) de l'expertise lors d’un duel 1 vs. 1 en rugby présentant une tentative de feinte de l’attaquant. Nous résumons ce travail en trois questions concrètes auxquelles nous répondons: Comment réaliser une feinte de corps en rugby? Comment la détecter? Comment y faire face? La première étude s'attache à analyser les différences biomécaniques observables entre un mouvement de simple changement de direction et un mouvement de feinte de corps en rugby. La seconde étude, en immersion virtuelle, met en lien les informations prospectives émanant des mouvements de feinte et la prise de décision du défenseur de rugby (expert ou novice). Enfin, en se basant sur une méthodologie quasi similaire, la troisième étude vient immerger les participants dans une situation complète de défense, nous permettant d'explorer leur stratégie perceptivo-motrice. L'ensemble de ce processus nous permet de témoigner de l'effet de l'expertise mais surtout d'expliquer pourquoi l'expert est meilleur en dressant les différences de stratégie perceptivo-motrice le caractérisant. Si les implications concernent majoritairement le domaine du rugby (connaissances fondamentales et entrainement), elles présentent au sens plus large, un éclairage justifié et démontré des processus mis en jeu lors de toute interaction, humaine ou animal, sollicitant la tromperie. Nous en sommes persuadés; un prédateur animal, un joueur de poker, un défenseur de basket... se doivent deprésenter des stratégies quasi-similaires pour détecter l'intention de bluff chez l'opposant / This work comes from several observations. First, it is now accepted that the ability to pick up and attend to the most relevant visual information from an opponent action is a determinant of performance and even more so if the action is uncertain and deceptive. The second observation, deriving from the first, is that deception is put into play in many human interactions, and it is particularly crucial in sports strategies. Finally the last observation is that exploration of athletic performance, for methodological or theoretical reasons, can’t explain, in certain cases, why an expert is better than a novice.The aim of our work is to understand these determinants (perception and action) of expertise in a 1 vs. 1 duel in rugby, presenting an attempt of deceptive movement by the attacker. We summarize this work in three specific questions we answer: How to make a deceptive movement? How to detect a deceptive movement? How to answer to a deceptive movement? The first study aims to analyse the biomechanical differences between a movement with a simple changeof direction and a deceptive movement in rugby. The second study, using virtual reality, links the prospective information of the attacker’s movement and the decision making of the rugby defender (expert or novice). Finally, based on a quasi-similar methodology, the third study immerses the participants in a situation requiring a complete action of defence, allowing us to explore the perception and action strategies of the participants. This whole process allows us to demonstrate the effectof expertise but also to explain why experts are better by drawing up the differences characterizing them in terms of perception and action strategies. If the implications mainly concern the field of rugby (basic knowledge and training), they are also of great interest for any interaction, human or animal, involving deception. We are convinced, a predatory animal, apoker player, a defender of basketball ... must present similar strategies to detect the deceptive intention of an opponent

Ce travail a pour objet d’étude l’émergence d’effets d’affordance dans la schizophrénie. En effet, les troubles de l’action, observés chez les patients schizophrènes, pourraient être la conséquence d'un déficit de l’intégration sensorimotrice, ne permettant pas le couplage entre perception et action. En lien avec les théories de la cognition incarnée et située, nous avons fait l’hypothèse que l’enrichissement de la situation puisse permette l’émergence de ce lien dans cette population. Pour ce faire, nous avons décliné 5 expériences qui utilisent le paradigme SRC (Stimulus-Response-Compatibility). L’expérience 2 se focalise sur l’intégration visuo-spatiale, tandis que les 4 autres portent sur l’effet de potentialisation des actions lors de la perception d’objets manipulables (Tucker & Ellis, 1998), selon différentes conditions: sans amorçage (expérience 1), avec amorçage renforçant le sentiment de propriété des objets (expérience 3) ou les buts de l’action (expérience 4) et avec amorçage moteur (expérience 5). Nos résultats mettent en évidence l’émergence d’un effet de compatibilité visuo-spatiale (expérience 2) et sensorimotrice (expérience 1), dans le groupe de témoins. Toutefois, cet effet n’est pas retrouvé lorsque l’amorçage est incongruent avec les objets présentés (expériences 3 et 4) ou lorsqu’il implique un amorçage moteur (expérience 5). Dans le groupe de patients schizophrènes, bien que l’effet de compatibilité visuo-spatiale soit mis en évidence (expérience 2), l’effet de compatibilité sensorimotrice n’est retrouvé que lorsque la perception des objets est précédée d’une préparation motrice (expérience 5). Ces résultats sont discutés à la lueur des connaissances issues des théories de la cognition incarnée et située, et des travaux centrés sur les troubles de l’action retrouvés dans la schizophrénie. / This thesis studies the emergence of affordance effects in schizophrenia. Indeed, the disorders of action present in schizophrenic patients might be the consequence of deficit in sensorimotor integration, making difficult the link between perception and action. Related to the embodied cognition framework, we hypothesized that the enhancement of context would allow the emergence of this link in this population. Then, we planned 5 experiments using the SRC paradigm (Stimulus-Response-Compatibility). Experiment 2 emphasizes visuo-spatial integration. The four others focus on the effect of potentiation of actions during perception of handling objects (Tucker & Ellis, 1998), depending on different experimental conditions: without priming (experiment 1), with priming reinforcing feeling of being owner of objects (experiment 3) or reinforcing the aims of action (experiment 4) and with motor priming (experiment 5). In the control group, our results show emergence of visuo-spatial (experiment 2) and sensorimotor (experiment 1) compatibility effects. However, this effect is not present when priming is not congruent with presented objects (experiment 3 and 4) or in case of motor priming (experiment 5). In patients, despite of the presence of visuo-spatial compatibility effect (experiment 2), the sensorimotor compatibility effect is present only when perception of objects is preceded by a motor preparation (experiment 5). These results are discussed in reference to the theories of embodiment, and from studies focused on disturbances of action in schizophrenia.

Le principal objectif de ce travail de thèse est de valider l’hypothèse générale selon laquelle les conducteurs automobiles sélectionnent un mode d’action et le régulent en percevant des possibilités de dépassement, appelées affordances. À travers trois études réalisées sur simulateur de conduite, nous avons montré tout d’abord que les conducteurs prennent la décision d’initier un dépassement en fonction d’une affordance définie par le ratio entre les propriétés du système agent-environnement (la vitesse minimale satisfaisante pour réussir un dépassement (MSV)) et la propriété de l’agent (vitesse maximale du véhicule conduit (Vmax)). Nous avons ensuite mis en évidence que la décision d’initier un dépassement dépendait majoritairement de l’affordance de haut niveau MSA/Amax, définie par le ratio entre une propriété du système agent-environnement (accélération minimale satisfaisante pour réussir un dépassement (MSA)) et la propriété de l’agent (accélération maximale du véhicule conduit (Amax)). Enfin, nous avons démontré la capacité des conducteurs à exploiter des possibilités de dépassement définies par le ratio entre les propriétés du système agent-environnement (MSA) et les propriétés de l’agent (accélération maximale du véhicule conduit en 4ème (Amax 4ème) et accélération maximale du véhicule conduit en 3ème (Amax 3ème)). La formalisation de ces affordances nous a ainsi permis d’élargir le domaine d’application de l’affordance-based control (Fajen, 2007a) à des tâches de dépassement très proches de situations réelles. De futures recherches permettront d’identifier les invariants optiques sous-jacents au dépassement et d’étudier les modalités de leur prise en compte. / The main objective of this thesis is to validate the general hypothesis that car drivers select a mode of action and regulate it by perceiving overtaking possibilities called affordances. Through three experiments carried out on driving simulator, we firstly showed that car drivers take the decision to initiate an overtaking on the basis of an affordance defined by the ratio between the properties of the agent-environment system (minimum satisfactory velocity to succeed an overtaking (MSV)) and the agent property (maximum speed of the vehicle driven (Vmax)). Secondly, we have highlighted that the decision to initiate an overtaking depended mainly on a high order affordance, the MSA/Amax, defined by the ratio between a property of the agent-environment system (minimum satisfactory acceleration to succeed an overtaking (MSA)) and the property of the agent (maximum acceleration of the vehicle driven (Amax)). Thirdly, we showed that drivers are able to exploit the overtaking opportunities defined by the ratio between the properties of the agent-environment system (MSA) and the agent’s properties (maximum acceleration of the vehicle driven in 4th gear (Amax 4th) and maximum acceleration of the vehicle driven in 3rd gear (Amax 3rd)). The formalization of these possibilities for action allowed us to expand the scope of the affordance-based control (Fajen, 2007a) to the overtaking tasks which are close of situations encountered in daily life and to take a new look at this kind of tasks. Future research will be needed to identify the optical invariant underlying the overtaking and to study how they are taken into account, especially when a competition between affordances occurs.

As a pedestrian at the roadside, the two most informative cues as to the distance and rate of closure of a vehicle are its optical size and the rate of expansion of the optical image. In addition, the time to arrival of an approaching vehicle can be perceptually estimated by the ratio of these two variables, referred to as tau (Lee, 1976). Sensitivity to optic expansion is critical for collision avoidance and was measured in populations of adults, typically developing children, and in children with Developmental Coordination Disorder (DCD), an idiopathic condition characterised by marked impairments in motor coordination that negatively impact on activities of daily living. A central tendency was found in adults (n = 193) between 18 to 59 years of age to make significant errors in judging the approach rates of two vehicles. Inflated errors were observed in children (n = 136) between 6 to 17 years of age, with decreased sensitivity in the youngest age group (6 to 11 years). Furthermore, a significant decrement was found in children (n = 9) with DCD between 6 to 11 years of age. Across all groups, a systematic vehicle size bias was found, whereby faster small vehicles were perceived as travelling slower than larger vehicles. This pattern of results suggest that in general, observers are not utilising tau in judgments of relative approach rates for speeds typically encountered at the roadside, but instead rely on optical expansion that does not compensate for image size. Errors due to a reliance on optic size were inflated in children with DCD, potentially placing them at significantly greater risk at the roadside. To examine the decreased sensitivity observed in DCD, thresholds for detecting visual looming were measured in children (n = 11) with DCD between 6 to 11 years of age. A significant deficit was found when vehicles were presented in perifoveal vision, whereby children with DCD may perceive vehicles that are 5 seconds away as stationary if they are travelling any faster than ~14 mph. This demonstration of a low-level visual processing deficit could suggest an immaturity in the dorsal stream network and explain some of the difficulties that characterise DCD. Critically, perceptual judgments at the roadside are inextricably linked to the motoric capability of the observer. If a pedestrians crossing time is greater than the time available, collision will occur. Crossing gap thresholds were measured and compared to walking times for a single vehicle approaching at varying speeds. Children (n = 9) with DCD between 6 to 11 years of age left considerably longer temporal crossing gaps than their action capabilities necessitated. However, when children with DCD were presented with multiple vehicles in a virtual reality environment, they accepted crossing gaps at all approach speeds that were shorter than the time it would take them to cross. This suggests that children with DCD may not have the perceptual accuracy to predict their required action gaps in a road crossing situation. One explanation for these findings could be a difference in DCD in how vision is dynamically allocated to facilitate the preparation of goal-directed actions. Dynamic allocation of visual attention was assessed in a series of experiments that measured eye movement latencies and hand movement accuracy in children (n = 5) with DCD between 6 to 11 years of age. Both measures were found to be comparable in DCD with their typically developing peers regardless of task complexity, indicating that the allocation of visual attention is not deficient in children with DCD. The prospective control of movement in our everyday lives is critically depended on estimating the immediacy of approaching objects. Combined, these results indicate that children with DCD may be particularly vulnerable at the roadside due to a visual motion processing deficit, consistent with atypical function across broad neural structures such as the dorsal stream.

This thesis investigates the classification of the behaviour of multiple persons when viewed from a video camera. Work upon a constrained case of multiple person interaction in the form of team games is investigated. A comparison between attempting to model individual features using a (hierarchical dynamic model) and modelling the team as a whole (using a support vector machine) is given. It is shown that for team games such as handball it is preferable to model the whole team. In such instances correct classification performance of over 80% are attained. A more general case of interaction is then considered. Classification of interacting people in a surveillance situation over several datasets is then investigated. We introduce a new feature set and compare several methods with the previous best published method (Oliver 2000) and demonstrate an improvement in performance. Classification rates of over 95% on real video data sequences are demonstrated. An investigation into how the length of time a sequence is observed is then performed. This results in an improved classifier (of over 2%) which uses a class dependent window size. The question of detecting pre/post and actual fighting situations is then addressed. A hierarchical AdaBoost classifier is used to demonstrate the ability to classify such situations. It is demonstrated that such an approach can classify 91% of fighting situations correctly.