While driving behavior is generally governed by the nature and the driving objectives of the driver, there are many situations (typically in crowded traffic conditions) where tacit communication between vehicle drivers and pedestrians govern driving behavior, significantly influencing transportation safety. The study aimed to formalize the tacit communication between vehicle drivers and pedestrians, in order to inform an investigation on effective communication mechanisms between autonomous vehicle and humans. Current autonomous vehicles engage in decision making primarily controlled by on-board or external sensory information, and do not explicitly consider communication with pedestrians. The study was a within subject 2x2x2 factorial experimental design. The three independent variables were driving context (normal driving vs. autonomous vehicle placard), driving route (1 vs. 2), and narration (yes vs. no). The primary outcome variable was driver-yield behavior. Each of the ten drivers completed the factorial design, requiring eight total drives. Data were collected using a data acquisition system (DAS) designed and installed on the experimental vehicle by the Virginia Tech Transportation Institute. The DAS collected video, audio, and kinematic data. Videos were coded using a proprietary software program, Hawkeye, based on an a priori data directory. Recommendations for future autonomous vehicle research and programming are provided. / Ph. D.

No description available.

Human activity recognition is one of the intensively studied areas in computer vision. Most existing works do not assume video resolution to be a problem due to general applications of interests. However, with continuous concerns about global security and emerging needs for intelligent video analysis tools, activity recognition from low-resolution and low-quality videos has become a crucial topic for further research. In this dissertation, We present a series of approaches which are developed specifically to address the related issues regarding low-level image preprocessing, single person activity recognition, and human-vehicle interaction reasoning from low-resolution surveillance videos. Human cast shadows are one of the major issues which adversely effect the performance of an activity recognition system. This is because human shadow direction varies depending on the time of the day and the date of the year. To better resolve this problem, we propose a shadow removal technique which effectively eliminates a human shadow cast from a light source of unknown direction. A multi-cue shadow descriptor is employed to characterize the distinctive properties of shadows. Our approach detects, segments, and then removes shadows. We propose two different methods to recognize single person actions and activities from low-resolution surveillance videos. The first approach adopts a joint feature histogram based representation, which is the concatenation of subspace projected gradient and optical flow features in time. However, in this problem, the use of low-resolution, coarse, pixel-level features alone limits the recognition accuracy. Therefore, in the second work, we contributed a novel mid-level descriptor, which converts an activity sequence into simultaneous temporal signals at body parts. With our representation, activities are recognized through both the local video content and the short-time spectral properties of body parts' movements. We draw the analogies between activity and speech recognition and show that our speech-like representation and recognition scheme improves recognition performance in several low-resolution datasets. To complete the research on this subject, we also tackle the challenging problem of recognizing human-vehicle interactions from low-resolution aerial videos. We present a temporal logic based approach which does not require training from event examples. At the low-level, we employ dynamic programming to perform fast model fitting between the tracked vehicle and the rendered 3-D vehicle models. At the semantic-level, given the localized event region of interest (ROI), we verify the time series of human-vehicle spatial relationships with the pre-specified event definitions in a piecewise fashion. Our framework can be generalized to recognize any type of human-vehicle interaction from aerial videos. / text

No description available.

This dissertation describes two distinctive methods for human-vehicle interaction recognition: one for ground level videos and the other for aerial videos. For ground level videos, this dissertation presents a novel methodology which is able to estimate a detailed status of a scene involving multiple humans and vehicles. The system tracks their configuration even when they are performing complex interactions with severe occlusion such as when four persons are exiting a car together. The motivation is to identify the 3-D states of vehicles (e.g. status of doors), their relations with persons, which is necessary to analyze complex human-vehicle interactions (e.g. breaking into or stealing a vehicle), and the motion of humans and car doors to detect atomic human-vehicle interactions. A probabilistic algorithm has been designed to track humans and analyze their dynamic relationships with vehicles using a dynamic context. We have focused on two ideas. One is that many simple events can be detected based on a low-level analysis, and these detected events must contextually meet with human/vehicle status tracking results. The other is that the motion clue interferes with states in the current and future frames, and analyzing the motion is critical to detect such simple events. Our approach updates the probability of a person (or a vehicle) having a particular state based on these basic observed events. The probabilistic inference is made for the tracking process to match event-based evidence and motion-based evidence. For aerial videos, the object resolution is low, the visual cues are vague, and the detection and tracking of objects is less reliable as a consequence. Any method that requires accurate tracking of objects or the exact matching of event definition are better avoided. To address these issues, we present a temporal logic based approach which does not require training from event examples. At the low-level, we employ dynamic programming to perform fast model fitting between the tracked vehicle and the rendered 3-D vehicle models. At the semantic-level, given the localized event region of interest (ROI), we verify the time series of human-vehicle relationships with the pre-specified event definitions in a piecewise fashion. With special interest in recognizing a person getting into and out of a vehicle, we have tested our method on a subset of the VIRAT Aerial Video dataset and achieved superior results. / text

Cette thèse présente une méthode de navigation autonome d’un véhicule routier robotisé dans un contexte de l’interaction conducteur - véhicule, dans lequel le conducteur humain et le système de navigation autonome coopèrent dans le but d’associer les avantages du contrôle manuel et automatique. La navigation du véhicule est réalisée en parallèle par le conducteur humain et le système de conduite automatique, basée sur la perception de l’environnement. La navigation coopérative est basée sur l’analyse et correction des gestes du conducteur humain par le système intelligent, dans le but d’exécuter une tâche de navigation locale qui concerne le suivie de voie avec évitement d’obstacles. L’algorithme d’interaction humain-véhicule est basé sur des composants de navigation référencée capteurs formés par des contrôleurs d’asservissement visuel (VS) et la méthode d’évitement d’obstacle « Dynamic Window Approach (DWA) » basée sur la grilles d’occupation. Ces méthodes prennent en entrée la perception de l’environnement fournie par des capteurs embarqués comprenant un système monovision et un LIDAR. Dû à des impossibilités techniques/légales, nous n’avons pas pu valider nos méthodes sur notre véhicule robotisé (une Renault Zoé robotisée), ainsi nous avons construit des structures « driver-in-theloop » dans des environnements de simulation Matlab et SCANeRTM Studio. En Matlab, le conducteur humain est modélisé par un algorithme appelé « Human Driver Behaviour controller (HDB) », lequel génère des gestes de conduite dangereux dans la partie manuelle de l’entrée de commande du système coopératif. En SCANeR Studio, la sortie de l’HDB est remplacée par des commandes manuelles générées directement par un conducteur humain dans l’interface utilisateur du simulateur. Des résultats de validation dans les deux environnements de simulation montrent la faisabilité et la performance du système de navigation coopérative par rapport aux tâches de suivie de voie, l’évitement d’obstacles et le maintien d’une distance de sécurité. / This thesis presents an approach of cooperative navigation control pattern for intelligent vehicles in the context of human-vehicle interaction, in which human driver and autonomous servoing system cooperate for the purpose of benefiting from mutual advantages of manual and auto control. The navigation of the vehicle is performed in parallel by the driver and the embedded intelligent system, based on the perception of the environment. The cooperative framework we specify concerns the analysis and correction of the human navigation gestures by the intelligent system for the purpose of performing local navigation tasks of road lane following with obstacle avoidance. The human-vehicle interaction algorithm is based on autonomous servoing components as Visual Servoing (VS) controllers and obstacle avoidance method Dynamic Window Approach (DWA) based on Occupancy Grid, which are supported by the environment perception performed carried out by on-boarded sensors including a monovision camera and a LIDAR sensor. Given the technical/legal impossibility of validating our interaction method on our robotic vehicle (a robotic Renault Zoé), the driver-in-the-loop structures of system are designed for simulative environment of both Matlab and SCANeRTM Studio. In Matlab environment human driver is modeled by a code-based Human Driver Behaviour (HDB) Controller, which generates potential dangerous behaviors on purpose as manual control of the cooperative system. In SCANeR Studio environment the HDB is replaced by real-time manual command (a real human driver) via driving interface of this simulator. Results of simulative validation show the feasibility and performance of the cooperative navigation system with respect to tasks of driving security including road lane following, obstacle avoidance and safe distance maintenance.