Back to search

Cornell University

PERCEPTION FOR AUTONOMOUS VEHICLES IN CHALLENGING WEATHERS AND OCCLUDED ENVIRONMENTS

Abstract

dc:description.abstract

Self-driving cars must detect other traffic participants like vehicles and pedestrians in 3D in order to plan safe routes and avoid collisions. State-of-the-art 3D object detectors, based on deep learning, have shown promising accuracy but are prone to over-fit domain idiosyncrasies, making them fail in new environments and across weather conditions--- a serious problem for the robustness of self-driving cars. Additionally, object detection in occluded cases still remains a challenge. In this dissertation, I present my work on a a novel learning approach that reduces the gap between domains by fine-tuning detectors on high-quality pseudo-labels in the target domain -- pseudo-labels that are automatically generated after driving based on replays of previously recorded driving sequences. In these replays, object tracks are smoothed forward and backward in time, and detections are interpolated and extrapolated---crucially, leveraging future information to catch hard cases such as missed detections due to occlusions or far ranges. I then, present a new dataset to enable robust autonomous driving via a novel data collection process --- data is repeatedly recorded along a 15 km route under diverse scene (urban, highway, rural, campus), weather (snow, rain, sun), time (day/night), and traffic conditions (pedestrians, cyclists and cars). The dataset includes road and object annotations using amodal masks to capture partial occlusions and 3D bounding boxes. We demonstrate the uniqueness of this dataset by analyzing the performance of baselines in amodal segmentation of road and objects. Finally, I present a novel approach that leverages repeated traversals to improve 3D object detection. I propose a simple and effective approach for combining visibility with current and past traversal LiDAR scans along with a per class past occupancy prediction task. We argue that these additional inputs and prediction tasks, add important information for detecting occluded objects.

Degree

thesis:*
Name thesis:degree_name
Ph. D., Mechanical Engineering
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Mechanical Engineering
Grantor
Cornell University
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Diaz-Ruiz, Carlos
Chair dc:contributor.chair
  • Campbell, Mark
Committee members dc:contributor.committeemember
  • Hariharan, Bharath
  • Kress Gazit, Hadas
  • Weinberger, Kilian

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en

Identifiers

dc:identifier.*
Dc Identifier Bibid
16219411
Dc Identifier Other
ProQuest Submission ID: 13958
ProQuest Publication ID: 30631340

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Diaz-Ruiz, Carlos. PERCEPTION FOR AUTONOMOUS VEHICLES IN CHALLENGING WEATHERS AND OCCLUDED ENVIRONMENTS. Doctor of Philosophy thesis, Cornell University, 2023. https://hdl.handle.net/1813/114613