{"id":{"repo_id":"msu","oai_identifier":"oai:d.lib.msu.edu:etd_50699"},"canonical_url":"https://search.ndltd.org/etd/msu/oai:d.lib.msu.edu:etd_50699","repository":{"repo_id":"msu","name":"Michigan State University","base_url":"https://d.lib.msu.edu/oai"},"display":{"title":"3D object detection and tracking for autonomous vehicles","abstract":"Autonomous driving systems require accurate 3D object detection and tracking to achieve reliable path planning and navigation. For object detection, there have been significant advances in neural networks for single-modality approaches. However, it has been surprisingly difficult to train networks to use multiple modalities in a way that demonstrates gain over single-modality networks. In this dissertation, we first propose three networks for Camera-LiDAR and Camera-Radar fusion. For Camera-LiDAR fusion, CLOCs (Camera-LiDAR Object Candidates fusion) and Fast-CLOCs are presented. CLOCs fusion provides a multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate 3D detection results. Fast-CLOCs can run in near real-time with less computational requirements compared to CLOCs. Fast-CLOCs eliminates the separate heavy 2D detector, and instead uses a 3D detector-cued 2D image detector (3D-Q-2D) to reduce memory and computation. For Camera-Radar fusion, we propose TransCAR, a Transformer-based Camera-And-Radar fusion solution for 3D object detection. The cross-attention layer within the transformer decoder can adaptively learn the soft-association between the radar features and vision queries instead of hard-association based on sensor calibration only. Then, we propose to solve the 3D multiple object tracking (MOT) problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M3). In particular, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M3 filter can naturally model these uncertainties accurately and elegantly. We combine learning-based detections with our RFS-M3 tracker by incorporating the detection confidence score into the PMBM prediction and update step. We have evaluated our CLOCs, Fast-CLOCs and TransCAR fusion-based 3D detector and RFS-M3 3D tracker using challenging datasets including KITTI, nuScenes, Argoverse and Waymo that are released by academia and industry leaders. Superior experimental results demonstrated the effectiveness of the proposed approaches.","abstract_html":"Autonomous driving systems require accurate 3D object detection and tracking to achieve reliable path planning and navigation. For object detection, there have been significant advances in neural networks for single-modality approaches. However, it has been surprisingly difficult to train networks to use multiple modalities in a way that demonstrates gain over single-modality networks. In this dissertation, we first propose three networks for Camera-LiDAR and Camera-Radar fusion. For Camera-LiDAR fusion, CLOCs (Camera-LiDAR Object Candidates fusion) and Fast-CLOCs are presented. CLOCs fusion provides a multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate 3D detection results. Fast-CLOCs can run in near real-time with less computational requirements compared to CLOCs. Fast-CLOCs eliminates the separate heavy 2D detector, and instead uses a 3D detector-cued 2D image detector (3D-Q-2D) to reduce memory and computation. For Camera-Radar fusion, we propose TransCAR, a Transformer-based Camera-And-Radar fusion solution for 3D object detection. The cross-attention layer within the transformer decoder can adaptively learn the soft-association between the radar features and vision queries instead of hard-association based on sensor calibration only. Then, we propose to solve the 3D multiple object tracking (MOT) problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M3). In particular, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M3 filter can naturally model these uncertainties accurately and elegantly. We combine learning-based detections with our RFS-M3 tracker by incorporating the detection confidence score into the PMBM prediction and update step. We have evaluated our CLOCs, Fast-CLOCs and TransCAR fusion-based 3D detector and RFS-M3 3D tracker using challenging datasets including KITTI, nuScenes, Argoverse and Waymo that are released by academia and industry leaders. Superior experimental results demonstrated the effectiveness of the proposed approaches.","abstract_has_math":false,"creators":["Pang, Su"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":["Radha, Hayder HR","Morris, Daniel DM","Tan, Xiaobo XT","Zockaie, Ali AZ"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022","date_published":"2022","updated_at":"2026-11-18T18:57:07Z","subjects":["Electrical engineering","Automated vehicles","Optoelectronic devices","Computer vision","Automatic tracking"],"languages":["English"],"rights":["Attribution-NonCommercial 4.0 International"],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["etd:50699","isbn:9798841789024","oclc:on1373601595","oclc:1373601595","umi:29327697","local:Pang_grad.msu_0128D_19357"],"render_values":[{"text":"etd:50699","href":null,"code":true},{"text":"isbn:9798841789024","href":null,"code":true},{"text":"oclc:on1373601595","href":null,"code":true},{"text":"oclc:1373601595","href":null,"code":true},{"text":"umi:29327697","href":null,"code":true},{"text":"local:Pang_grad.msu_0128D_19357","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/doi:10.25335/nt7t-8z08","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/doi:10.25335/nt7t-8z08","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://d.lib.msu.edu/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Ad.lib.msu.edu%3Aetd_50699","prefix":"oai_dc"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Radha, Hayder HR","Morris, Daniel DM","Tan, Xiaobo XT","Zockaie, Ali AZ"]},{"key":"dc:creator","label":"Author","values":["Pang, Su"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022"]},{"key":"dc:relation","label":"Dc Relation","values":["Electronic Theses & Dissertations"]},{"key":"dc:type","label":"Dc Type","values":["Text","Theses"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Electrical engineering","Automated vehicles","Optoelectronic devices","Computer vision","Automatic tracking"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["English"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution-NonCommercial 4.0 International"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["etd:50699","isbn:9798841789024","oclc:on1373601595","oclc:1373601595","umi:29327697","local:Pang_grad.msu_0128D_19357","https://doi.org/doi:10.25335/nt7t-8z08"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Autonomous driving systems require accurate 3D object detection and tracking to achieve reliable path planning and navigation. For object detection, there have been significant advances in neural networks for single-modality approaches. However, it has been surprisingly difficult to train networks to use multiple modalities in a way that demonstrates gain over single-modality networks. In this dissertation, we first propose three networks for Camera-LiDAR and Camera-Radar fusion. For Camera-LiDAR fusion, CLOCs (Camera-LiDAR Object Candidates fusion) and Fast-CLOCs are presented. CLOCs fusion provides a multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate 3D detection results. Fast-CLOCs can run in near real-time with less computational requirements compared to CLOCs. Fast-CLOCs eliminates the separate heavy 2D detector, and instead uses a 3D detector-cued 2D image detector (3D-Q-2D) to reduce memory and computation. For Camera-Radar fusion, we propose TransCAR, a Transformer-based Camera-And-Radar fusion solution for 3D object detection. The cross-attention layer within the transformer decoder can adaptively learn the soft-association between the radar features and vision queries instead of hard-association based on sensor calibration only. Then, we propose to solve the 3D multiple object tracking (MOT) problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M3). In particular, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M3 filter can naturally model these uncertainties accurately and elegantly. We combine learning-based detections with our RFS-M3 tracker by incorporating the detection confidence score into the PMBM prediction and update step. We have evaluated our CLOCs, Fast-CLOCs and TransCAR fusion-based 3D detector and RFS-M3 3D tracker using challenging datasets including KITTI, nuScenes, Argoverse and Waymo that are released by academia and industry leaders. Superior experimental results demonstrated the effectiveness of the proposed approaches.","Thesis (Ph. D.)--Michigan State University. Electrical Engineering, 2022","Includes bibliographical references (pages 113-126)"]},{"key":"dc:format","label":"Dc Format","values":["xviii, 126 pages","application/pdf"]},{"key":"dc:title","label":"Title","values":["3D object detection and tracking for autonomous vehicles"]}]}],"canonical_facts":{"dc:contributor":["Radha, Hayder HR","Morris, Daniel DM","Tan, Xiaobo XT","Zockaie, Ali AZ"],"dc:creator":["Pang, Su"],"dc:date":["2022"],"dc:description":["Autonomous driving systems require accurate 3D object detection and tracking to achieve reliable path planning and navigation. For object detection, there have been significant advances in neural networks for single-modality approaches. However, it has been surprisingly difficult to train networks to use multiple modalities in a way that demonstrates gain over single-modality networks. In this dissertation, we first propose three networks for Camera-LiDAR and Camera-Radar fusion. For Camera-LiDAR fusion, CLOCs (Camera-LiDAR Object Candidates fusion) and Fast-CLOCs are presented. CLOCs fusion provides a multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate 3D detection results. Fast-CLOCs can run in near real-time with less computational requirements compared to CLOCs. Fast-CLOCs eliminates the separate heavy 2D detector, and instead uses a 3D detector-cued 2D image detector (3D-Q-2D) to reduce memory and computation. For Camera-Radar fusion, we propose TransCAR, a Transformer-based Camera-And-Radar fusion solution for 3D object detection. The cross-attention layer within the transformer decoder can adaptively learn the soft-association between the radar features and vision queries instead of hard-association based on sensor calibration only. Then, we propose to solve the 3D multiple object tracking (MOT) problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M3). In particular, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M3 filter can naturally model these uncertainties accurately and elegantly. We combine learning-based detections with our RFS-M3 tracker by incorporating the detection confidence score into the PMBM prediction and update step. We have evaluated our CLOCs, Fast-CLOCs and TransCAR fusion-based 3D detector and RFS-M3 3D tracker using challenging datasets including KITTI, nuScenes, Argoverse and Waymo that are released by academia and industry leaders. Superior experimental results demonstrated the effectiveness of the proposed approaches.","Thesis (Ph. D.)--Michigan State University. Electrical Engineering, 2022","Includes bibliographical references (pages 113-126)"],"dc:format":["xviii, 126 pages","application/pdf"],"dc:identifier":["etd:50699","isbn:9798841789024","oclc:on1373601595","oclc:1373601595","umi:29327697","local:Pang_grad.msu_0128D_19357","https://doi.org/doi:10.25335/nt7t-8z08"],"dc:language":["English"],"dc:relation":["Electronic Theses & Dissertations"],"dc:rights":["Attribution-NonCommercial 4.0 International"],"dc:subject":["Electrical engineering","Automated vehicles","Optoelectronic devices","Computer vision","Automatic tracking"],"dc:title":["3D object detection and tracking for autonomous vehicles"],"dc:type":["Text","Theses"]},"updated_at":"2026-11-18T18:57:07Z"}