Cost-Effective Radar-Based Depth Perception and Scene Interpretation

Details

• Creators:
  Dolan, John M.
• Corporate Creators:
  Carnegie Mellon University
• Corporate Contributors:
  Carnegie Mellon University. Traffic21 Institute. Safety21 University Transportation Center (UTC) ; United States. Department of Transportation. Office of the Assistant Secretary for Research and Technology ; United States. Department of Transportation. University Transportation Centers (UTC) Program ; United States. Department of Transportation. Federal Highway Administration
• Subject/TRT Terms:
  • Autonomous vehicles
  • Dimensional analysis
  • Sensors
  • Fusion
  • Estimation theory
  • Cost effectiveness
  • Video cameras
  • Data collection
  • Algorithms
  • Urban areas
  More +
• Resource Type:
  Tech Report
• Geographical Coverage:
  United States
• Edition:
  Final Report (July 1, 2025 - July 31, 2026)
• Corporate Publisher:
  Carnegie Mellon University. Traffic21 Institute. Safety21 University Transportation Center (UTC)
• Description:
  One of the current challenges for autonomous vehicles (AV) is providing provable safety in urban driving and highway driving. LIDAR provides accurate depth and shape information, but remains quite expensive. In this research we explore the viability of fusing 4D radar images with camera images as an alternative to using LIDAR. Our research aims to establish the data fusion of radar and camera images in autonomous driving as a replacement for data fusion using traditional LIDAR, radar, and camera images. Our current approach demonstrates that a trained radar detector can achieve approximately 3.5 meters of Unidirectional Chamfer Distance (UCD) against ground truth LiDAR data on a dataset recently released by Delft University of Technology (DUT) in the Netherlands, which is 28% better than the state of the art (SOTA).Our approach integrates the images obtained from deep neural network (DNN) 4D radar detectors with conventional camera RGB images. It introduces a novel pixel positional encoding algorithm inspired by Bartlett's spectrum estimation technique. This algorithm transforms radar depth maps and RGB images into a unified pixel image subspace facilitating the learning of their potential correspondence. Our method effectively leverages high-resolution camera images to train radar depth map generative models, addressing the limitations of conventional radar detectors in complex vehicular environments while sharpening the radar output. We develop spectrum estimation algorithms tailored for radar depth maps and RGB images, a comprehensive training framework for data-driven generative models, and a camera-radar deployment scheme for AV operation.
• Format:
  PDF
• Funding:
  69A3552344811 ; 69A3552348316
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/93538/dot_93538_DS1.pdf
• File Type:
  [PDF - 888.16 KB]
• Collection(s):
  Autonomous Vehicles ; University Transportation Centers
• Main Document Checksum:
  urn:sha-512:8d8ec956e02934c5bafab4d0e57186bf89fa85fde909667e1a7e67c9f9de419640cfaec020a79c57b50420fb3ebf7d5da4687a52d06adf9d3e35b73a9b521735