Generating Unsafe Road Activity From LLMs and Road Imagery to Learn Better Safety

Details

• Creators:
  Tamburo, Robert ; Narasimhan, Srinivasa
• Corporate Creators:
  Carnegie Mellon University ; Carnegie Mellon University. Traffic21 Institute. Safety21 University Transportation Center (UTC)
• Corporate Contributors:
  United States. Department of Transportation. University Transportation Centers (UTC) Program ; United States. Department of Transportation. Office of the Assistant Secretary for Research and Technology
• Subject/TRT Terms:
  • Autonomous land vehicles
  • Digital simulation
  • Vulnerable road users
  • Navigation
  • Planning
  • Training
  • Video cameras
  • Cyclists
  • Datasets
  • autonomous vehicle guidance
  More +
• Resource Type:
  Tech Report
• Geographical Coverage:
  United States ; Pittsburgh (Pennsylvania)
• Edition:
  Final Report (July 1, 2025 - July 31, 2026)
• Corporate Publisher:
  Carnegie Mellon University. Traffic21 Institute. Safety21 University Transportation Center (UTC)
• Description:
  In the proposed work, we will address these challenges by developing a system to simulate unsafe vehicle, pedestrian, bicyclist, and road worker activity from previously acquired images and large language models (LLMs). The proposed work also enables safety technology development without exhausting resources for camera installation. Focus will be on generating visual data of normal and anomalous activities such as near misses, erratic driving, blocked bicycle lanes, etc. Methods will be developed to model realistic driver behavior based on observations from datasets acquired from prior UTC projects in collaboration with deployment partners City of Pittsburgh Department of Mobility & Infrastructure and Shaler Township. Realistic bicyclist behavior will be modeled using bicyclist-centric data provided by partner dashcam.bike. Methods will be developed to leverage these models with LLMs to generate unsafe activity involving vehicles and VRUs based on text entered by users. The system will be deployed to a website so that anyone can create their own dataset. The benefit of such a road activity system will enable safety research and permit data driven evaluation of road or intersection safety without installing cameras and storing/analyzing hours upon hours of images per camera.
• Format:
  PDF
• Funding:
  69A3552344811 ; 69A3552348316
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/93546/dot_93546_DS1.pdf
• File Type:
  [PDF - 836.98 KB]
• Collection(s):
  University Transportation Centers
• Main Document Checksum:
  urn:sha-512:eb0412cc7443b4db6aa35bdeca5b858f602a5e9df741f450bec47b00d67f933f4808f8838f18ccd9427ba62093b5a1253e93d5b8786a23fd32c8e19bc48dde19