Building Reliable Sim Driving Agents by Scaling Self-Play

Details

• Creators:
  Cornelisse, Daphne ; Pandya, Aarav ; Joseph, Kevin ; Suarez, Joseph ; Vinitsky, Eugene
• Corporate Creators:
  New York University. Tandon School of Engineering ; Connected Communities for Smart Mobility Toward Accessible and Resilient Transportation for Equitably Reducing Congestion (C2SMARTER) Tier-1 University Transportation Center (UTC)
• Corporate Contributors:
  United States. Department of Transportation. University Transportation Centers (UTC) Program ; United States. Department of Transportation. Office of the Assistant Secretary for Research and Technology
• Subject/TRT Terms:
  Artificial Intelligence Autonomous Vehicles Driving Behavior Performance Tests Simulation Waymo
• DOI:
  https://doi.org/10.48550/arXiv.2502.14706
• Resource Type:
  Manuscript
• Geographical Coverage:
  United States
• Corporate Publisher:
  Connected Communities for Smart Mobility Toward Accessible and Resilient Transportation for Equitably Reducing Congestion (C2SMARTER) Tier-1 University Transportation Center (UTC)
• Abstract:
  Simulation agents are essential for designing and testing systems that interact with humans, such as autonomous vehicles (AVs). These agents serve various purposes, from benchmarking AV performance to stress-testing system limits, but all applications share one key requirement: reliability. To enable sound experimentation, a simulation agent must behave as intended. It should minimize actions that may lead to undesired outcomes, such as collisions, which can distort the signal-to-noise ratio in analyses. As a foundation for reliable sim agents, we propose scaling self-play to thousands of scenarios on the Waymo Open Motion Dataset under semi-realistic limits on human perception and control. Training from scratch on a single GPU, our agents solve almost the full training set within a day. They generalize to unseen test scenes, achieving a 99.8% goal completion rate with less than 0.8% combined collision and of-road incidents across 10,000 held- out scenarios. Beyond in-distribution generalization, our agents show partial robustness to out-of-distribution scenes and can be fine-tuned in minutes to reach near-perfect performance in such cases. We open-source the pre-trained agents and integrate them with a batched multi-agent simulator. Demonstrations of agent behaviors can be viewed at https://sites.google.com/view/reliable-sim-agents, and we open-source our agents at https://github.com/Emerge-Lab/gpudrive.
• Content Notes:
  Please cite this article as: Cornelisse, D., Pandya, A., Joseph, K., Suárez, J., and Vinitsky, E. (2025). Building reliable sim driving agents by scaling self-play. https://doi.org/10.48550/arXiv.2502.14706.
• Format:
  PDF
• Funding:
  69A3552348326
• Collection(s):
  University Transportation Centers Autonomous Vehicles
• Main Document Checksum:
  urn:sha-512:7888a372d89ca702364c22e3677d71114d350271a820465254ed96c30e3cb98fc560ab092465acdb157195531815d201ebf69aaea41f9be85dd69754f33b2ffe
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/86404/dot_86404_DS1.pdf
• File Type:
  [PDF - 4.31 MB ]