Challenges to the Monitoring of Deployed AI Systems: Center for AI Standards and Innovation

Details

• Alternative Title:
  NIST Trustworthy and Responsible AI NIST AI 800-4
• Creators:
  Rao, Anita ; Keller, Andrew ; Kalra, Neha ; Steed, Ryan ; Kwegyir-Aggrey, Kweku ; Klyman, Kevin ; Staheli, Diane ; Bergman, Stevie
• Corporate Creators:
  United States. Department of Commerce. National Institute of Standards and Technology
• Subject/TRT Terms:
  • Artificial intelligence
  • Monitoring
  • Deployment
  • Measurement
  • Evaluation
  • Technology assessment
  • Generative AI
• Publication/ Report Number:
  NIST AI 800-4
• DOI:
  https://doi.org/10.6028/NIST.AI.800-4
• Resource Type:
  Tech Report
• Geographical Coverage:
  United States
• Corporate Publisher:
  United States. Department of Commerce. National Institute of Standards and Technology
• Abstract:
  As artificial intelligence (AI) systems are increasingly integrated into commercial and government applications, there is a growing need to monitor these systems in real-world settings. Although pre-deployment evaluations are valuable for assessing AI system capabilities at multiple points prior to release, they are predominantly conducted in controlled testing environments. Post-deployment monitoring is crucial for (1) validating that AI systems operate reliably as expected in real-world scenarios, (2) tracking unforeseen outputs that occur due to, e.g., model non-determinism or dynamic input conditions, and (3) visibility into unexpected consequences of AI systems in deployment contexts. Stakeholders across the AI ecosystem agree on the need for post-deployment monitoring; however, monitoring best practices, validated methodologies, and common terminology are still nascent and scattered across the field. This report proposes monitoring categories and surfaces challenges to robust post-deployment AI system monitoring, rooted in practitioner workshops and a literature review. The identified gaps, barriers, and open questions highlight opportunities for further investigation and innovation. Notably, this report quotes practitioners' repeated calls for guidance on post-deployment AI system monitoring methods.
• Format:
  PDF
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/93272/dot_93272_DS1.pdf
• File Type:
  [PDF - 1.19 MB]
• Collection(s):
  Artificial Intelligence Deployment Center of Excellence (AI COE)
• Main Document Checksum:
  urn:sha-512:609a93a076bd503d2e2a8bf4f3b0ead9abf4b81d8f04af41cc0f89093915427b5106765e0fd4b8274c32154b48eec89ef4e3939ce315ce4be7ff0b1c6114f2b1