Verification of the Learned Models in Machine Learning and Artificial Intelligence-based Aviation Systems
Details
-
Creators:
-
Corporate Creators:
-
Corporate Publisher:
-
Corporate Contributors:
-
Subject/TRT Terms:
-
Publication/ Report Number:
-
DOI:
-
Resource Type:
-
Geographical Coverage:
-
Edition:Final Report
-
Description:This report presents a comprehensive verification framework for machine learning (ML) and artificial intelligence (AI) models integrated into safety-critical avionic software stacks, with a specific focus on computer vision for autonomous Unmanned Aircraft System (UAS) package delivery and safe landing operations. Acknowledging that traditional static validation metrics (such as mean Average Precision (mAP) fail to capture temporal anomalies and continuous perception dropouts during flight, this research introduces a three-thrust approach to ensure deterministic safety and statistical reliability.
Thrust I establishes mathematical metrics for evaluating training data sufficiency, introducing Frame Interval Error (FIE) to quantify time-varying perception blackouts and utilizing an Optimal Transport (OT) framework with Wasserstein distances to measure statistical mismatch between training data and the Operational Design Domain (ODD). Thrust II implements conformal prediction to quantify runtime uncertainty, providing finite-sample confidence guarantees with bounded error rates; a comparative benchmarking between a Real-Time Detection Transformer (RT-DETR) and YOLOv8s highlights critical trade-offs between statistical calibration accuracy and onboard computational latency. Thrust III accelerates the verification pipeline by deploying a high-fidelity synthetic data generation suite using Microsoft AirSim and Unreal Engine to expose edge-case model vulnerabilities under extreme weather and lighting conditions. Collectively, these methodologies provide a mathematically rigorous toolkit for the Federal Aviation Administration (FAA) to certify learning-based perception systems in next-generation aviation. -
Format:
-
Funding:
-
Download URL:
-
File Type:
-
Collection(s):
-
Main Document Checksum:urn:sha-512:0e424b4d41aeee3204d676cce686bc3d2f9b518f9723cb9b760189281fcc34989857e2270446144ccad8f56a4d431019d00ca65f938ab5493cbfc673484d8369