Separation Management: Automation Reliability Meta-Analysis and Conflict Probe Reliability Analysis
Advanced Search
Select up to three search categories and corresponding keywords using the fields to the right. Refer to the Help section for more detailed instructions.

Search our Collections & Repository

For very narrow results

When looking for a specific result

Best used for discovery & interchangable words

Recommended to be used in conjunction with other fields

Dates

to

Document Data
Library
People
Clear All
Clear All

For additional assistance using the Custom Query please check out our Help Page

i

Separation Management: Automation Reliability Meta-Analysis and Conflict Probe Reliability Analysis

Filetype[PDF-525.29 KB]


English
Details You May Also Like

Details:

  • Creators:
  • Corporate Creators:
    United States. Department of Transportation. Federal Aviation Administration. William J. Hughes Technical Center
  • Subject/TRT Terms:
  • Publication/ Report Number:
    DOT/FAA/TC-TN12/65
  • Resource Type:
    Tech Report
  • Geographical Coverage:
    United States
  • Corporate Publisher:
    United States. Department of Transportation. Federal Aviation Administration. William J. Hughes Technical Center
  • Abstract:
    Objective: The purpose of this study is to establish whether a valid performance criterion exists to determine the acceptability of the En Route Automation Modernization (ERAM) Conflict Probe’s conflict-detection accuracy and to evaluate observed accuracy against this criterion. Background: The Conflict Probe can exhibit very high or very low accuracy depending on the analysis technique. It is necessary to establish both an empirically backed criterion for accuracy and the appropriate accuracy analyses and metrics. Method: A meta-analysis was conducted on Human Factors automation reliability literature; an additional analysis was done on the results of probe reliability studies by the Federal Aviation Administration (FAA) Concept Analysis branch to derive various accuracy metrics. The results were compared to determine the acceptability of the conflict probe accuracy. Results: The meta-analysis produced an estimated criterion of 65% correct responses for automation to improve performance, but this estimate is subject to a broad confidence interval due to variability in the source data from the literature. The probe performance exceeded the 65% value when giving credit for all correct rejections, but it fell short when not giving credit for correct rejections. Another metric, Positive Predictive Value (PPV, the percent of alerts that are valid), is operationally meaningful and its values demonstrated large accuracy improvements over baseline with the FAA Concept Analysis’ parametric adjustments, but a PPV cutoff criterion could not be established from the meta-analysis. Conclusion: The present results provide insight on several fronts, but operational input is essential to determine (1) a more justifiable air traffic control-specific accuracy criterion and (2) which aircraft encounters are appropriate to include in a test set for accuracy assessment. Automation responses should only increase the value of an accuracy metric to the extent that the responses add informational value for the controller. Applications: The present results will be used in the design of an evaluation to derive a set of operationally meaningful aircraft encounters. Different measures of accuracy, such as PPV, also merit further exploration.
  • Format:
    PDF
  • Collection(s):
  • Main Document Checksum:
  • Download URL:
  • File Type:

Supporting Files

  • No Additional Files
More +

You May Also Like

Checkout today's featured content at rosap.ntl.bts.gov