Separation Management: Automation Reliability Meta-Analysis and Conflict Probe Reliability Analysis
Advanced Search
Select up to three search categories and corresponding keywords using the fields to the right. Refer to the Help section for more detailed instructions.

Search our Collections & Repository

All these words:

For very narrow results

This exact word or phrase:

When looking for a specific result

Any of these words:

Best used for discovery & interchangable words

None of these words:

Recommended to be used in conjunction with other fields

Language:

Dates

Publication Date Range:

to

Document Data

Title:

Document Type:

Library

Collection:

Series:

People

Author:

Help
Clear All

Query Builder

Query box

Help
Clear All

For additional assistance using the Custom Query please check out our Help Page

i

Separation Management: Automation Reliability Meta-Analysis and Conflict Probe Reliability Analysis

Filetype[PDF-525.29 KB]


  • English
    • Details You May Also Like

    Details:

    • Creators:
    • Corporate Creators:
      United States. Department of Transportation. Federal Aviation Administration. William J. Hughes Technical Center
    • Subject/TRT Terms:
    • Publication/ Report Number:
      DOT/FAA/TC-TN12/65
    • Resource Type:
      Tech Report
    • Geographical Coverage:
      United States
    • Corporate Publisher:
      United States. Department of Transportation. Federal Aviation Administration. William J. Hughes Technical Center
    • Abstract:
      Objective: The purpose of this study is to establish whether a valid performance criterion exists to determine the acceptability of the En Route Automation Modernization (ERAM) Conflict Probe’s conflict-detection accuracy and to evaluate observed accuracy against this criterion. Background: The Conflict Probe can exhibit very high or very low accuracy depending on the analysis technique. It is necessary to establish both an empirically backed criterion for accuracy and the appropriate accuracy analyses and metrics. Method: A meta-analysis was conducted on Human Factors automation reliability literature; an additional analysis was done on the results of probe reliability studies by the Federal Aviation Administration (FAA) Concept Analysis branch to derive various accuracy metrics. The results were compared to determine the acceptability of the conflict probe accuracy. Results: The meta-analysis produced an estimated criterion of 65% correct responses for automation to improve performance, but this estimate is subject to a broad confidence interval due to variability in the source data from the literature. The probe performance exceeded the 65% value when giving credit for all correct rejections, but it fell short when not giving credit for correct rejections. Another metric, Positive Predictive Value (PPV, the percent of alerts that are valid), is operationally meaningful and its values demonstrated large accuracy improvements over baseline with the FAA Concept Analysis’ parametric adjustments, but a PPV cutoff criterion could not be established from the meta-analysis. Conclusion: The present results provide insight on several fronts, but operational input is essential to determine (1) a more justifiable air traffic control-specific accuracy criterion and (2) which aircraft encounters are appropriate to include in a test set for accuracy assessment. Automation responses should only increase the value of an accuracy metric to the extent that the responses add informational value for the controller. Applications: The present results will be used in the design of an evaluation to derive a set of operationally meaningful aircraft encounters. Different measures of accuracy, such as PPV, also merit further exploration.
    • Format:
      PDF
    • Collection(s):
    • Main Document Checksum:
    • Download URL:
    • File Type:

    Supporting Files

    • No Additional Files

    More +

    You May Also Like

    Checkout today's featured content at rosap.ntl.bts.gov

    Version 3.26