Supplemental Data Collection and Processing for Bridge Safety Inspections Utilizing Mixed Reality and Artificial Intelligence

Details

• Creators:
  Sarlo, Rodrigo ; Gabbard, Joseph ; Smith, Alan ; Luksas, John
• Corporate Creators:
  Virginia Tech ; Virginia Transportation Research Council (VTRC)
• Corporate Contributors:
  Virginia. Department of Transportation ; United States. Department of Transportation. Federal Highway Administration
• Subject/TRT Terms:
  Artificial Intelligence Augmented Reality Automated Defect Detection Automation Bridge Inspection Bridges Bridge Substructures Computer Vision Data Collection Human-Computer Interaction Human Factors User Experience

  More +
• Publication/ Report Number:
  FHWA/VTRC 25-R4
• Resource Type:
  Tech Report
• Geographical Coverage:
  Virginia ; United States
• Edition:
  Final Report
• Corporate Publisher:
  Virginia Transportation Research Council (VTRC)
• Abstract:
  This research work focused on evaluating both the technical and design elements necessary to perform augmented reality (AR) assisted bridge inspections. The inspection prototype developed by this project aims to demonstrate the potential benefits and limitations of incorporating head-worn displays into the inspection workflow. By virtue of switching to an electronic data collection method, many common quality assurance issues can be mitigated. Furthermore, AR has an additional advantage over tablet-based solutions in its hands-free operation, which can allow for increased safety and flexibility. Furthermore, data collected in an AR system is georeferenced locally to the bridge and shown to be adequately accurate to allow navigation to previously documented defects during the inspection. Finally, and most importantly, AR serves as a central platform within which many up-and-coming digital tools (such as artificial intelligence) can be integrated. To better understand this potential, the work also considered the use of AI for a specific use case (concrete crack measurements). A prototype application was developed using the Microsoft HoloLens 2 headset, but the principles employed could be translated to other similar mixed reality platforms. The prototype targeted efficient integration into the entire field inspection workflow. Three key functionalities were identified and addressed as main challenges to this holistic inspection approach: 1) a structured data entry interface, 2) georeferenced annotations for visualizing historical data, and 3) computer vision for automated defect labeling. A unique research focus was how to address these challenges in imperfect, real-world field scenarios involving both novice and professional bridge inspectors. To this end, the research team explored a range of AR interaction techniques of various degrees of automation, drawing on human-computer interaction principles. The study's evaluation metrics focused on measurement accuracy, time on task, as well as the tool's impact on perceived workload and usability.
• Format:
  PDF
• Funding:
  120109
• Collection(s):
  US Transportation Collection
• Main Document Checksum:
  urn:sha-512:e5a77429f6a0feff49890be8a6a273e40da121f4f14673d06d3f44b34860f573ab892a819e133c524763f1c37c9e491f4533fec3d120872c158da38d4424f148
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/79383/dot_79383_DS1.pdf
• File Type:
  [PDF - 5.61 MB ]