An End-to-End Deep Learning System for Pavement Distress Detection, Severity Estimation, and Condition Reporting

Details

• Creators:
  Aboah, Armstrong ; Tolliver, Denver ; Kyem, Blessing Agyei ; Denteh, Eugene ; Asamoah, Joshua Kofi
• Corporate Creators:
  Center for Transformative Infrastructure Preservation and Sustainability (CTIPS) Region 8 University Transportation Center (UTC) ; North Dakota State University ; Upper Great Plains Transportation Institute
• Corporate Contributors:
  United States. Department of Transportation. University Transportation Centers (UTC) Program ; United States. Department of Transportation. Office of the Assistant Secretary for Research and Technology
• Subject/TRT Terms:
  • Pavement condition
  • Deep learning
  • Pavement distress
  • Risk assessment
  • Estimation theory
  • Numerical analysis
  • Decision making
  • Artificial intelligence
  • Potholes
  • Pavement Condition Index
  More +
• Publication/ Report Number:
  CTIPS-26-010
• Resource Type:
  Tech Report
• Geographical Coverage:
  United States
• Edition:
  Final Report
• Corporate Publisher:
  Center for Transformative Infrastructure Preservation and Sustainability (CTIPS) Region 8 University Transportation Center (UTC)
• Description:
  Regular assessment of pavement condition is critical for safe and functional roadways, yet traditional inspections are subjective, labor-intensive, and difficult to scale. Recent AI-based approaches have automated distress detection, but they produce only numerical outputs without the plain-language explanations needed to communicate findings to engineers, agencies, and the public. To address this gap, this project introduces PaveCap, a multimodal framework that assesses pavement condition through both numerical prediction and natural language description. The framework consists of two components: (1) a single-shot pavement condition index (PCI) estimation network that combines YOLOv8, the segment anything model (SAM), and a four-layer convolutional neural network to predict PCI values on a 0 to 100 scale; (2) a dense captioning network that uses a YOLOv8 backbone, a transformer architecture, and a convolutional feed-forward module to generate structured descriptions of pavement severity and distress conditions. To train and evaluate the framework, we developed a multimodal dataset of 7,158 pavement images with bounding box annotations, expert-authored textual descriptions, and PCI values aligned with ASTM D6433 guidelines. The PCI estimation network achieved a mean absolute error of 16.21, and the dense captioning network generated accurate descriptions with BLEU-1, GLEU, and METEOR scores of 0.3799, 0.3234, and 0.4253, outperforming five alternative backbones. Overall, PaveCap advances pavement condition assessment from numerical prediction to interpretable, human-centered AI, providing transportation agencies with a scalable tool that both quantifies pavement condition and explains it in natural language to support more transparent maintenance decisions.
• Format:
  PDF
• Funding:
  69A3552348308
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/94246/dot_94246_DS1.pdf
• File Type:
  [PDF - 6.46 MB]
• Collection(s):
  University Transportation Centers
• Main Document Checksum:
  urn:sha-512:f485a1a644b656581710d25bdab90bb93d98f147416420c53e9f42fbafb0fb7e340a7512560e3a7cf8bbe72fdeacac8e089c4bee67a56b91170dce0fde95cb33