Human-centered Steel Bridge Inspection enabled by Augmented Reality and Artificial Intelligence

State departments of transportation (DOTs) currently rely on trained inspectors to visually inspect bridge components for detecting structural deterioration and damage, which can be limited in accuracy, speed, repeatability, and reliability. On the other hand, computer vision (CV) can see what human eyes cannot, and artificial intelligence (AI) such as deep learning has shown tremendous ability to conceptualize and generalize. By integrating CV and Augmented Reality (AR), a recent NCHRP Highway IDEA project (Li et al., 2022) completed by this project team successfully demonstrated how human-centered AR environment and automated CV algorithms can empower bridge inspectors to perform more accurate and efficient field inspections of steel bridges for fatigue cracks. The inspector wearing an AR headset (Microsoft HoloLens 2) examines the steel bridge and records a short video of the target structural surface through the AR headset. The video is then automatically uploaded to the server, where the computer vision algorithm analyzes the video by detecting and analyzing surface motion through feature points (pinks dots in the upper right figure). These feature points are then projected in near real time in front of the inspector’s eyes as holograms through the AR headset, allowing the inspector to interact with the hologram through a virtual menu to examine the results under different threshold values for crack detection, enabling human-in-the-loop decision-making. The NCHRP Highway IDEA project has successfully demonstrated the concept of human-centered bridge inspection by integrating CV and AR using an AR headset as the hardware platform. However, further developments are needed for successful adoption of this tool in practical bridge inspections. In addition, the idea of human-centered bridge inspection would have a broader impact if realized on a wider range of mobile platforms such as tablet devices. The goal of this proposed pooled fund study is to develop a full-fledged AR-based bridge inspection tool that leverages CV and AI to support field detection, quantification, and documentation of various damages and deteriorations for steel bridges.  The main objective of this proposed research is to provide state DOTs practical tools for supporting human-centered steel bridge inspection with real-time defect (e.g., fatigue cracks and corrosion) detection, documentation, tracking, and decision making. The proposed research will not only bridge the gaps identified in the IDEA project, but also expand the existing capability by developing AI algorithms for crack and corrosion detection. In addition to AR headsets, the project will also develop AR-based inspection capability using tablet devices.  The tablet device can be used to perform AR-based inspection directly in a similar way to the AR headset. It can also leverage Unmanned Aerial Vehicles (UAV) for remote image and video acquisition during inspections, enabling bridge inspections from a distance in a human-centered manner. 

• Record URL:
  • http://www.pooledfund.org/Details/Study/769

Language

• English

Project

• Status: Programmed
• Funding: $600,000.00
• Contract Numbers:

  TPF-5(535)

• Sponsor Organizations:

  California Department of Transportation

  1227 O Street
  Sacramento, CA  United States  95814

  Kansas Department of Transportation

  Eisenhower State Office Building
  700 SW Harrison Street
  Topeka, KS  United States  66603-3754

  Texas Department of Transportation

  125 E. 11th Street
  Austin, TX  United States  78701-2483

  North Carolina Department of Transportation

  P.O. Box 25201
  1 South Wilmington Street
  Raleigh, NC  United States  27611
• Managing Organizations:

  Kansas Department of Transportation

  Eisenhower State Office Building
  700 SW Harrison Street
  Topeka, KS  United States  66603-3754
• Project Managers:

  Behzadpour, David

• Start Date: 20250121
• Expected Completion Date: 0
• Actual Completion Date: 0

Subject/Index Terms

• TRT Terms: Artificial intelligence; Cracking; Drones; Inspection; Steel bridges; Virtual reality
• Subject Areas: Bridges and other structures; Data and Information Technology; Highways; Maintenance and Preservation;

Filing Info

• Accession Number: 01922765
• Record Type: Research project
• Source Agency: Federal Highway Administration
• Contract Numbers: TPF-5(535)
• Files: RIP, USDOT, STATEDOT
• Created Date: Jun 26 2024 9:55AM