Mixed Reality Assisted Infrastructure Inspections

This project developed an artificial intelligence (AI)-based application in Mixed Reality platforms aimed at assisting inspectors in the localization and quantification of concrete surface defects. AI-based methods of defect localization and quantification can be more efficient, potentially reduce the subjectivity of the results, improve accuracy as a complementary technology. A main goal of the project was to develop real time AI-based inspection technology to allow the inspector to oversee and approve its results. Rather than removing the human involvement, this research wanted to benefit from the inspector’s expertise in AI-based inspections using Human-AI collaboration in the Mixed Reality platform. An AI-based application was developed for deployment in Mixed Reality devices to take advantage of developed AI models while allowing the inspector to oversee the results and intervene if necessary. Different from a fully automated system and post-processing the data, the inspector continuously interacts with the AI model, approves its results, or corrects its errors. The developed AI model also follows an attention-guided method where the defect localization and quantification are carried out separately, which both help improve the accuracy of the results and increase the inspector’s involvement. The developed application was successfully deployed in a Mixed Reality headset and tested. One of the main challenges of AI-based inspection in real-time is the processing power. This project successfully displayed that the optimized model could perform in real-time in certain Mixed reality devices. It was also shown that human involvement in the inspection can improve the accuracy of the AI and minimize post-processing time to a minimum. In short, the developed methodology improves the accuracy of inspections by facilitating the use of AI models for visual inspection. It reduces the subjectivity of the inspection results by accurately measuring the defect sizes and also improves the documentation of the inspection results by benefiting from the Mixed Reality platform. The device allows the inspector to access hard-to-reach areas, therefore reducing risks of inspection. It is also faster and more accurate than handheld tools commonly used in visual inspection. This study demonstrated the potential of such devices as a very beneficial complimentary technology. The final report is avaliable. 

• Record URL:
  http://apps.trb.org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=4915

Language

• English

Project

• Status: Completed
• Funding: $135000
• Contract Numbers:

  Project 20-30, IDEA 222

• Sponsor Organizations:

  National Cooperative Highway Research Program

  Transportation Research Board
  500 Fifth Street, NW
  Washington, DC  United States  20001

  American Association of State Highway and Transportation Officials (AASHTO)

  444 North Capitol Street, NW
  Washington, DC  United States  20001

  Federal Highway Administration

  1200 New Jersey Avenue, SE
  Washington, DC  United States  20590

  Safety Innovations Deserving Exploratory Analysis (IDEA)

  Transportation Research Board
  500 Fifth Street, NW
  Washington, DC  United States  20001
• Project Managers:

  Jawed, Inam

• Performing Organizations:

  University of Central Florida

  ,    
• Principal Investigators:

  Catbas, Necati

• Start Date: 20190101
• Expected Completion Date: 0
• Actual Completion Date: 0

Subject/Index Terms

• TRT Terms: Artificial intelligence; Bridges; Concrete structures; Data collection; Defects; Field tests; Infrastructure; Inspection; Machine learning; Methodology; Optimization; Surfaces; Virtual reality
• Subject Areas: Bridges and other structures; Data and Information Technology; Highways; Maintenance and Preservation;

Filing Info

• Accession Number: 01739005
• Record Type: Research project
• Source Agency: Transportation Research Board
• Contract Numbers: Project 20-30, IDEA 222
• Files: TRB, RIP
• Created Date: May 11 2020 5:48PM