Visible and Thermal Imaging in a Deep-Learning Approach to Robust Automated Pothole Detection and Highway Maintenance Prioritization

Potholes are a primary pavement distress that can compromise safety and cause expensive damage claims. U.S. motorists suffer repair costs of $3 billion annually from damage caused by potholes. 2D image-based sensing presents as a low-cost option for rapid pavement data collection and for detecting potholes. However, quality of potholes detection using only visible images may be significantly compromised due to poor lighting, weather conditions (e.g., fog, rain), low contract to surrounding pavement. Thermal images are more robust to lighting and weather conditions and may offer additional unique features (e.g., temperature difference between pothole and surrounding pavement) that can be used for pothole detection. On the other hand, current practice for image-based potholes detection still mainly involves manual identification, which is time-and-cost consuming. Algorithms that can automate the processing of collected data and provide accurate and robust detection of potholes are needed to enable timely and cost-effective highway maintenance. This project proposes the integration of both visible and thermal images captured by visible & thermal dual camera and the use of deep learning to enable robust, accurate and automated detection of potholes to help prioritize highway maintenance. The major objectives include: (1) Create a unique and valuable database of geotagged and labeled trios of visible, thermal and fused images for training pothole detection algorithms; (2) Develop deep learning algorithms for pothole detection, and investigate the impact of incorporating thermal and fused images on pothole detection accuracy and robustness; and (3) Develop automated tools for pothole detection, pothole mapping and updating.


  • English


  • Status: Active
  • Funding: $98400
  • Contract Numbers:


  • Sponsor Organizations:

    Office of the Assistant Secretary for Research and Technology

    University Transportation Centers Program
    Department of Transportation
    Washington, DC  United States  20590
  • Managing Organizations:

    Mountain-Plains Consortium

    North Dakota State University
    Fargo, ND  United States  58108
  • Project Managers:

    Tolliver, Denver

  • Performing Organizations:

    Dept. of Civil and Environmental Engineering

    Colorado State University
    Fort Collins, CO  United States 
  • Principal Investigators:

    Jia, Gaofeng

  • Start Date: 20200218
  • Expected Completion Date: 20240731
  • Actual Completion Date: 0
  • USDOT Program: University Transportation Centers Program
  • Source Data: MPC-620

Subject/Index Terms

Filing Info

  • Accession Number: 01732721
  • Record Type: Research project
  • Source Agency: Mountain-Plains Consortium
  • Contract Numbers: 69A3551747108
  • Files: UTC, RIP
  • Created Date: Mar 5 2020 8:26AM