DeepHyd: A Deep Learning-based Artificial Intelligence Approach for the Automated Classification of Hydraulic Structures from LiDAR and Sonar Data

The research team proposes to develop a spatially explicit 3D modeling framework and a software package that are based on deep learning, a cutting-edge artificial intelligence approach, for automated and reliable classification of point cloud data of hydraulic structures (DeepHyd). Point cloud data, collected through Geiger and terrestrial LiDAR and bathymetric sonar technologies, provide rich information in terms of hydraulic structures (e.g., bridge piers, culverts or scour features). However, the efficient processing of these point cloud data and their subsequent classification into hydraulic features of interest represent a grand challenge. This is because the volume of the point cloud data involved is often huge (i.e. a big data analytics challenge), and also because the hydraulic features of interest are often complicated in terms of, for example, their complex shapes and temporal changes related to erosion or structural modification. Deep learning is a state-of-the-art artificial intelligence approach that is based on combination of unsupervised and supervised machine learning of many-layered neural networks for complex problem-solving in general and classification of hydraulic structures here. The utility of deep learning in object recognition and computer vision, as often required by unmanned autonomous systems (e.g., self-driving cars), has been increasingly acknowledged. In particular, the deep learning-based approach is well-suited to handling classification tasks which involve massive data. Therefore, the team plans to develop such a deep learning-based artificial intelligence solution (DeepHyd, including the framework and software tools) to resolve the big data-driven computational challenge facing the extraction and classification of hydraulic features from large volume LiDAR and sonar data. The proposed deep learning-based model and software product will provide substantial support for the rapid, automated, and reliable detection of hydraulic structures of interest, which will greatly facilitate the asset management facing NCDOT in terms of efficiency and effectiveness. Ultimately, the results of this research will also inform NCDOT as to the utility of using Geiger and terrestrial LiDAR and sonar data to efficiently and reliably extract the elevation and spatial data relationships of hydraulic structures with the necessary accuracy to develop 3D models of those structures that can be used in 2 or 3D hydraulic models for water level and scour calculations to improve hydraulic structural designs.