1. Home
  2. RIP
  3. View Record
https://www.nationalacademies.org/event/07-18-2022/trbs-automated-road-transportation-symposium

Mixed Reality Interfaces for Simulation Training and Control

This project aims to extend and further investigate and develop mixed reality interfaces for the Simulation Training and Control System (STACS). STACS helps bridge inspectors equipped with virtual or augmented reality hardware to work with semi-autonomous and autonomous robots to more quickly, thoroughly, efficiently, and effectively inspect simulated truss bridges. The project team's efforts to date have resulted in a simulation training system for bridge inspection enabling a heterogeneous group of simulated robots to speed up comprehensive bridge inspection. Starting with simulation training with simulated robots inspecting simulated bridges, this new project will investigate and develop techniques to (1) enable automated bridge inspection scenario generation for workforce training, (2) enable real robots with simulated sensor feeds to inspect simulated bridges and, if time permits, enable a simulated inspection robot in an augmented reality environment to inspect a real-world bridge and (3) improve our algorithms for maximizing inspection speed and completeness for human-robot bridge inspection teams on real and virtual bridges. Note that the same system and command interfaces are being used for training human bridge operators in simulation and in an operational real-world environment. The project team will transition their work to MST’s DOT focused labs for workforce training and to help build the next generation of mixed reality enhanced robotic bridge inspection systems. This will enable the DOT to leverage our system’s AI and autonomy development to increase safety and reduce cost in bridge inspection tasks.