Developing a Robotic Simulator and Video Games for Professional and Public Training (WD-2)

Civil engineers are not educated with robotics. They need to be trained on the job with effective tools. The most recent simulation trainer that the PI has built is currently being used by the US Navy to train surface warfare officers in decision making under stress. In a crowded in-port environment, the crew on a ship’s bridge is trained to probe and identify suspicious boat behavior within the port’s traffic pattern. Officers in charge of the simulation training lesson use software for high-level control of dozens of other ships, boats, and aircrafts that quickly react and adapt to the crewed ship’s actions based on lower-level programmed autonomy and game-like user interaction. Without this virtual “experience,” improperly trained crews put lives in danger. Scenarios that would be catastrophic in reality can also be simulated and, without this training, especially for recovering from error states, operators may inadvertently lose valuable hardware, produce erroneous results, and even compromise system and human safety. Approach and Methodology. An iterative software engineering process will be used to investigate and build a series of Simulation Training And Control System (STACS) prototypes. These prototypes will run on Android/iOS devices, PCs, or MACs and will be used for operator training (in simulation) and operator control of a robot team during inspection. The tradeoff between operator control and robot autonomy in simulation will be investigated to gain insight into the design of efficient control software in robots and develop an effective robot team, including multiple UAVs and climbing robots. Overall Objectives: This project aims to build a STACS prototype within a 3D simulation game-like environment and develop a realistic training environment. Specific objectives include: (1) Investigate and optimize the design of user interaction and user interfaces within a full 3D game-like environment for training and control, (2) Investigate and optimize the tradeoff between manual and autonomous control of multi-robot teams for bridge inspection, (3) Train bridge inspectors in the use of the proposed multi-robot system, and (4) Provide human operators with complete situational awareness and operational control during an ongoing inspection. Scope of Work in Year 1: (1) Design, build, and test four STACS prototypes in a full 3D environment within the Unity3D game engine; (2) Design and build a simulation “game” with tele-operative, semi-autonomous, and fully autonomous control of the proposed multi-robot system for bridge inspection; and (3) Establish a communication link between the STACS and real robots to facilitate transition from simulation to real-robot monitoring, interaction, and control. Scope of Work in Year 2: (1) Further improve the user interaction and user interfaces design of STACS, (2) Extend the message support for the two-way communication infrastructure, and (3) Apply virtual reality techniques for immersive robot controlling. Scope of Work in Year 3: (1) Conduct the third iteration of user interaction and user interfaces design for STACS with the consideration of a large number of inspection robots, (2) Use evolutionary algorithms to optimize route planning over multiple heterogeneous inspection robots with different battery capacity and different velocity, and (3) Design a Virtual Reality user interface and user interaction in STACS to provide human operators with a high level of immersion for robot control and interaction. Scope of Work in Year 3: (1). Conduct the third iteration of user interaction and user interfaces design for STACS with the consideration of a large number of inspection robots (2). Use evolutionary algorithms to optimize route planning over multiple heterogeneous inspection robots with different battery capacity and different velocity, (3). Design a Virtual Reality user interface and user interaction in STACS to provide human operators high level of immersion for robot control and interaction. Scope of Work in Year 4: We aim to have a single trained operator control multiple robots. The software system being proposed thus enables a single person to control multiple heterogeneous inspection robots being investigated and developed under other INSPIRE thrust areas. Fewer expensive human operators reduced cost while multiple inspection robots inspecting in parallel reduces time. Our proposed system thus directly meets the INSPIRE UTC goal of making inspection and maintenance more reliable and cost-effective. Scope of Work in Year 5: This new project will investigate and develop techniques to 1) Move STACS towards a Virtual Reality (VR) interface for training and control, 2) connect simulated robots with real robots being developed in other INSPIRE projects, and 3) investigate and develop algorithms, protocols, and autonomy for maximizing inspection speed and completeness for human-robot bridge inspection teams on real bridges.

• Record URL:
  http://inspire-utc.mst.edu/researchprojects/wd-2/

Language

• English

Project

• Status: Completed
• Funding: $186,600
• Contract Numbers:

  69A3551747126

• Sponsor Organizations:

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590
• Managing Organizations:

  Inspecting and Preserving Infrastructure through Robotic Exploration University Transportation Center

  Missouri University of Science and Technology
  Rolla, MO  United States  65409
• Performing Organizations:

  University of Nevada, Reno

  1664 N. Virginia Street
  Reno, Nevada  United States  89557
• Principal Investigators:

  Louis, Sushil

• Start Date: 20170301
• Expected Completion Date: 20220831
• Actual Completion Date: 0
• USDOT Program: University Transportation Centers

Subject/Index Terms

• TRT Terms: Bridges; Control systems; Inspection; Optimization; Prototypes; Robots; Simulation; Training programs
• Subject Areas: Bridges and other structures; Education and Training; Highways; Maintenance and Preservation;

Filing Info

• Accession Number: 01646003
• Record Type: Research project
• Source Agency: Inspecting and Preserving Infrastructure through Robotic Exploration University Transportation Center
• Contract Numbers: 69A3551747126
• Files: UTC, RIP
• Created Date: Sep 14 2017 8:02PM