1. Home
  2. RIP
  3. View Record
https://www.nationalacademies.org/trb/events

AI and Immersive AR/VR for Sustained Material and Construction Inspection Training and Knowledge Retention

This project leverages advancements in Large Language Models (LLMs), such as OpenAI’s ChatGPT, which can interpret and generate human-like responses using extensive data. LLMs have demonstrated proven capabilities in related areas, such as contextual knowledge retrieval, interactive training, and generating domain-specific insights, making them highly suitable for addressing challenges associated with inspection training and knowledge retention. Building on these capabilities, the research team proposes the development of InspectionGPT, a specialized LLM tailored for inspection training. The system will consist of two key components: the InspectionGPT chatbot and a 3D VR training environment. InspectionGPT chatbot will be developed using existing foundation LLM models, such as LLaMA, GPT-4, Falcon, and BLOOM. These models will be trained with NDOT’s inspection guidelines, standards, reports, and past training materials stored in NDOT’s local repository(called knowledge base). Once trained, InspectionGPT chatbot will provide personalized guidance and dynamic learning experiences to trainees, communicating in English. The system also allows for easy updates as new knowledge can be added simply by storing relevant documents and descriptions in the knowledge base. A central feature of the system is the GPT Protocol, a plain-English document that defines how InspectionGPT should respond to user requests. This protocol enables NDOT to adjust and customize InspectionGPT’s behavior without any programming expertise, ensuring the system is flexible, user- friendly, and easy to maintain.