Multi-modal AI Agents for Railway Safety

Artificial Intelligent (AI) agents, powered by foundation models, such as ChatGPT, have transformed every aspect of everyday life, in personal and professional settings, and have also started making substantial progress in specializing and producing results in various scientific and engineering domains. In this project, continuing the effort the research team started in Year 2 which entailed the development of a prototype for a large language foundation model for railway safety, the team will work towards developing a multi-modal AI agent for railway safety, which will be able to seamlessly integrate structured and unstructured text (such as accident reports and policy documents) with image data pertaining to a railway crossing and perform a number of tasks such as analyzing, comparing, and contrasting different railway crossings with respect to their risk factors and/or accident history, and come up with safety recommendations specifically tailored to a crossing. The proposed AI agent will combine rich domain expertise and the ability to sift through and analyze vast amounts of data that no human operator or policy maker may realistically be able to, thus empowering large scale data-driven railway safety.

• Record URL:
  • https://www.utrgv.edu/railwaysafety/research/operations/index.htm

Language

• English

Project

• Status: Active
• Funding: $150,000.00
• Contract Numbers:

  69A3552348340

• Sponsor Organizations:

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590

  University of California, Riverside

  1084 Columbia Ave.
  Riverside, CA  United States  92507
• Managing Organizations:

  University of California, Riverside

  1084 Columbia Ave.
  Riverside, CA  United States  92507
• Project Managers:

  Stearns, Amy

• Performing Organizations:

  University of California, Riverside

  1084 Columbia Ave.
  Riverside, CA  United States  92507
• Principal Investigators:

  Papalexakis, Evangelos

  Chen, Jia

  Dong, Yue

  Xu, Ping

• Start Date: 20250601
• Expected Completion Date: 20260831
• Actual Completion Date: 0
• USDOT Program: University Transportation Centers Program

Subject/Index Terms

• TRT Terms: Artificial intelligence; Data analysis; Railroad safety; Risk analysis; Risk assessment
• Subject Areas: Data and Information Technology; Railroads; Safety and Human Factors;

Filing Info

• Accession Number: 01960677
• Record Type: Research project
• Source Agency: University Transportation Center for Railway Safety
• Contract Numbers: 69A3552348340
• Files: UTC, RIP
• Created Date: Jul 14 2025 7:24PM