Promoting Automated Vehicle Safety Using Multi Modal Data and Large Language Models
Automated vehicles must operate in complex environments that involve interactions among human drivers, cyclists, and pedestrians. Traditional rule based approaches may miss subtle factors that contribute to safety risks in these settings. This project will integrate multi modal datasets, including video, radar, LiDAR, global positioning system (GPS), and text based information from logs and incident reports, to train large language models capable of analyzing and predicting potential risk scenarios. The research will create a unified pipeline that fuses sensor data with language inputs so that the model can generate structured assessments and clear explanations of conditions that may influence safety. The project will test the system in simulation and controlled real world environments. Additional evaluations will include cybersecurity challenges and human machine interface testing to ensure that the system is robust, practical, and informative for users. The final framework will support transportation agencies, developers, and industry partners in understanding and addressing AV safety challenges by providing tools for improved risk detection, situational reasoning, and communication.
Language
- English
Project
- Status: Active
- Funding: $197,000.00
-
Contract Numbers:
69A3552348301
-
Sponsor Organizations:
Office of the Assistant Secretary for Research and Technology
University Transportation Centers Program
Department of Transportation
Washington, DC United States 20590 -
Managing Organizations:
University of Massachusetts, Amherst
Department of Civil and Environmental Engineering
130 Natural Resources Road
Amherst, MA United States 01003 -
Performing Organizations:
University of Connecticut, Storrs
Connecticut Transportation Institute
270 Middle Turnpike, Unit 5202
Storrs, CT United States 06269-5202 -
Principal Investigators:
Zhang, Wei
- Start Date: 20260101
- Expected Completion Date: 20261231
- Actual Completion Date: 0
- USDOT Program: University Transportation Centers Program
- Subprogram: University Transportation Centers
Subject/Index Terms
- TRT Terms: Autonomous vehicles; Computer security; Data fusion; Human machine systems; Risk assessment; Sensors; Traffic safety; Vehicle mix
- Subject Areas: Data and Information Technology; Highways; Safety and Human Factors; Vehicles and Equipment;
Filing Info
- Accession Number: 01973943
- Record Type: Research project
- Source Agency: New England University Transportation Center
- Contract Numbers: 69A3552348301
- Files: UTC, RIP
- Created Date: Dec 11 2025 1:43PM