ViCARUS: Vision-centric Connected Autonomy for Vulnerable Road Users' Safety

In the United States, pedestrian fatalities have risen dramatically, with a 57% increase from 4,779 deaths in 2013 to 7,522 in 2022 (NHTSA, 2024). Intersections remain particularly dangerous, accounting for approximately 25% of all traffic fatalities and 50% of injuries (FHWA, 2024). Vulnerable Road Users (VRUs) - including pedestrians, cyclists, and motorcyclists - are especially at risk in these scenarios. On the other hand, early road tests show autonomous driving systems (Waymo Safety Report, 2024) have the potential to do better than human drivers safety-wise. However, previous work under Safety 21 UTC project #440, Connected Vision for Improved Pedestrian Safety (CVIPS) (Bhagavatula et al., 2024) has shown that many state of the art computer vision algorithms, when tested on autonomous driving benchmarks such as nuScenes (nuScenes detection task, 2020) show significant performance degradation when detecting VRUs as shown in Fig 1 of supplementary materials. Recent studies suggest that connected autonomous vehicles using Vehicle-to-Everything (V2X) communication can offer enhanced capability in detecting and responding to potential safety critical scenarios (Gao et al., 2022), particularly in complex urban environments. However, current V2X datasets and methods are limited in their scope to single-vehicle detection. Extending this by benchmarking perception performance for VRUs and other road users in a multi-agent environment is a critical opportunity for improving road safety (Xiang et al., 2024). In addition, as part of CVIPS (Bhagavatula et al., 2024), the research team simulated the impact of communications limitations (Shenkut & Vijaya Kumar, 2024) such as latency, bandwidth, and communication interruptions. Fig 3 and Fig 4 of supplementary materials show these impacts in different V2X scenarios. As shown in Fig 3, communication interruption from even a single agent causes ~25% performance degradation. Fig 4 illustrates the impact caused by communication latency, where a delay of about 200 ms causes a 10% performance drop in detection. These communication limitations make it hard to consistently detect and track VRUs, especially in scenarios with occluded views and varying weather conditions. Building upon the findings from CVIPS, ViCARUS aims to enhance VRU safety through four primary research thrusts. Primary Research Thrusts (1) Investigation of Detection Performance Gap: Extending the CVIPS findings regarding camera-only systems' inferior VRU detection performance by analyzing the disparity in detection performance between camera-only systems and sensor fusion methods and studying specific scenarios where detection gaps are most pronounced and developing mechanisms to minimize these detection gaps through cooperative perception. (2) Cooperative Perception Framework development: Implementing a connected perception approach where cameras in vehicles and infrastructure units share information via V2X to improve detection performance on VRUs. (3) Simulation-to-Real Adaptation: Creating comprehensive synthetic datasets using CARLA, developing scenarios not well-represented in existing datasets, using existing real-world datasets such as WTS (Kong et al., 2024), V2X-Real (Xiang et al., 2024), TUMTraf V2X (Zimmer et al., 2024), designing adaptation techniques to bridge the gap between synthetic training data and real-world performance and developing computationally lightweight methods suitable for real-world deployment. (4) V2X Communication Analysis: Using advanced C-V2X simulators (e.g., Fraunhofer IIS C-V2XSim platform (Simulation C-V2X, 2024)) to investigate the impact of critical C-V2X parameters such as Bandwidth, latency, packet losses and error rates, Global Navigation Satellite System (GNSS)-related location errors and quantifying tradeoffs between communication constraints and system performance The project will be implemented through the following tasks: (1) Creation of synthetic image sequences and acquisition of existing real datasets with different VRU scenarios; (2) Development and testing of deep learning-based cooperative perception algorithms on both synthetic and real datasets; (3) Developing methods to adapt algorithms trained on synthetic data to real-world datasets (Sim2Real adaptation); (4) C-V2X parameter impact analysis; and (5) Evaluation under challenging imaging conditions and safety improvement demonstration.

• Record URL:
  • https://ppms.cit.cmu.edu/projects/detail/582

Language

• English

Project

• Status: Completed
• Funding: $175,526.00
• Contract Numbers:

  69A3552344811

  69A3552348316

• Sponsor Organizations:

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590
• Managing Organizations:

  Safety21 University Transportation Center

  Carnegie Mellon University
  Pittsburgh, PA  United States  15213

  Carnegie Mellon University

  Pittsburgh, PA  United States 
• Performing Organizations:

  Carnegie Mellon University

  Pittsburgh, PA  United States 
• Principal Investigators:

  Bhagavatula, Vijayakumar

• Start Date: 20250701
• Expected Completion Date: 20260831
• Actual Completion Date: 20260831
• USDOT Program: University Transportation Centers

Subject/Index Terms

• TRT Terms: Algorithms; Autonomous vehicles; Connected vehicles; Deep learning; Pedestrian detectors; Pedestrian safety; Vehicle to everything communications; Vulnerable road users
• Subject Areas: Data and Information Technology; Highways; Pedestrians and Bicyclists; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 02002849
• Record Type: Research project
• Source Agency: Safety21 University Transportation Center
• Contract Numbers: 69A3552344811, 69A3552348316
• Files: UTC, RIP
• Created Date: Sep 17 2026 3:12PM