Connected Vision for Increased Pedestrian Safety (CVIPS)

This proposal is aimed at increasing the safety of vulnerable road users (VRUs) such as pedestrians, cyclists, and scooter riders, specifically at intersections. One approach to increase traffic safety is to bring more autonomy into vehicles with the goal of avoiding human-specific problems such as distractions and drunken driving. Towards this goal, computer vision algorithms are applied on data from vision sensors such as RGB cameras and LIDAR for detecting other objects in the scene. Most of these algorithms perform well on detecting larger objects such as vehicles but not as well on detecting pedestrians and other VRUs. Another challenge is that, unlike human drivers with whom pedestrians can communicate with a plethora of verbal and non-verbal cues, interacting with autonomous vehicles is a nascent concept, Further, the performance of deep learning-based vision techniques can degrade significantly in challenging conditions such as low-light environments, sun glare and severe occlusions, e.g., a pedestrian darting out from between two parked cars. In line with US DOT’s vision for connected and autonomous vehicles, this proposal is aimed at developing and evaluating computer vision solutions that offer increased VRU safety by using C-V2X capabilities. This proposed Connected Vision for Increased Pedestrian Safety (CVIPS) system is illustrated schematically in Fig. 1 (see supplement). Each agent extracts pedestrian location and trajectory information using a video vision transformer and communicates that information to other participating agents using a unified representation such as the Bird’s Eye View (BEV). An example of the benefits of CVIPS is shown schematically in Fig. 2 (in the supplement) which shows a scenario with 4 vehicle cameras and 1 infrastructure (e.g., traffic light) camera. Here a pedestrian is not visible to two of the vehicle cameras but is visible to another vehicle camera and the infrastructure camera. CVIPS relies on C-V2X connectivity and it is important to understand the impact of the C-V2X parameters (e.g., bandwidth, latency, etc.) and limitations on the data available to the multiple agents. For example, is there enough bandwidth to share the full video frames or should we share only the bounding boxes of detected VRUs? Also, achieving a unified BEV representation requires knowledge of the locations of participating cameras and the impact of location errors (e.g., caused by GNSS) needs to be studied. It is also important to consider challenging imaging conditions such as rain/snow, sun glare, night time, etc. Deep learning solutions can be highly demanding in storage and computational complexity and so one of the goals will be to develop light-weight implementations. Also, there are no known datasets that provide videos from multiple cameras covering the diverse range of pedestrian scenarios the research team proposes to investigate. The team proposes to generate synthetic data using the high-fidelity CARLA simulator. Initial algorithm development and evaluation will be based on synthetic data, but best-performing algorithms will be tested on real data that will be collected after identifying relevant driving scenarios and obtaining the necessary IRB approvals. CVIPS project consists of the following major research tasks: (1) Creating synthetic image sequences for the VRU scenario, (2) acquiring real data after IRB approval, (3) development and testing of baseline deep learning algorithms for pedestrian detection and pedestrian trajectory estimation, (4) investigation of the impact of C-V2X parameters on the accuracy of pedestrian detection and trajectory estimation, (5) evaluating the developed algorithms under challenging imaging conditions, and (6) quantifying the increased pedestrian safety through CVIPS.

• Record URL:
  https://ppms.cit.cmu.edu/projects/detail/440

Language

• English

Project

• Status: Active
• Funding: $290497
• Contract Numbers:

  69A3552344811

• Sponsor Organizations:

  Carnegie Mellon University

  Safety21 National UTC for Promoting Safety
  Pittsburgh, PA  United States  15213

  Office of the Assistant Secretary for Research and Technology

  University Transportation Center Program
  ,    
• Managing Organizations:

  Carnegie Mellon University

  Safety21 National UTC for Promoting Safety
  Pittsburgh, PA  United States  15213
• Project Managers:

  Stearns, Amy

• Performing Organizations:

  Carnegie Mellon University

  ,    
• Principal Investigators:

  Bhagavatula, Vijayakumar

• Start Date: 20230701
• Expected Completion Date: 20240630
• Actual Completion Date: 0
• USDOT Program: University Transportation Centers

Subject/Index Terms

• TRT Terms: Algorithms; Cameras; Machine vision; Mobile communication systems; Pedestrian detectors; Pedestrian safety; Vulnerable road users
• Subject Areas: Data and Information Technology; Highways; Pedestrians and Bicyclists; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 01900243
• Record Type: Research project
• Source Agency: Safety21
• Contract Numbers: 69A3552344811
• Files: UTC, RIP
• Created Date: Nov 20 2023 8:48PM