Humanoid as Human Assessment for the Safety of Vulnerable Road Users

Despite rapid advancements in autonomous driving and connected vehicle technology, significant challenges persist in the safety assessment of the new technologies. Ensuring the functional safety of these complex systems is essential, yet there still needs to be a critical gap in methodologies that can comprehensively cover all necessary testing conditions. This gap is especially pronounced when evaluating scenarios involving vulnerable road users, defined as pedestrians, bicyclists, individuals using personal conveyances, and those in work zones as outlined by the ANSI D16.1-2007 standard and U.S. safety regulations. Three main challenges highlight the complexity of this task. First, critical scenarios involving vulnerable road users are rare in real-world settings, making it difficult to gather sufficient data for testing. Simulated environments often lack the fidelity to replicate nuanced interactions, complicating the evaluation of autonomous vehicle performance in high-risk situations. Second, despite advances in simulation technologies, a sim-to-real gap remains due to inherent limitations in simulating vehicle behavior and environmental complexity. This requires real-world testing to validate simulation findings, which can otherwise undermine confidence in autonomous vehicle safety. Third, most real-world evaluations rely on simple dummies to represent pedestrians, failing to capture diverse, realistic human behaviors and interactions. This limits the ability to evaluate autonomous systems under realistic conditions thoroughly. To the best of the research team's knowledge, neither the academic community nor industry leaders have converged on a robust, standardized approach for conducting real-world evaluations that accurately reproduce complex, high-risk scenarios involving vulnerable road users and intricate traffic conditions. Addressing these challenges requires multidisciplinary research and innovative solutions that enhance both simulation fidelity and real-world testing methodologies. Inspired by advancements in large language models (LLMs), the research team proposes to develop a specialized foundation model for the evaluation of autonomous vehicles, focused on scenarios involving vulnerable road users. The team aims to create a platform that integrates humanoid robots capable of simulating these road users' movements to test autonomous vehicle responses. Leveraging their expertise in critical scenario generation, the team aims to build a foundation model that produces scenarios posing safety challenges to vulnerable road users. These scenarios will be generated randomly, conditionally based on text prompts, or through retrieval-augmented generation using specific scenarios as input. The team will also develop a platform with humanoid robots programmed to replicate the behaviors of pedestrians, scooter users, and individuals in wheelchairs. The research will include developing algorithms to simulate the movements of pedestrians with specific physical limitations, ensuring a comprehensive range of real-world testing. Additionally, the team will apply their prior work in accelerated evaluation to enhance testing efficiency. This initiative seeks to address current validation challenges and improve the safety and reliability of autonomous systems. The team has built a strong connection to PennSTART who will deploy the vulnerable robots to their test track which will be ready in 2026. Over the next half-decade, while Pittsburgh leverages its strong industrial background and leadership in the autonomy sector, it will face increased national and international competition. By contributing to autonomous driving evaluation and focusing on vulnerable road user safety, this project aims to reinforce Pittsburgh’s position as a leader in the industry and support its continued innovation and growth.

• Record URL:
  • https://ppms.cit.cmu.edu/projects/detail/583

Language

• English

Project

• Status: Completed
• Funding: $200,000.00
• Contract Numbers:

  69A3552344811

  69A3552348316

• Sponsor Organizations:

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590
• Managing Organizations:

  Safety21 University Transportation Center

  Carnegie Mellon University
  Pittsburgh, PA  United States  15213

  Carnegie Mellon University

  Pittsburgh, PA  United States 
• Performing Organizations:

  Carnegie Mellon University

  Pittsburgh, PA  United States 
• Principal Investigators:

  Zhao, Ding

• Start Date: 20250701
• Expected Completion Date: 20260831
• Actual Completion Date: 20260831
• USDOT Program: University Transportation Centers

Subject/Index Terms

• TRT Terms: Artificial intelligence; Autonomous vehicles; Behavior; Connected vehicles; Pedestrian movement; Pedestrian safety; Robots; Testing; Vulnerable road users
• Subject Areas: Data and Information Technology; Highways; Pedestrians and Bicyclists; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 02002850
• Record Type: Research project
• Source Agency: Safety21 University Transportation Center
• Contract Numbers: 69A3552344811, 69A3552348316
• Files: UTC, RIP
• Created Date: Sep 17 2026 3:16PM