Safety through Agility: Using Mixed Reality to tune shared autonomy systems

The research team is developing an virtual reality Driver Training system with augmented reality pass through where a 16 year-olds or patients with compromised cognitive capabilities can sit in a stationary real vehicle and use mixed-reality to learn driving skills and get exposed to increasingly challenging driving scenarios. Sensors are strapped onto the steering and brake/gas pedals of their car and capture driver movements, that are fed back into the simulation. The windshield and windows are overlaid with a VR generated driving simulator scenario. As a result, the driver is sitting in a real vehicle with AR passthrough showing the steering wheel and the driver’s hands hands, but the risky driving scenarios are simulated. The goal of this system is to develop a simulator that can be retrofitted in any car so novice drivers can train at home in a safe way and experience risky scenarios that cannot be demonstrated in real-life. Mixed-reality or XR means VR with AR passthrough. So some visual elements are VR and others are camera passthroughs of reality. The Problem: The high costs of elder care, both to the individual and the government, combined with the demographic shift towards an increasing number of older adults as a percentage of the overall US population is creating a major healthcare crisis. The number of senior citizens in the US in 2030 will be twice that of 2000, leading to a shortage of working-age caregivers and putting increased pressure on labor costs. Equally important is maintaining, or preferably ameliorating, the quality of life of a growing elderly population. Maintaining elders’ autonomy is correlated with increasing quality of life and autonomy enhancement is correlated with improving functionality. Driving is typically a symbol of autonomy. The revocation of driving privileges is often the first step taken by families worried about cognitive decline and emerging dementia of the older adult. Dementia including Alzheimer's disease is a chronic, progressive syndrome that is characterized by a reduction in the ability to perform daily activities, e.g. a cognitive decline with increasing unpredictability and psychological symptoms. Dementia affects about 5 million people in the USA and 35 million worldwide. Coincidentally, Autonomous Vehicles (AVs) are a game-changing AI and robotic solution that can enable older people to maintain independence. For this technology to be effectively deployed, Safety and Trust are however key. Older people, but also caregivers and clinicians need to view the technology as safe and trustworthy. To realize this potential, a robust shared autonomy strategy is needed. The term shared autonomy is an oxymoron, but it embodies the tension observed as caregivers, clinicians, and patients negotiate the need to trust the autonomous system and the desire to stay in control. This research project aims to address the question on how to mediate autonomy between participating actors to allow the human control of the system up to their level of performance and autotune the degree of intervention by the machine to maintain safety. Approach: To these ends, the team proposes the development of an interactive imitation learning system for safe human autonomous systems. The system is trained by an expert for multiple levels of performance following a curriculum. When the system is deployed with a human non-expert user (e.g. an older driver), the safe by construction neural network controller ensures safety at any level of performance (see figure on the left). This enables the system to personalize its capabilities to suit the human partner while ensuring safety from any mismatches in the expectations of the controller and that of the human user. The AVs will therefore learn how the user desires to share autonomy and ensure the system does not reach an unsafe state under all operating conditions and inputs from the human.

• Record URL:
  https://ppms.cit.cmu.edu/projects/detail/446

Language

• English

Project

• Status: Active
• Funding: $30000
• Contract Numbers:

  69A3552344811

• Sponsor Organizations:

  Carnegie Mellon University

  Safety21 National UTC for Promoting Safety
  Pittsburgh, PA  United States  15213

  Office of the Assistant Secretary for Research and Technology

  University Transportation Center Program
  ,    
• Managing Organizations:

  Carnegie Mellon University

  Safety21 National UTC for Promoting Safety
  Pittsburgh, PA  United States  15213
• Project Managers:

  Stearns, Amy

• Performing Organizations:

  University of Pennsylvania

  ,    
• Principal Investigators:

  Mangharam, Rahul

• Start Date: 20230701
• Expected Completion Date: 20240630
• Actual Completion Date: 0
• USDOT Program: University Transportation Centers

Subject/Index Terms

• TRT Terms: Aged drivers; Autonomous vehicle handover; Autonomous vehicles; Driver training; Driving simulators; Teenage drivers; Virtual reality
• Subject Areas: Education and Training; Highways; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 01900373
• Record Type: Research project
• Source Agency: Safety21
• Contract Numbers: 69A3552344811
• Files: UTC, RIP
• Created Date: Nov 21 2023 8:34PM