Exploring Top-Down Visual Attention for Transportation Behavior Analysis

This project stands at the intersection of cognitive psychology, artificial intelligence (AI) and computer vision, and transportation safety and efficiency. By focusing on the nuanced ways in which humans allocate their visual attention, and how this can inform the development of AI and machine learning (ML) to aid in self-driving cars, transportation safety automation, and transportation planning and scheduling in general, this project promises to contribute significantly to the field, ensuring safer, more intuitive driving experiences, and smoother traveling experiences for the traveling public. By performing human behavior analysis with visual attention, the research team aims to develop best practices for safe and efficient interaction of automated roadway vehicles with existing vehicles, roadside hardware, pedestrians, cyclists, and motorcyclists. The advantages of a top-down attention approach include: prioritizing relevance, improving accuracy, enhancing machine learning efficiency, adapting models to scenarios, and enabling better human interaction. By exploring top-down visual attention, the team aims to build machine learning models to achieve the following objectives that are coherently connected with each other, where the first two will be the objectives in the base phase of this proposal and the last two would be in a second phase for a follow-on effort: (1) Develop human behavior analysis machine learning architectures that allow autonomous driving and other transportation systems to anticipate the attention and reaction patterns of both human drivers and pedestrians, thereby preventing accidents. These include the human behavior analysis of the interaction between a driver and their vehicle, driver and pedestrians, humans with the existing vehicles and roadside hardware. The ML architectures explored will be CNN for image encoding for improving accuracy and reducing computation, GCN for relation reasoning focusing on human interaction and actions, and transformers for self-attention and feedback. (2) Investigate the potential of using visual attention models to improve autonomous and/or automated vehicle navigation and decision-making processes in complex environments. The visual attention mechanisms will be driven by both data and knowledge, including dynamic transportation information, roadside hardware information, location-based information (maps, events, tasks). As a start point, the team will leverage the state-of-the-art (SOTA) model such as Analysis-by-Synthesis Vision Transformer (AbSViT) to encode feature selection, higher-level feedback and top-down input, added on the typical bottom-up process in deep models. (3) Develop multimodal human-machine interface dashboards in self-driving cars and vehicle safety automation system, making them more intuitive for human users. These include audio, visual and haptic features as well as accessibility functions that the team has studied for helping the navigation of people who are blind or have low vision. Supported by the AI/ML-based architectures and attention models, the interface as dashboards will also allow developers, engineers and users to access the intelligent transportation systems for interaction, interpretation and diagnosis. (4) Furthermore, collaborative opportunities may arise with existing projects, especially in applying the findings to enhance the travel pattern analysis and other safety features of self-driving and/or existing vehicles and pedestrians. Collaboration could involve sharing data, methodologies, and insights to refine autonomous driving technologies' perception and decision-making capabilities.

• Record URL:
  • https://tbd.ctr.utexas.edu/research-product/exploring-top-down-visual-attention-for-transportation-behavior-analysis/

Language

• English

Project

• Status: Active
• Funding: $245,046.00
• Contract Numbers:

  69A3552344815

  69A3552348320

• Sponsor Organizations:

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590
• Managing Organizations:

  Center for Understanding Future of Travel Behavior and Demand

  University of Texas
  Austin, TX  United States 
• Project Managers:

  Bhat, Chandra

• Performing Organizations:

  City College of New York

  Civil Engineering, Steinman T-127
  140th Street and Convent Avenue
  New York, NY  United States  10031
• Principal Investigators:

  Zhu, Zhigang

• Start Date: 20240601
• Expected Completion Date: 20250531
• Actual Completion Date: 0
• USDOT Program: University Transportation Centers Program

Subject/Index Terms

• TRT Terms: Artificial intelligence; Attention; Autonomous vehicles; Behavior; Computer vision; Human machine systems; Interfaces; Machine learning
• Subject Areas: Data and Information Technology; Planning and Forecasting; Safety and Human Factors; Transportation (General); Vehicles and Equipment;

Filing Info

• Accession Number: 01954940
• Record Type: Research project
• Source Agency: Center for Understanding Future of Travel Behavior and Demand
• Contract Numbers: 69A3552344815, 69A3552348320
• Files: UTC, RIP
• Created Date: May 13 2025 7:14PM