A2X: An Agent and Environment Interaction Benchmark for Multimodal Human Trajectory Prediction

Samuel S. Sohn, Mihee Lee, Seonghyeon Moon, Gang Qiao, Muhammad Usman, Sejong Yoon, Vladimir Pavlovic, Mubbasir Kapadia

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

In recent years, human trajectory prediction (HTP) has garnered attention in computer vision literature. Although this task has much in common with the longstanding task of crowd simulation, there is little from crowd simulation that has been borrowed, especially in terms of evaluation protocols. The key difference between the two tasks is that HTP is concerned with forecasting multiple steps at a time and capturing the multimodality of real human trajectories. A majority of HTP models are trained on the same few datasets, which feature small, transient interactions between real people and little to no interaction between people and the environment. Unsurprisingly, when tested on crowd egress scenarios, these models produce erroneous trajectories that accelerate too quickly and collide too frequently, but the metrics used in HTP literature cannot convey these particular issues. To address these challenges, we propose (1) the A2X dataset, which has simulated crowd egress and complex navigation scenarios that compensate for the lack of agent-to-environment interaction in existing real datasets, and (2) evaluation metrics that convey model performance with more reliability and nuance. A subset of these metrics are novel multiverse metrics, which are better-suited for multimodal models than existing metrics. The dataset is available at: https://mubbasir.github.io/HTP-benchmark/.

Original languageEnglish (US)
Title of host publicationProceedings - MIG 2021
Subtitle of host publication14th ACM SIGGRAPH Conference on Motion, Interaction, and Games
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450391313
DOIs
StatePublished - Nov 10 2021
Event14th ACM SIGGRAPH Conference on Motion, Interaction, and Games, MIG 2021 - Virtual, Online, Switzerland
Duration: Nov 10 2021Nov 12 2021

Publication series

NameProceedings - MIG 2021: 14th ACM SIGGRAPH Conference on Motion, Interaction, and Games

Conference

Conference14th ACM SIGGRAPH Conference on Motion, Interaction, and Games, MIG 2021
Country/TerritorySwitzerland
CityVirtual, Online
Period11/10/2111/12/21

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Human-Computer Interaction
  • Education

Keywords

  • datasets
  • evaluation metrics
  • human trajectory prediction

Fingerprint

Dive into the research topics of 'A2X: An Agent and Environment Interaction Benchmark for Multimodal Human Trajectory Prediction'. Together they form a unique fingerprint.

Cite this