Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groups

Jan Ondras, Abrar Anwar, Tong Wu, Fanjun Bu, Malte Jung, Jorge Jose Ortiz, Tapomayukh Bhattacharjee

Research output: Contribution to journalConference articlepeer-review

Abstract

We develop data-driven models to predict when a robot should feed during social dining scenarios. Being able to eat independently with friends and family is considered one of the most memorable and important activities for people with mobility limitations. While existing robotic systems for feeding people with mobility limitations focus on solitary dining, commensality, the act of eating together, is often the practice of choice. Sharing meals with others introduces the problem of socially appropriate bite timing for a robot, i.e. the appropriate timing for the robot to feed without disrupting the social dynamics of a shared meal. Our key insight is that bite timing strategies that take into account the delicate balance of social cues can lead to seamless interactions during robot-assisted feeding in a social dining scenario. We approach this problem by collecting a Human-Human Commensality Dataset (HHCD) containing 30 groups of three people eating together. We use this dataset to analyze human-human commensality behaviors and develop bite timing prediction models in social dining scenarios. We also transfer these models to human-robot commensality scenarios. Our user studies show that prediction improves when our algorithm uses multimodal social signaling cues between diners to model bite timing. The HHCD dataset, videos of user studies, and code are available at https://emprise.cs.cornell.edu/hrcom/.

Original languageEnglish (US)
Pages (from-to)921-933
Number of pages13
JournalProceedings of Machine Learning Research
Volume205
StatePublished - 2023
Externally publishedYes
Event6th Conference on Robot Learning, CoRL 2022 - Auckland, New Zealand
Duration: Dec 14 2022Dec 18 2022

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Keywords

  • Assistive Robotics
  • Group Dynamics
  • HRI
  • Multimodal Learning

Fingerprint

Dive into the research topics of 'Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groups'. Together they form a unique fingerprint.

Cite this