An analysis of speech as a modality for activity recognition during complex medical teamwork

Swathi Jagannath, Aleksandra Sarcevic, Ivan Marsic

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

We analyzed the nature of verbal communication among team members in a dynamic medical setting of trauma resuscitation to inform the design of a speech-based automatic activity recognition system. Using speech transcripts from 20 resuscitations, we identified common keywords and speech patterns for different resuscitation activities. Based on these patterns, we developed narrative schemas (speech -workflow- models) for five most frequently performed activities and applied linguistic models to represent relationships between sentences. We evaluated the narrative schemas with 17 new cases, finding that all five schemas adequately represented speech during activities and could serve as a basis for speech-based activity recognition. We also identified similarities between narrative schemas of different activities. We conclude with design implications and challenges associated with speechbased activity recognition in complex medical processes.

Original languageEnglish (US)
Title of host publicationProceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2018
PublisherAssociation for Computing Machinery
Pages88-97
Number of pages10
ISBN (Electronic)9781450364508
DOIs
StatePublished - May 21 2018
Event12th EAI International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2018 - New York, United States
Duration: May 21 2018May 24 2018

Publication series

NameACM International Conference Proceeding Series

Other

Other12th EAI International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2018
CountryUnited States
CityNew York
Period5/21/185/24/18

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Keywords

  • Activity recognition
  • Decision support
  • Emergency medicine
  • Narrative schema
  • Speech analysis
  • Speech modeling

Fingerprint Dive into the research topics of 'An analysis of speech as a modality for activity recognition during complex medical teamwork'. Together they form a unique fingerprint.

  • Cite this

    Jagannath, S., Sarcevic, A., & Marsic, I. (2018). An analysis of speech as a modality for activity recognition during complex medical teamwork. In Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2018 (pp. 88-97). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3240925.3240941