Assessing the feasibility of speech-based activity recognition in dynamic medical settings

Swathi Jagannath; Aleksandra Sarcevic; Neha Kamireddi; Ivan Marsic

doi:10.1145/3290607.3312983

Assessing the feasibility of speech-based activity recognition in dynamic medical settings

Swathi Jagannath, Aleksandra Sarcevic, Neha Kamireddi, Ivan Marsic

School of Engineering, Electrical & Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We describe an experiment conducted with three domain experts to understand how well they can recognize types and performance stages of activities using speech data transcribed from verbal communications during dynamic medical teamwork. The insights gained from this experiment will inform the design of an automatic activity recognition system to alert medical teams to process deviations in real time. We contribute to the literature by (1) characterizing how domain experts perceive the dynamics of activity-related speech, and (2) identifying the challenges associated with system design for speech-based activity recognition in complex team-based work settings.

Original language	English (US)
Title of host publication	CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450359719
DOIs	https://doi.org/10.1145/3290607.3312983
State	Published - May 2 2019
Event	2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019 - Glasgow, United Kingdom Duration: May 4 2019 → May 9 2019

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019
Country	United Kingdom
City	Glasgow
Period	5/4/19 → 5/9/19

All Science Journal Classification (ASJC) codes

Software
Human-Computer Interaction
Computer Graphics and Computer-Aided Design

Keywords

Activity recognition
Decision support
Emergency medicine
Narrative schema
Speech analysis
Speech modeling

Access to Document

10.1145/3290607.3312983

Fingerprint Dive into the research topics of 'Assessing the feasibility of speech-based activity recognition in dynamic medical settings'. Together they form a unique fingerprint.

Cite this

Jagannath, S., Sarcevic, A., Kamireddi, N., & Marsic, I. (2019). Assessing the feasibility of speech-based activity recognition in dynamic medical settings. In CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems [3312983] (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3290607.3312983

Jagannath, Swathi ; Sarcevic, Aleksandra ; Kamireddi, Neha ; Marsic, Ivan. / Assessing the feasibility of speech-based activity recognition in dynamic medical settings. CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2019. (Conference on Human Factors in Computing Systems - Proceedings).

@inproceedings{3fda98c117034f47b31df08b8971a963,

title = "Assessing the feasibility of speech-based activity recognition in dynamic medical settings",

abstract = "We describe an experiment conducted with three domain experts to understand how well they can recognize types and performance stages of activities using speech data transcribed from verbal communications during dynamic medical teamwork. The insights gained from this experiment will inform the design of an automatic activity recognition system to alert medical teams to process deviations in real time. We contribute to the literature by (1) characterizing how domain experts perceive the dynamics of activity-related speech, and (2) identifying the challenges associated with system design for speech-based activity recognition in complex team-based work settings.",

keywords = "Activity recognition, Decision support, Emergency medicine, Narrative schema, Speech analysis, Speech modeling",

author = "Swathi Jagannath and Aleksandra Sarcevic and Neha Kamireddi and Ivan Marsic",

year = "2019",

month = may,

day = "2",

doi = "10.1145/3290607.3312983",

language = "English (US)",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

booktitle = "CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems",

note = "2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019 ; Conference date: 04-05-2019 Through 09-05-2019",

}

Jagannath, S, Sarcevic, A, Kamireddi, N & Marsic, I 2019, Assessing the feasibility of speech-based activity recognition in dynamic medical settings. in CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems., 3312983, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019, Glasgow, United Kingdom, 5/4/19. https://doi.org/10.1145/3290607.3312983

Assessing the feasibility of speech-based activity recognition in dynamic medical settings. / Jagannath, Swathi; Sarcevic, Aleksandra; Kamireddi, Neha; Marsic, Ivan.

CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2019. 3312983 (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Assessing the feasibility of speech-based activity recognition in dynamic medical settings

AU - Jagannath, Swathi

AU - Sarcevic, Aleksandra

AU - Kamireddi, Neha

AU - Marsic, Ivan

PY - 2019/5/2

Y1 - 2019/5/2

N2 - We describe an experiment conducted with three domain experts to understand how well they can recognize types and performance stages of activities using speech data transcribed from verbal communications during dynamic medical teamwork. The insights gained from this experiment will inform the design of an automatic activity recognition system to alert medical teams to process deviations in real time. We contribute to the literature by (1) characterizing how domain experts perceive the dynamics of activity-related speech, and (2) identifying the challenges associated with system design for speech-based activity recognition in complex team-based work settings.

AB - We describe an experiment conducted with three domain experts to understand how well they can recognize types and performance stages of activities using speech data transcribed from verbal communications during dynamic medical teamwork. The insights gained from this experiment will inform the design of an automatic activity recognition system to alert medical teams to process deviations in real time. We contribute to the literature by (1) characterizing how domain experts perceive the dynamics of activity-related speech, and (2) identifying the challenges associated with system design for speech-based activity recognition in complex team-based work settings.

KW - Activity recognition

KW - Decision support

KW - Emergency medicine

KW - Narrative schema

KW - Speech analysis

KW - Speech modeling

UR - http://www.scopus.com/inward/record.url?scp=85067310173&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067310173&partnerID=8YFLogxK

U2 - 10.1145/3290607.3312983

DO - 10.1145/3290607.3312983

M3 - Conference contribution

AN - SCOPUS:85067310173

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

T2 - 2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019

Y2 - 4 May 2019 through 9 May 2019

ER -

Jagannath S, Sarcevic A, Kamireddi N, Marsic I. Assessing the feasibility of speech-based activity recognition in dynamic medical settings. In CHI EA 2019 - Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2019. 3312983. (Conference on Human Factors in Computing Systems - Proceedings). https://doi.org/10.1145/3290607.3312983