Leveraging Audio Signals for Early Recognition of Inattentive Driving with Smartphones

Xiangyu Xu, Jiadi Yu, Yingying Chen, Yanmin Zhu, Shiyou Qian, Minglu Li

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Real-time driving behavior monitoring is a corner stone to improve driving safety. Most of the existing studies on driving behavior monitoring using smartphones only provide detection results after an abnormal driving behavior is finished, not sufficient for driver alerting and avoiding car accidents. In this paper, we leverage built-in audio devices on smartphones to realize early recognition of inattentive driving events including Fetching Forward, Picking up Drops, Turning Back, and Eating or Drinking. Through empirical studies of driving traces collected in real driving environments, we find that each type of inattentive driving event exhibits unique patterns on Doppler profiles of audio signals. This enables us to develop an Early Recognition system, ER, which can recognize inattentive driving events at an early stage and alert drivers timely. ER employs machine learning methods to first generate binary classifiers for every pair of inattentive driving events, and then develops a modified vote mechanism to form a multi-classifier for all four types of inattentive driving events, for atypical inattentive driving events along with other driving behaviors. It next turns the multi-classifier into a gradient model forest to achieve early recognition of inattentive driving. Through extensive experiments with eight volunteers driving for about two months, ER can achieve an average total accuracy of 94.80 percent for inattentive driving recognition and recognize over 80 percent inattentive driving events before the event is 50 percent finished.

Original languageEnglish (US)
Pages (from-to)1553-1567
Number of pages15
JournalIEEE Transactions on Mobile Computing
Volume17
Issue number7
DOIs
StatePublished - Jul 1 2018

Fingerprint

Smartphones
Classifiers
Monitoring
Learning systems
Accidents
Railroad cars
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Keywords

  • Early recognition
  • audio signal sensing
  • inattentive driving behaviors
  • smartphone sensors

Cite this

Xu, Xiangyu ; Yu, Jiadi ; Chen, Yingying ; Zhu, Yanmin ; Qian, Shiyou ; Li, Minglu. / Leveraging Audio Signals for Early Recognition of Inattentive Driving with Smartphones. In: IEEE Transactions on Mobile Computing. 2018 ; Vol. 17, No. 7. pp. 1553-1567.
@article{843041d936244343a9578511a35d1a9e,
title = "Leveraging Audio Signals for Early Recognition of Inattentive Driving with Smartphones",
abstract = "Real-time driving behavior monitoring is a corner stone to improve driving safety. Most of the existing studies on driving behavior monitoring using smartphones only provide detection results after an abnormal driving behavior is finished, not sufficient for driver alerting and avoiding car accidents. In this paper, we leverage built-in audio devices on smartphones to realize early recognition of inattentive driving events including Fetching Forward, Picking up Drops, Turning Back, and Eating or Drinking. Through empirical studies of driving traces collected in real driving environments, we find that each type of inattentive driving event exhibits unique patterns on Doppler profiles of audio signals. This enables us to develop an Early Recognition system, ER, which can recognize inattentive driving events at an early stage and alert drivers timely. ER employs machine learning methods to first generate binary classifiers for every pair of inattentive driving events, and then develops a modified vote mechanism to form a multi-classifier for all four types of inattentive driving events, for atypical inattentive driving events along with other driving behaviors. It next turns the multi-classifier into a gradient model forest to achieve early recognition of inattentive driving. Through extensive experiments with eight volunteers driving for about two months, ER can achieve an average total accuracy of 94.80 percent for inattentive driving recognition and recognize over 80 percent inattentive driving events before the event is 50 percent finished.",
keywords = "Early recognition, audio signal sensing, inattentive driving behaviors, smartphone sensors",
author = "Xiangyu Xu and Jiadi Yu and Yingying Chen and Yanmin Zhu and Shiyou Qian and Minglu Li",
year = "2018",
month = "7",
day = "1",
doi = "10.1109/TMC.2017.2772253",
language = "English (US)",
volume = "17",
pages = "1553--1567",
journal = "IEEE Transactions on Mobile Computing",
issn = "1536-1233",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "7",

}

Leveraging Audio Signals for Early Recognition of Inattentive Driving with Smartphones. / Xu, Xiangyu; Yu, Jiadi; Chen, Yingying; Zhu, Yanmin; Qian, Shiyou; Li, Minglu.

In: IEEE Transactions on Mobile Computing, Vol. 17, No. 7, 01.07.2018, p. 1553-1567.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Leveraging Audio Signals for Early Recognition of Inattentive Driving with Smartphones

AU - Xu, Xiangyu

AU - Yu, Jiadi

AU - Chen, Yingying

AU - Zhu, Yanmin

AU - Qian, Shiyou

AU - Li, Minglu

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Real-time driving behavior monitoring is a corner stone to improve driving safety. Most of the existing studies on driving behavior monitoring using smartphones only provide detection results after an abnormal driving behavior is finished, not sufficient for driver alerting and avoiding car accidents. In this paper, we leverage built-in audio devices on smartphones to realize early recognition of inattentive driving events including Fetching Forward, Picking up Drops, Turning Back, and Eating or Drinking. Through empirical studies of driving traces collected in real driving environments, we find that each type of inattentive driving event exhibits unique patterns on Doppler profiles of audio signals. This enables us to develop an Early Recognition system, ER, which can recognize inattentive driving events at an early stage and alert drivers timely. ER employs machine learning methods to first generate binary classifiers for every pair of inattentive driving events, and then develops a modified vote mechanism to form a multi-classifier for all four types of inattentive driving events, for atypical inattentive driving events along with other driving behaviors. It next turns the multi-classifier into a gradient model forest to achieve early recognition of inattentive driving. Through extensive experiments with eight volunteers driving for about two months, ER can achieve an average total accuracy of 94.80 percent for inattentive driving recognition and recognize over 80 percent inattentive driving events before the event is 50 percent finished.

AB - Real-time driving behavior monitoring is a corner stone to improve driving safety. Most of the existing studies on driving behavior monitoring using smartphones only provide detection results after an abnormal driving behavior is finished, not sufficient for driver alerting and avoiding car accidents. In this paper, we leverage built-in audio devices on smartphones to realize early recognition of inattentive driving events including Fetching Forward, Picking up Drops, Turning Back, and Eating or Drinking. Through empirical studies of driving traces collected in real driving environments, we find that each type of inattentive driving event exhibits unique patterns on Doppler profiles of audio signals. This enables us to develop an Early Recognition system, ER, which can recognize inattentive driving events at an early stage and alert drivers timely. ER employs machine learning methods to first generate binary classifiers for every pair of inattentive driving events, and then develops a modified vote mechanism to form a multi-classifier for all four types of inattentive driving events, for atypical inattentive driving events along with other driving behaviors. It next turns the multi-classifier into a gradient model forest to achieve early recognition of inattentive driving. Through extensive experiments with eight volunteers driving for about two months, ER can achieve an average total accuracy of 94.80 percent for inattentive driving recognition and recognize over 80 percent inattentive driving events before the event is 50 percent finished.

KW - Early recognition

KW - audio signal sensing

KW - inattentive driving behaviors

KW - smartphone sensors

UR - http://www.scopus.com/inward/record.url?scp=85034227673&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034227673&partnerID=8YFLogxK

U2 - 10.1109/TMC.2017.2772253

DO - 10.1109/TMC.2017.2772253

M3 - Article

AN - SCOPUS:85034227673

VL - 17

SP - 1553

EP - 1567

JO - IEEE Transactions on Mobile Computing

JF - IEEE Transactions on Mobile Computing

SN - 1536-1233

IS - 7

ER -