Enable Traditional Laptops with Virtual Writing Capability Leveraging Acoustic Signals

Li Lu, Jian Liu, Jiadi Yu, Yingying Chen, Yanmin Zhu, Linghe Kong, Minglu Li

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Human-computer interaction through touch screens plays an increasingly important role in our daily lives. Besides smartphones and tablets, laptops are the most prevalent mobile devices for both work and leisure. To satisfy the requirements of some applications, it is desirable to re-equip a typical laptop with both handwriting and drawing capability. In this paper, we design a virtual writing tablet system, VPad, for traditional laptops without touch screens. VPad leverages two speakers and one microphone, which are available in most commodity laptops, to accurately track hand movements and recognize writing characters in the air without additional hardware. Specifically, VPad emits inaudible acoustic signals from two speakers in a laptop and then analyzes energy features and Doppler shifts of acoustic signals received by the microphone to track the trajectory of hand movements. Furthermore, we propose a state machine-based trajectory optimization method to correct the unexpected trajectory and employ a stroke direction sequence model based on probability estimation to recognize characters users write in the air. Experimental results show that VPad achieves the average error of 1.55 cm for trajectory tracking and the accuracy over 90% of character recognition merely through built-in audio devices on a laptop.

Original languageEnglish (US)
Pages (from-to)1814-1831
Number of pages18
JournalComputer Journal
Issue number12
StatePublished - Dec 1 2021

All Science Journal Classification (ASJC) codes

  • Computer Science(all)


  • acoustic signals
  • laptops
  • trajectory tracking
  • virtual writing


Dive into the research topics of 'Enable Traditional Laptops with Virtual Writing Capability Leveraging Acoustic Signals'. Together they form a unique fingerprint.

Cite this