Parallel hidden Markov models for American sign language recognition

Christian Vogler, Dimitris Metaxas

Research output: Contribution to journalConference articlepeer-review

165 Scopus citations


The major challenge that faces American Sign Language (ASL) recognition now is to develop methods that will scale well with increasing vocabulary size. Unlike in spoken languages, phonemes can occur simultaneously in ASL. The number of possible combinations of phonemes after enforcing linguistic constraints is approximately 5.5×108. Gesture recognition, which is less constrained than ASL recognition, suffers from the same problem. Thus, it is not feasible to train conventional hidden Markov models (HMMs) for large-scale ASL applications. Factorial HMMs and coupled HMMs are two extensions to HMMs that explicitly attempt to model several processes occurring in parallel. Unfortunately, they still require consideration of the combinations at training time. In this paper we present a novel approach to ASL recognition that aspires to being a solution to the scalability problems. It is based on parallel HMMs (PaHMMs), which model the parallel processes independently. Thus, they can also be trained independently, and do not require consideration of the different combinations at training time. We develop the recognition algorithm for PaHMMs and show that it runs in time polynomial in the number of states, and in time linear in the number of parallel processes. We run several experiments with a 22 sign vocabulary and demonstrate that PaHMMs can improve the robustness of HMM-based recognition even on a small scale. Thus, PaHMMs are a very promising general recognition scheme with applications in both gesture and ASL recognition.

Original languageEnglish (US)
Pages (from-to)116-122
Number of pages7
JournalProceedings of the IEEE International Conference on Computer Vision
StatePublished - 1999
Externally publishedYes
EventProceedings of the 1999 7th IEEE International Conference on Computer Vision (ICCV'99) - Kerkyra, Greece
Duration: Sep 20 1999Sep 27 1999

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Parallel hidden Markov models for American sign language recognition'. Together they form a unique fingerprint.

Cite this