CVPR 2011 workshop on gesture
recognition
Advances
in Phonetics-based Sub-Unit Modeling for Transcription Alignment and
Sign Language Recognition
Vassilis
Pitsikalis, Stavros Theodorakis, Christian Vogler, Petros Maragos
National
Technical University of Athens and Institute for Language and Speech
Processing, Athens, Greece
We explore novel directions for incorporating phonetic
transcriptions into sub-unit based statistical models for sign
language recognition. First, we employ a new symbolic processing
approach for converting sign language annotations,
based on HamNoSys symbols, into structured sequences
of labels according to the Posture-Detention-Transition-
Steady Shift phonetic model. Next, we exploit these labels,
and their correspondence with visual features to construct
phonetics-based statistical sub-unit models. We also
align these sequences, via the statistical sub-unit construction
and decoding, to the visual data to extract time boundary
information that they would lack otherwise. The resulting
phonetic sub-units offer new perspectives for sign language
analysis, phonetic modeling, and automatic recognition.
We evaluate this approach via sign language recognition
experiments on an extended Lemmas Corpus of Greek
Sign Language, which results not only in improved performance
compared to pure data-driven approaches, but also
in meaningful phonetic sub-unit models that can be further
exploited in interdisciplinary sign language analysis.