CVPR 2011 workshop on gesture
recognition
Segmentation-robust representations,
matching, modeling for sign
language recognition
Sudeep
Sarkar
University of South Florida
Distinguishing true signs from transitional, extraneous movements made
by the signer as s/he moves from one sign to the next is a serious
hurdle in the design of continuous Sign Language recognition systems.
This problem is further compounded by the ambiguity of segmentation and
occlusions, resulting in propagation of errors to higher levels. This
talk will describe our experience with representations and matching
methods, particularly those that can handle errors in low-level
segmentation methods and uncertainties in segmentation of signs in
sentences. We have formulated a novel framework that can address
both these problems (i) using a nested level-building-based dynamic
programming approach, when there is dearth of training data, and (ii)
using a HMM-based approach generalized to handle multiple possible
observations, when we have statistical models of signs. We will also
discuss an automated approach to both extract and learn models for
continuous signs from continuous sentences in a weakly unsupervised
manner. This can help build training data for the recognition process.
Our publications that discuss these issues can be found at
http://marathon.csee.usf.edu/ASL/