This is the latest version of this eprint.
Charles, J, Pfister, T, Everingham, M et al. (1 more author) (2013) Automatic and efficient human pose estimation for sign language videos. International Journal of Computer Vision, 110 (1). 70 - 90. ISSN 0920-5691
Abstract
We present a fully automatic arm and hand tracker that detects joint positions over continuous sign language video sequences of more than an hour in length. To achieve this, we make contributions in four areas: (i) we show that the overlaid signer can be separated from the background TV broadcast using co-segmentation over all frames with a layered model; (ii) we show that joint positions (shoulders, elbows, wrists) can be predicted per-frame using a random forest regressor given only this segmentation and a colour model; (iii) we show that the random forest can be trained from an existing semi-automatic, but computationally expensive, tracker; and, (iv) introduce an evaluator to assess whether the predicted joint positions are correct for each frame. The method is applied to 20 signing footage videos with changing background, challenging imaging conditions, and for different signers. Our framework outperforms the state-of-the-art long term tracker by Buehler et al. (International Journal of Computer Vision 95:180–197, 2011), does not require the manual annotation of that work, and, after automatic initialisation, performs tracking in real-time. We also achieve superior joint localisation results to those obtained using the pose estimation method of Yang and Ramanan (Proceedings of the IEEE conference on computer vision and pattern recognition, 2011).
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Keywords: | Sign language; human pose estimation; co-segmentation; random forest |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) > Artificial Intelligence & Biological Systems (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 17 Jun 2015 10:44 |
Last Modified: | 03 Nov 2016 10:49 |
Published Version: | http://dx.doi.org/10.1007/s11263-013-0672-6 |
Status: | Published |
Publisher: | Springer Verlag |
Identification Number: | 10.1007/s11263-013-0672-6 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:84576 |
Available Versions of this Item
-
Automatic and efficient human pose estimation for sign language videos. (deposited 27 Jun 2014 10:50)
- Automatic and efficient human pose estimation for sign language videos. (deposited 17 Jun 2015 10:44) [Currently Displayed]