White Rose University Consortium logo
University of Leeds logo University of Sheffield logo York University logo

Efficient training algorithms for HMMs using incremental estimation

Gotoh, Y., Hochberg, M.M. and Silverman, H.F. (1998) Efficient training algorithms for HMMs using incremental estimation. IEEE Transactions on Speech and Audio Processing, 6 (6). pp. 539-548. ISSN 1063-6676


Download (227Kb)


Typically, parameter estimation for a hidden Markov model (HMM) is performed using an expectation-maximization (EM) algorithm with the maximum-likelihood (ML) criterion. The EM algorithm is an iterative scheme that is well-defined and numerically stable, but convergence may require a large number of iterations. For speech recognition systems utilizing large amounts of training material, this results in long training times. This paper presents an incremental estimation approach to speed-up the training of HMMs without any loss of recognition performance. The algorithm selects a subset of data from the training set, updates the model parameters based on the subset, and then iterates the process until convergence of the parameters. The advantage of this approach is a substantial increase in the number of iterations of the EM algorithm per training token, which leads to faster training. In order to achieve reliable estimation from a small fraction of the complete data set at each iteration, two training criteria are studied; ML and maximum a posteriori (MAP) estimation. Experimental results show that the training of the incremental algorithms is substantially faster than the conventional (batch) method and suffers no loss of recognition performance. Furthermore, the incremental MAP based training algorithm improves performance over the batch version

Item Type: Article
Copyright, Publisher and Additional Information: © Copyright 1998 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE
Institution: The University of Sheffield
Academic Units: The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield)
Depositing User: Repository Officer
Date Deposited: 24 Jan 2008 19:47
Last Modified: 04 Jun 2014 09:49
Published Version: http://dx.doi.org/10.1109/89.725320
Status: Published
Publisher: IEEE
Refereed: Yes
Identification Number: 10.1109/89.725320
URI: http://eprints.whiterose.ac.uk/id/eprint/3597

Actions (repository staff only: login required)