Discriminative Training of Subspace Gaussian Mixture.pdf

D.-S. Huang et al. (Eds.): ICIC 2010, LNCS 6215, pp. 213–221, 2010. ? Springer-Verlag Berlin Heidelberg 2010 Discriminative Training of Subspace Gaussian Mixture Model for Pattern Classification Xiao-Hua Liu and Cheng-Lin Liu National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences 95 Zhongguancun East Road, Beijing 100190, P.R. China {xhliu,liucl}@ Abstract. The Gaussian mixture model (GMM) has been widely used in pattern recognition problems for clustering and probability density estimation. For pat- tern classification, however, the GMM has to consider two issues: model struc- ture in high-dimensional space and discriminative training for optimizing the decision boundary. In this paper, we propose a classification method using subspace GMM density model and discriminative training. During discrimina- tive training under the minimum classification error (MCE) criterion, both the GMM parameters and the subspace parameters are optimized discriminatively. Our experimental results on the MNIST handwritten digit data and UCI datasets demonstrate the superior classification performance of the proposed method. Keywords: Subspace GMM, EM algorithm, Discriminative training, MCE. 1 Introduction The Gaussian mixture model (GMM) is widely used in pattern recognition problems for clustering, probability density estimation and classification. Many methods have been proposed for GMM parameter estimation and model selection (e.g. [1][2]). De- spite the capability of GMM to approximate arbitrary distributions, the precise density estimation requires a large number of training samples, especially in high-dimensional space (say, dimensionality over 20). Researchers have proposed structure-constrained GMMs for high-dimensional data, such as diagonal covariance, tied covariance, semi-tied covariance [3], and GMM in subspace [4-5]. On the other hand, the GMM is a generative model, with parameters estimated for