an assessment of hermite function based approximations of mutual information applied to independent component analysis埃尔米特函数的评估基于近似的互信息应用于独立分量分析.pdfVIP

Entropy 2008, 10, 745-756; DOI: 10.3390/ OPEN ACCESS entropy ISSN 1099-4300 /journal/entropy Article An Assessment of Hermite Function Based Approximations of Mutual Information Applied to Independent Component Analysis Julian Sorensen C3ID, DSTO, PO Box, Edinburgh, SA 5111, Australia, E-mail: Julian.Sorensen@.au Received: 23 May 2008 / Accepted: 28 November 2008 / Published: 4 December 2008 Abstract: At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estima- tion. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via a truncated series expansion whose coefficients are calculated from the observed data). While such estimators do not necessarily yield a valid density, which kernel density estimators do, they are faster to calculate than kernel density estimators, in particular for a modified version of Renyi’s entropy of order 2. In this paper, we compare the performance of ICA using Hermite series based estimates of Shannon’s and Renyi’s mutual information, to that of Gaussian kernel based estimates. The comparisons also include ICA using the RADICAL estimate of Shannon’s entropy and a FastICA estimate of neg-entropy. Keywords: ICA, nonparametric estimation, Hermite functions, kernel density estimation 1. Introduction Many of the techniques used to perform Independent Component Analysis (ICA) minimize an “infor- mation” measure. One of the pioneeri