人工智能课件AI2014S_Chap08ContinuousLatentVariables幻灯片.pptxVIP

Artificial Intelligence Continuous Latent VariablesDonghui WangAI Institute@ZJU2014.4ContentsPrincipal Component Analysis (PCA)Probabilistic PCAKernel PCANonlinear Latent Variable ModelsReferences:Bishop. “Pattern Recognition and Machine Learning”, Chapter 12. 2006.Principal Component Analysis (PCA)What’s PCA?PCA - Principal Component Analysis Karhunen-Loève transformTwo commonly used definitions:The orthogonal projection of the data onto a lower dimensional linear space, known as the principal subspace, such that the variance of the projected data is maximized.The linear projection that minimizes the average projection cost, defined as the mean squared distance between the data points and their projections.Why PCA?Dimensionality reductionAvoid the curse of dimensionalityLossy data compressionFeature extractionData visualizationHow to visualize high-dimensional data?…Why PCA?Dimensionality reductionAvoid the curse of dimensionalityLossy data compressionFeature extractionData visualizationHow to visualize high-dimensional data?…Maximum variance formulationMaximizing it.The variance will be a maximum when we set u1 equal to the eigenvector having the largest eigenvalue λ1.Consider a data set of observations {xn} where n=1,…,N, and xn is a D-dimensional vector.Goal: project the data onto a M-dimensional subspace (MD) while maximizing the variance of the projected data.M=1:define the direction of this space using a D-dimensional unit vector u1 and uT1 u1 =1.M1:Optimal linear projection is now defined by the M eigenvectors of the data covariance matrix S corresponding to the M largest eigenvalues.Minimum-error formulationMinimizing it.Introduce a complete orthonormal set of D-dimensional basis vector {ui} where i=1,…,D that satisfy uTiuj =δij. For a data set of observations {xn} , we haveGoal: approximate xn using a representation involving a restricted number MD of variables corresponding to a projection onto a lower-dimensional.Applications of PCAData compression (off