May 9th, 2014, 10:27 pm
Its not true that you cannot perform dimensionality reduction of uncorrelated series. PCA will do exactly that when presented with multivariate Gaussian data. Whether or not it is meaningful is another matter. But it will certainly tell you about the variance of the data in the relevant dimensions and this can be useful regardless of correlation structure. In fact covariance will tend to decrease the efficacy of PCA in which case one must resort to such methods as kernel PCA.PCA is the eigensystem of the covariance matrixThe eigenvalues thus determined are equivalent to the diagonal entries of the diagonalised covariance matrixHence, for a diagonal matrix, the eigenvalues adopt the values of the diagonal elements.Intuitively, if you have non-uniform diagonal elements (i.e. variances) your data defines a N dimensional sausage.PCA simply aligns N orthogonal axes with that sausage so as to maximise the overlap of sausage and axes.
Last edited by
neuroguy on May 9th, 2014, 10:00 pm, edited 1 time in total.