Dimensionality Reduction Techniques

Abstract

Dimensionality reduction technique involves finding out the transformation matrix that maps from the random vector in the higher dimensional space to the lower dimensional space. This is obtained by identifying the orthonormal basis using PCA, LDA, KLDA, and ICA. In PCA, the basis vectors are identified in the direction of the maximum variance of the data. PCA doesn’t care about the class index associated with the training vectors. This is circumvented using LDA. In this case, the distance between the vectors within the class is minimized and the distance between the centroids of various classes is made apart. After subjected to PCA, the data are made uncorrelated. These are further made independent by projecting the data using ICA basis. In almost all the techniques in pattern recognition techniques, the individual classes are assumed as Gaussian distributed. But in real-time applications, data aren’t Gaussian distributed. Thus the Gaussianity of the data can be minimizing the absolute of the kurtosis measured using the given data or by maximizing the neg entropy of the given data.