Intrinsic dimension estimation of data by principal component analysis
  Estimating intrinsic dimensionality of data is a classic problem in pattern
recognition and statistics. Principal Component Analysis (PCA) is a powerful
tool in discovering dimensionality of data sets with a linear structure; it,
however, becomes ineffective when data have a nonlinear structure. In this
paper, we propose a new PCA-based method to estimate intrinsic dimension of
data with nonlinear structures. Our method works by first finding a minimal
cover of the data set, then performing PCA locally on each subset in the cover
and finally giving the estimation result by checking up the data variance on
all small neighborhood regions. The proposed method utilizes the whole data set
to estimate its intrinsic dimension and is convenient for incremental learning.
In addition, our new PCA procedure can filter out noise in data and converge to
a stable estimation with the neighborhood region size increasing. Experiments
on synthetic and real world data sets show effectiveness of the proposed
method.
