Fisher and Kernel Fisher Discriminant Analysis: Tutorial
  This is a detailed tutorial paper which explains the Fisher discriminant
Analysis (FDA) and kernel FDA. We start with projection and reconstruction.
Then, one- and multi-dimensional FDA subspaces are covered. Scatters in two-
and then multi-classes are explained in FDA. Then, we discuss on the rank of
the scatters and the dimensionality of the subspace. A real-life example is
also provided for interpreting FDA. Then, possible singularity of the scatter
is discussed to introduce robust FDA. PCA and FDA directions are also compared.
We also prove that FDA and linear discriminant analysis are equivalent. Fisher
forest is also introduced as an ensemble of fisher subspaces useful for
handling data with different features and dimensionality. Afterwards, kernel
FDA is explained for both one- and multi-dimensional subspaces with both two-
and multi-classes. Finally, some simulations are performed on AT&T face dataset
to illustrate FDA and compare it with PCA.
