Dynamic Feature Scaling for K-Nearest Neighbor Algorithm
  Nearest Neighbors Algorithm is a Lazy Learning Algorithm, in which the
algorithm tries to approximate the predictions with the help of similar
existing vectors in the training dataset. The predictions made by the K-Nearest
Neighbors algorithm is based on averaging the target values of the spatial
neighbors. The selection process for neighbors in the Hermitian space is done
with the help of distance metrics such as Euclidean distance, Minkowski
distance, Mahalanobis distance etc. A majority of the metrics such as Euclidean
distance are scale variant, meaning that the results could vary for different
range of values used for the features. Standard techniques used for the
normalization of scaling factors are feature scaling method such as Z-score
normalization technique, Min-Max scaling etc. Scaling methods uniformly assign
equal weights to all the features, which might result in a non-ideal situation.
This paper proposes a novel method to assign weights to individual feature with
the help of out of bag errors obtained from constructing multiple decision tree
models.
