Adaptive Random SubSpace Learning (RSSL) Algorithm for Prediction
  We present a novel adaptive random subspace learning algorithm (RSSL) for
prediction purpose. This new framework is flexible where it can be adapted with
any learning technique. In this paper, we tested the algorithm for regression
and classification problems. In addition, we provide a variety of weighting
schemes to increase the robustness of the developed algorithm. These different
wighting flavors were evaluated on simulated as well as on real-world data sets
considering the cases where the ratio between features (attributes) and
instances (samples) is large and vice versa. The framework of the new algorithm
consists of many stages: first, calculate the weights of all features on the
data set using the correlation coefficient and F-statistic statistical
measurements. Second, randomly draw n samples with replacement from the data
set. Third, perform regular bootstrap sampling (bagging). Fourth, draw without
replacement the indices of the chosen variables. The decision was taken based
on the heuristic subspacing scheme. Fifth, call base learners and build the
model. Sixth, use the model for prediction purpose on test set of the data. The
results show the advancement of the adaptive RSSL algorithm in most of the
cases compared with the synonym (conventional) machine learning algorithms.
