Storage Fit Learning with Feature Evolvable Streams
  Feature evolvable learning has been widely studied in recent years where old
features will vanish and new features will emerge when learning with streams.
Conventional methods usually assume that a label will be revealed after
prediction at each time step. However, in practice, this assumption may not
hold whereas no label will be given at most time steps. A good solution is to
leverage the technique of manifold regularization to utilize the previous
similar data to assist the refinement of the online model. Nevertheless, this
approach needs to store all previous data which is impossible in learning with
streams that arrive sequentially in large volume. Thus we need a buffer to
store part of them. Considering that different devices may have different
storage budgets, the learning approaches should be flexible subject to the
storage budget limit. In this paper, we propose a new setting: Storage-Fit
Feature-Evolvable streaming Learning (SF$^2$EL) which incorporates the issue of
rarely-provided labels into feature evolution. Our framework is able to fit its
behavior to different storage budgets when learning with feature evolvable
streams with unlabeled data. Besides, both theoretical and empirical results
validate that our approach can preserve the merit of the original feature
evolvable learning i.e., can always track the best baseline and thus perform
well at any time step.
