Towards Open-World Feature Extrapolation: An Inductive Graph Learning
  Approach
  We target open-world feature extrapolation problem where the feature space of
input data goes through expansion and a model trained on partially observed
features needs to handle new features in test data without further retraining.
The problem is of much significance for dealing with features incrementally
collected from different fields. To this end, we propose a new learning
paradigm with graph representation and learning. Our framework contains two
modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model
takes features as input and outputs predicted labels; 2) a graph neural network
as an upper model learns to extrapolate embeddings for new features via message
passing over a feature-data graph built from observed data. Based on our
framework, we design two training strategies, a self-supervised approach and an
inductive learning approach, to endow the model with extrapolation ability and
alleviate feature-level over-fitting. We also provide theoretical analysis on
the generalization error on test data with new features, which dissects the
impact of training features and algorithms on generalization performance. Our
experiments over several classification datasets and large-scale advertisement
click prediction datasets demonstrate that our model can produce effective
embeddings for unseen features and significantly outperforms baseline methods
that adopt KNN and local aggregation.
