Novel Prediction Techniques Based on Clusterwise Linear Regression
  In this paper we explore different regression models based on Clusterwise
Linear Regression (CLR). CLR aims to find the partition of the data into $k$
clusters, such that linear regressions fitted to each of the clusters minimize
overall mean squared error on the whole data. The main obstacle preventing to
use found regression models for prediction on the unseen test points is the
absence of a reasonable way to obtain CLR cluster labels when the values of
target variable are unknown. In this paper we propose two novel approaches on
how to solve this problem. The first approach, predictive CLR builds a separate
classification model to predict test CLR labels. The second approach,
constrained CLR utilizes a set of user-specified constraints that enforce
certain points to go to the same clusters. Assuming the constraint values are
known for the test points, they can be directly used to assign CLR labels. We
evaluate these two approaches on three UCI ML datasets as well as on a large
corpus of health insurance claims. We show that both of the proposed algorithms
significantly improve over the known CLR-based regression methods. Moreover,
predictive CLR consistently outperforms linear regression and random forest,
and shows comparable performance to support vector regression on UCI ML
datasets. The constrained CLR approach achieves the best performance on the
health insurance dataset, while enjoying only $\approx 20$ times increased
computational time over linear regression.
