An $l_1$-oracle inequality for the Lasso in mixture-of-experts
  regression models
  Mixture-of-experts (MoE) models are a popular framework for modeling
heterogeneity in data, for both regression and classification problems in
statistics and machine learning, due to their flexibility and the abundance of
available statistical estimation and model choice tools. Such flexibility comes
from allowing the mixture weights (or gating functions) in the MoE model to
depend on the explanatory variables, along with the experts (or component
densities). This permits the modeling of data arising from more complex data
generating processes when compared to the classical finite mixtures and finite
mixtures of regression models, whose mixing parameters are independent of the
covariates. The use of MoE models in a high-dimensional setting, when the
number of explanatory variables can be much larger than the sample size, is
challenging from a computational point of view, and in particular from a
theoretical point of view, where the literature is still lacking results for
dealing with the curse of dimensionality, for both the statistical estimation
and feature selection problems. We consider the finite MoE model with soft-max
gating functions and Gaussian experts for high-dimensional regression on
heterogeneous data, and its $l_1$-regularized estimation via the Lasso. We
focus on the Lasso estimation properties rather than its feature selection
properties. We provide a lower bound on the regularization parameter of the
Lasso function that ensures an $l_1$-oracle inequality satisfied by the Lasso
estimator according to the Kullback--Leibler loss.
