Conditional Sparse Coding and Grouped Multivariate Regression
  We study the problem of multivariate regression where the data are naturally
grouped, and a regression matrix is to be estimated for each group. We propose
an approach in which a dictionary of low rank parameter matrices is estimated
across groups, and a sparse linear combination of the dictionary elements is
estimated to form a model within each group. We refer to the method as
conditional sparse coding since it is a coding procedure for the response
vectors Y conditioned on the covariate vectors X. This approach captures the
shared information across the groups while adapting to the structure within
each group. It exploits the same intuition behind sparse coding that has been
successfully developed in computer vision and computational neuroscience. We
propose an algorithm for conditional sparse coding, analyze its theoretical
properties in terms of predictive accuracy, and present the results of
simulation and brain imaging experiments that compare the new technique to
reduced rank regression.
