High-Dimensional Bernoulli Autoregressive Process with Long-Range
  Dependence
  We consider the problem of estimating the parameters of a multivariate
Bernoulli process with auto-regressive feedback in the high-dimensional setting
where the number of samples available is much less than the number of
parameters. This problem arises in learning interconnections of networks of
dynamical systems with spiking or binary-valued data. We allow the process to
depend on its past up to a lag $p$, for a general $p \ge 1$, allowing for more
realistic modeling in many applications. We propose and analyze an
$\ell_1$-regularized maximum likelihood estimator (MLE) under the assumption
that the parameter tensor is approximately sparse. Rigorous analysis of such
estimators is made challenging by the dependent and non-Gaussian nature of the
process as well as the presence of the nonlinearities and multi-level feedback.
We derive precise upper bounds on the mean-squared estimation error in terms of
the number of samples, dimensions of the process, the lag $p$ and other key
statistical properties of the model. The ideas presented can be used in the
high-dimensional analysis of regularized $M$-estimators for other sparse
nonlinear and non-Gaussian processes with long-range dependence.
