Online Statistical Inference for Parameters Estimation with
  Linear-Equality Constraints
  Stochastic gradient descent (SGD) and projected stochastic gradient descent
(PSGD) are scalable algorithms to compute model parameters in unconstrained and
constrained optimization problems. In comparison with SGD, PSGD forces its
iterative values into the constrained parameter space via projection. From a
statistical point of view, this paper studies the limiting distribution of
PSGD-based estimate when the true parameters satisfy some linear-equality
constraints. Our theoretical findings reveal the role of projection played in
the uncertainty of the PSGD-based estimate. As a byproduct, we propose an
online hypothesis testing procedure to test the linear-equality constraints.
Simulation studies on synthetic data and an application to a real-world dataset
confirm our theory.
