Dimension-free Concentration Bounds on Hankel Matrices for Spectral
  Learning
  Learning probabilistic models over strings is an important issue for many
applications. Spectral methods propose elegant solutions to the problem of
inferring weighted automata from finite samples of variable-length strings
drawn from an unknown target distribution. These methods rely on a singular
value decomposition of a matrix $H_S$, called the Hankel matrix, that records
the frequencies of (some of) the observed strings. The accuracy of the learned
distribution depends both on the quantity of information embedded in $H_S$ and
on the distance between $H_S$ and its mean $H_r$. Existing concentration bounds
seem to indicate that the concentration over $H_r$ gets looser with the size of
$H_r$, suggesting to make a trade-off between the quantity of used information
and the size of $H_r$. We propose new dimension-free concentration bounds for
several variants of Hankel matrices. Experiments demonstrate that these bounds
are tight and that they significantly improve existing bounds. These results
suggest that the concentration rate of the Hankel matrix around its mean does
not constitute an argument for limiting its size.
