Range-Net: A High Precision Streaming SVD for Big Data Applications
  In a Big Data setting computing the dominant SVD factors is restrictive due
to the main memory requirements. Recently introduced streaming Randomized SVD
schemes work under the restrictive assumption that the singular value spectrum
of the data has exponential decay. This is seldom true for any practical data.
Although these methods are claimed to be applicable to scientific computations
due to associated tail-energy error bounds, the approximation errors in the
singular vectors and values are high when the aforementioned assumption does
not hold. Furthermore from a practical perspective, oversampling can still be
memory intensive or worse can exceed the feature dimension of the data. To
address these issues, we present Range-Net as an alternative to randomized SVD
that satisfies the tail-energy lower bound given by Eckart-Young-Mirsky (EYM)
theorem. Range-Net is a deterministic two-stage neural optimization approach
with random initialization, where the main memory requirement depends
explicitly on the feature dimension and desired rank, independent of the sample
dimension. The data samples are read in a streaming setting with the network
minimization problem converging to the desired rank-r approximation. Range-Net
is fully interpretable where all the network outputs and weights have a
specific meaning. We provide theoretical guarantees that Range-Net extracted
SVD factors satisfy EYM tail-energy lower bound at machine precision. Our
numerical experiments on real data at various scales confirms this bound. A
comparison against the state of the art streaming Randomized SVD shows that
Range-Net accuracy is better by six orders of magnitude while being memory
efficient.
