Adaptive Shrinkage Estimation for Streaming Graphs
  Networks are a natural representation of complex systems across the sciences,
and higher-order dependencies are central to the understanding and modeling of
these systems. However, in many practical applications such as online social
networks, networks are massive, dynamic, and naturally streaming, where
pairwise interactions among vertices become available one at a time in some
arbitrary order. The massive size and streaming nature of these networks allow
only partial observation, since it is infeasible to analyze the entire network.
Under such scenarios, it is challenging to study the higher-order structural
and connectivity patterns of streaming networks. In this work, we consider the
fundamental problem of estimating the higher-order dependencies using adaptive
sampling. We propose a novel adaptive, single-pass sampling framework and
unbiased estimators for higher-order network analysis of large streaming
networks. Our algorithms exploit adaptive techniques to identify edges that are
highly informative for efficiently estimating the higher-order structure of
streaming networks from small sample data. We also introduce a novel
James-Stein shrinkage estimator to reduce the estimation error. Our approach is
fully analytic, computationally efficient, and can be incrementally updated in
a streaming setting. Numerical experiments on large networks show that our
approach is superior to baseline methods.
