CausalSim: Unbiased Trace-Driven Network Simulation
  Evaluating the real-world performance of network protocols is challenging.
Randomized control trials (RCT) are expensive and inaccessible to most, while
simulators fail to capture complex behaviors in real networks. We present
CausalSim, a trace-driven counterfactual simulator for network protocols that
addresses this challenge. Counterfactual simulation aims to predict what would
happen using different protocols under the same conditions as a given trace.
This is complicated due to the bias introduced by the protocols used during
trace collection. CausalSim uses traces from an initial RCT under a set of
protocols to learn a causal network model, effectively removing the biases
present in the data. Key to CausalSim is mapping the task of counterfactual
simulation to that of tensor completion with extremely sparse observations.
Through an adversarial neural network training technique that exploits
distributional invariances that are present in training data coming from an
RCT, CausalSim enables a novel tensor completion method despite the sparsity of
observations. Our extensive evaluation of CausalSim on both real and synthetic
datasets and two use cases, including more than nine months of real data from
the Puffer video streaming system, shows that it provides accurate
counterfactual predictions, reducing prediction error by 44% and 53% on average
compared to expert-designed and supervised learning baselines.
