Low-loss connection of weight vectors: distribution-based approaches
  Recent research shows that sublevel sets of the loss surfaces of
overparameterized networks are connected, exactly or approximately. We describe
and compare experimentally a panel of methods used to connect two low-loss
points by a low-loss curve on this surface. Our methods vary in accuracy and
complexity. Most of our methods are based on "macroscopic" distributional
assumptions, and some are insensitive to the detailed properties of the points
to be connected. Some methods require a prior training of a "global connection
model" which can then be applied to any pair of points. The accuracy of the
method generally correlates with its complexity and sensitivity to the endpoint
detail.
