Dataless Model Selection with the Deep Frame Potential
  Choosing a deep neural network architecture is a fundamental problem in
applications that require balancing performance and parameter efficiency.
Standard approaches rely on ad-hoc engineering or computationally expensive
validation on a specific dataset. We instead attempt to quantify networks by
their intrinsic capacity for unique and robust representations, enabling
efficient architecture comparisons without requiring any data. Building upon
theoretical connections between deep learning and sparse approximation, we
propose the deep frame potential: a measure of coherence that is approximately
related to representation stability but has minimizers that depend only on
network structure. This provides a framework for jointly quantifying the
contributions of architectural hyper-parameters such as depth, width, and skip
connections. We validate its use as a criterion for model selection and
demonstrate correlation with generalization error on a variety of common
residual and densely connected network architectures.
