On the validity of kernel approximations for orthogonally-initialized
  neural networks
  In this note we extend kernel function approximation results for neural
networks with Gaussian-distributed weights to single-layer networks initialized
using Haar-distributed random orthogonal matrices (with possible rescaling).
This is accomplished using recent results from random matrix theory.
