Discriminative Learning of Similarity and Group Equivariant
  Representations
  One of the most fundamental problems in machine learning is to compare
examples: Given a pair of objects we want to return a value which indicates
degree of (dis)similarity. Similarity is often task specific, and pre-defined
distances can perform poorly, leading to work in metric learning. However,
being able to learn a similarity-sensitive distance function also presupposes
access to a rich, discriminative representation for the objects at hand. In
this dissertation we present contributions towards both ends. In the first part
of the thesis, assuming good representations for the data, we present a
formulation for metric learning that makes a more direct attempt to optimize
for the k-NN accuracy as compared to prior work. We also present extensions of
this formulation to metric learning for kNN regression, asymmetric similarity
learning and discriminative learning of Hamming distance. In the second part,
we consider a situation where we are on a limited computational budget i.e.
optimizing over a space of possible metrics would be infeasible, but access to
a label aware distance metric is still desirable. We present a simple, and
computationally inexpensive approach for estimating a well motivated metric
that relies only on gradient estimates, discussing theoretical and experimental
results. In the final part, we address representational issues, considering
group equivariant convolutional neural networks (GCNNs). Equivariance to
symmetry transformations is explicitly encoded in GCNNs; a classical CNN being
the simplest example. In particular, we present a SO(3)-equivariant neural
network architecture for spherical data, that operates entirely in Fourier
space, while also providing a formalism for the design of fully Fourier neural
networks that are equivariant to the action of any continuous compact group.
