Monotonic Differentiable Sorting Networks
  Differentiable sorting algorithms allow training with sorting and ranking
supervision, where only the ordering or ranking of samples is known. Various
methods have been proposed to address this challenge, ranging from optimal
transport-based differentiable Sinkhorn sorting algorithms to making classic
sorting networks differentiable. One problem of current differentiable sorting
methods is that they are non-monotonic. To address this issue, we propose a
novel relaxation of conditional swap operations that guarantees monotonicity in
differentiable sorting networks. We introduce a family of sigmoid functions and
prove that they produce differentiable sorting networks that are monotonic.
Monotonicity ensures that the gradients always have the correct sign, which is
an advantage in gradient-based optimization. We demonstrate that monotonic
differentiable sorting networks improve upon previous differentiable sorting
methods.
