Stochastic Transformer Networks with Linear Competing Units: Application
  to end-to-end SL Translation
  Automating sign language translation (SLT) is a challenging real world
application. Despite its societal importance, though, research progress in the
field remains rather poor. Crucially, existing methods that yield viable
performance necessitate the availability of laborious to obtain gloss sequence
groundtruth. In this paper, we attenuate this need, by introducing an
end-to-end SLT model that does not entail explicit use of glosses; the model
only needs text groundtruth. This is in stark contrast to existing end-to-end
models that use gloss sequence groundtruth, either in the form of a modality
that is recognized at an intermediate model stage, or in the form of a parallel
output process, jointly trained with the SLT model. Our approach constitutes a
Transformer network with a novel type of layers that combines: (i) local
winner-takes-all (LWTA) layers with stochastic winner sampling, instead of
conventional ReLU layers, (ii) stochastic weights with posterior distributions
estimated via variational inference, and (iii) a weight compression technique
at inference time that exploits estimated posterior variance to perform
massive, almost lossless compression. We demonstrate that our approach can
reach the currently best reported BLEU-4 score on the PHOENIX 2014T benchmark,
but without making use of glosses for model training, and with a memory
footprint reduced by more than 70%.
