Fuzzy Tiling Activations: A Simple Approach to Learning Sparse
  Representations Online
  Recent work has shown that sparse representations -- where only a small
percentage of units are active -- can significantly reduce interference. Those
works, however, relied on relatively complex regularization or meta-learning
approaches, that have only been used offline in a pre-training phase. In this
work, we pursue a direction that achieves sparsity by design, rather than by
learning. Specifically, we design an activation function that produces sparse
representations deterministically by construction, and so is more amenable to
online training. The idea relies on the simple approach of binning, but
overcomes the two key limitations of binning: zero gradients for the flat
regions almost everywhere, and lost precision -- reduced discrimination -- due
to coarse aggregation. We introduce a Fuzzy Tiling Activation (FTA) that
provides non-negligible gradients and produces overlap between bins that
improves discrimination. We first show that FTA is robust under covariate shift
in a synthetic online supervised learning problem, where we can vary the level
of correlation and drift. Then we move to the deep reinforcement learning
setting and investigate both value-based and policy gradient algorithms that
use neural networks with FTAs, in classic discrete control and Mujoco
continuous control environments. We show that algorithms equipped with FTAs are
able to learn a stable policy faster without needing target networks on most
domains.
