Neural Communication Systems with Bandwidth-limited Channel
  Reliably transmitting messages despite information loss due to a noisy
channel is a core problem of information theory. One of the most important
aspects of real world communication, e.g. via wifi, is that it may happen at
varying levels of information transfer. The bandwidth-limited channel models
this phenomenon. In this study we consider learning coding with the
bandwidth-limited channel (BWLC). Recently, neural communication models such as
variational autoencoders have been studied for the task of source compression.
We build upon this work by studying neural communication systems with the BWLC.
Specifically,we find three modelling choices that are relevant under expected
information loss. First, instead of separating the sub-tasks of compression
(source coding) and error correction (channel coding), we propose to model both
jointly. Framing the problem as a variational learning problem, we conclude
that joint systems outperform their separate counterparts when coding is
performed by flexible learnable function approximators such as neural networks.
To facilitate learning, we introduce a differentiable and computationally
efficient version of the bandwidth-limited channel. Second, we propose a design
to model missing information with a prior, and incorporate this into the
channel model. Finally, sampling from the joint model is improved by
introducing auxiliary latent variables in the decoder. Experimental results
justify the validity of our design decisions through improved distortion and
FID scores.
