Deep Learning for Joint Source-Channel Coding of Text
  We consider the problem of joint source and channel coding of structured data
such as natural language over a noisy channel. The typical approach to this
problem in both theory and practice involves performing source coding to first
compress the text and then channel coding to add robustness for the
transmission across the channel. This approach is optimal in terms of
minimizing end-to-end distortion with arbitrarily large block lengths of both
the source and channel codes when transmission is over discrete memoryless
channels. However, the optimality of this approach is no longer ensured for
documents of finite length and limitations on the length of the encoding. We
will show in this scenario that we can achieve lower word error rates by
developing a deep learning based encoder and decoder. While the approach of
separate source and channel coding would minimize bit error rates, our approach
preserves semantic information of sentences by first embedding sentences in a
semantic space where sentences closer in meaning are located closer together,
and then performing joint source and channel coding on these embeddings.
