Hybrid Neural Coded Modulation: Design and Training Methods
  We propose a hybrid coded modulation scheme which composes of inner and outer
codes. The outer-code can be any standard binary linear code with efficient
soft decoding capability (e.g. low-density parity-check (LDPC) codes). The
inner code is designed using a deep neural network (DNN) which takes the
channel coded bits and outputs modulated symbols. For training the DNN, we
propose to use a loss function that is inspired by the generalized mutual
information. The resulting constellations are shown to outperform the
conventional quadrature amplitude modulation (QAM) based coding scheme for
modulation order 16 and 64 with 5G standard LDPC codes.
