A Pragmatic Coded Modulation Scheme for High-Spectral-Efficiency   Fiber-Optic Communications
  A pragmatic coded modulation system is presented that incorporates signal shaping and exploits the excellent performance and efficient high-speed decoding architecture of staircase codes. Reliable communication within 0.62 bits/s/Hz of the estimated capacity (per polarization) of a system with L=2000 km is provided by the proposed system, with an error floor below 1E-20. Also, it is shown that digital backpropagation increases the achievable spectral efficiencies---relative to linear equalization---by 0.55 to 0.75 bits/s/Hz per polarization. 
Benjamin P. Smith and Frank R. Kschischang
cs.IT math.IT