On Optimal TCM Encoders
  An asymptotically optimal trellis-coded modulation (TCM) encoder requires the joint design of the encoder and the binary labeling of the constellation. Since analytical approaches are unknown, the only available solution is to perform an exhaustive search over the encoder and the labeling. For large constellation sizes and/or many encoder states, however, an exhaustive search is unfeasible. Traditional TCM designs overcome this problem by using a labeling that follows the set-partitioning principle and by performing an exhaustive search over the encoders. In this paper we study binary labelings for TCM and show how they can be grouped into classes, which considerably reduces the search space in a joint design. For 8-ary constellations, the number of different binary labelings that must be tested is reduced from 8!=40320 to 240. For the particular case of an 8-ary pulse amplitude modulation constellation, this number is further reduced to 120 and for 8-ary phase shift keying to only 30. An algorithm to generate one labeling in each class is also introduced. Asymptotically optimal TCM encoders are tabulated which are up to 0.3 dB better than the previously best known encoders. 
Alex Alvarado, Alexandre Graell i Amat, Fredrik Brannstrom, Erik   Agrell
cs.IT math.IT