Modulation Classification via Subspace Detection in MIMO Systems
  The problem of efficient modulation classification (MC) in multiple-input multiple-output (MIMO) systems is considered. Per-layer likelihood-based MC is proposed by employing subspace decomposition to partially decouple the transmitted streams. When detecting the modulation type of the stream of interest, a dense constellation is assumed on all remaining streams. The proposed classifier outperforms existing MC schemes at a lower complexity cost, and can be efficiently implemented in the context of joint MC and subspace data detection. 
Hadi Sarieddeen, Mohammad M. Mansour, and Ali Chehab
cs.IT math.IT