The Importance of Being Earnest: Performance of Modulation
  Classification for Real RF Signals
  Digital modulation classification (DMC) can be highly valuable for equipping
radios with increased spectrum awareness in complex emerging wireless networks.
However, as the existing literature is overwhelmingly based on theoretical or
simulation results, it is unclear how well DMC performs in practice. In this
paper we study the performance of DMC in real-world wireless networks, using an
extensive RF signal dataset of 250,000 over-the-air transmissions with
heterogeneous transceiver hardware and co-channel interference. Our results
show that DMC can achieve a high classification accuracy even under the
challenging real-world conditions of modulated co-channel interference and
low-grade hardware. However, this only holds if the training dataset fully
captures the variety of interference and hardware types in the real radio
environment; otherwise, the DMC performance deteriorates significantly. Our
work has two important engineering implications. First, it shows that it is not
straightforward to exchange learned classifier models among dissimilar radio
environments and devices in practice. Second, our analysis suggests that the
key missing link for real-world deployment of DMC is designing signal features
that generalize well to diverse wireless network scenarios. We are making our
RF signal dataset publicly available as a step towards a unified framework for
realistic DMC evaluation.
