Quantifying and Extrapolating Data Needs in Radio Frequency Machine
  Learning
  Understanding the relationship between training data and a model's
performance once deployed is a fundamental component in the application of
machine learning. While the model's deployed performance is dependent on
numerous variables within the scope of machine learning, beyond that of the
training data itself, the effect of the dataset is isolated in this work to
better understand the role training data plays in the problem. This work
examines a modulation classification problem in the Radio Frequency domain
space, attempting to answer the question of how much training data is required
to achieve a desired level of performance, but the procedure readily applies to
classification problems across modalities. By repurposing the metrics of
transfer potential developed within transfer learning an approach to bound data
quantity needs developed given a training approach and machine learning
architecture; this approach is presented as a means to estimate data quantity
requirements to achieve a target performance. While this approach will require
an initial dataset that is germane to the problem space to act as a target
dataset on which metrics are extracted, the goal is to allow for the initial
data to be orders of magnitude smaller than what is required for delivering a
system that achieves the desired performance. An additional benefit of the
techniques presented here is that the quality of different datasets can be
numerically evaluated and tied together with the quantity of data, and the
performance of the system.
