Deriving a Quantitative Relationship Between Resolution and Human
  Classification Error
  For machine learning perception problems, human-level classification
performance is used as an estimate of top algorithm performance. Thus, it is
important to understand as precisely as possible the factors that impact
human-level performance. Knowing this 1) provides a benchmark for model
performance, 2) tells a project manager what type of data to obtain for human
labelers in order to get accurate labels, and 3) enables ground-truth
analysis--largely conducted by humans--to be carried out smoothly. In this
empirical study, we explored the relationship between resolution and human
classification performance using the MNIST data set down-sampled to various
resolutions. The quantitative heuristic we derived could prove useful for
predicting machine model performance, predicting data storage requirements, and
saving valuable resources in the deployment of machine learning projects. It
also has the potential to be used in a wide variety of fields such as remote
sensing, medical imaging, scientific imaging, and astronomy.
