Uncheatable Machine Learning Inference
  Classification-as-a-Service (CaaS) is widely deployed today in machine
intelligence stacks for a vastly diverse set of applications including anything
from medical prognosis to computer vision tasks to natural language processing
to identity fraud detection. The computing power required for training complex
models on large datasets to perform inference to solve these problems can be
very resource-intensive. A CaaS provider may cheat a customer by fraudulently
bypassing expensive training procedures in favor of weaker, less
computationally-intensive algorithms which yield results of reduced quality.
Given a classification service supplier $S$, intermediary CaaS provider $P$
claiming to use $S$ as a classification backend, and customer $C$, our work
addresses the following questions: (i) how can $P$'s claim to be using $S$ be
verified by $C$? (ii) how might $S$ make performance guarantees that may be
verified by $C$? and (iii) how might one design a decentralized system that
incentivizes service proofing and accountability? To this end, we propose a
variety of methods for $C$ to evaluate the service claims made by $P$ using
probabilistic performance metrics, instance seeding, and steganography. We also
propose a method of measuring the robustness of a model using a blackbox
adversarial procedure, which may then be used as a benchmark or comparison to a
claim made by $S$. Finally, we propose the design of a smart contract-based
decentralized system that incentivizes service accountability to serve as a
trusted Quality of Service (QoS) auditor.
