Reward-Based 1-bit Compressed Federated Distillation on Blockchain
  The recent advent of various forms of Federated Knowledge Distillation (FD)
paves the way for a new generation of robust and communication-efficient
Federated Learning (FL), where mere soft-labels are aggregated, rather than
whole gradients of Deep Neural Networks (DNN) as done in previous FL schemes.
This security-per-design approach in combination with increasingly performant
Internet of Things (IoT) and mobile devices opens up a new realm of
possibilities to utilize private data from industries as well as from
individuals as input for artificial intelligence model training. Yet in
previous FL systems, lack of trust due to the imbalance of power between
workers and a central authority, the assumption of altruistic worker
participation and the inability to correctly measure and compare contributions
of workers hinder this technology from scaling beyond small groups of already
entrusted entities towards mass adoption. This work aims to mitigate the
aforementioned issues by introducing a novel decentralized federated learning
framework where heavily compressed 1-bit soft-labels, resembling 1-hot label
predictions, are aggregated on a smart contract. In a context where workers'
contributions are now easily comparable, we modify the Peer Truth Serum for
Crowdsourcing mechanism (PTSC) for FD to reward honest participation based on
peer consistency in an incentive compatible fashion. Due to heavy reductions of
both computational complexity and storage, our framework is a fully
on-blockchain FL system that is feasible on simple smart contracts and
therefore blockchain agnostic. We experimentally test our new framework and
validate its theoretical properties.
