SHEARer: Highly-Efficient Hyperdimensional Computing by
  Software-Hardware Enabled Multifold Approximation
  Hyperdimensional computing (HD) is an emerging paradigm for machine learning
based on the evidence that the brain computes on high-dimensional, distributed,
representations of data. The main operation of HD is encoding, which transfers
the input data to hyperspace by mapping each input feature to a hypervector,
accompanied by so-called bundling procedure that simply adds up the
hypervectors to realize encoding hypervector. Although the operations of HD are
highly parallelizable, the massive number of operations hampers the efficiency
of HD in embedded domain. In this paper, we propose SHEARer, an
algorithm-hardware co-optimization to improve the performance and energy
consumption of HD computing. We gain insight from a prudent scheme of
approximating the hypervectors that, thanks to inherent error resiliency of HD,
has minimal impact on accuracy while provides high prospect for hardware
optimization. In contrast to previous works that generate the encoding
hypervectors in full precision and then ex-post quantizing, we compute the
encoding hypervectors in an approximate manner that saves a significant amount
of resources yet affords high accuracy. We also propose a novel FPGA
implementation that achieves striking performance through massive parallelism
with low power consumption. Moreover, we develop a software framework that
enables training HD models by emulating the proposed approximate encodings. The
FPGA implementation of SHEARer achieves an average throughput boost of 104,904x
(15.7x) and energy savings of up to 56,044x (301x) compared to state-of-the-art
encoding methods implemented on Raspberry Pi 3 (GeForce GTX 1080 Ti) using
practical machine learning datasets.
