Bandana: Using Non-volatile Memory for Storing Deep Learning Models
  Typical large-scale recommender systems use deep learning models that are
stored on a large amount of DRAM. These models often rely on embeddings, which
consume most of the required memory. We present Bandana, a storage system that
reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as
the primary storage medium, with a small amount of DRAM as cache. The main
challenge in storing embeddings on NVM is its limited read bandwidth compared
to DRAM. Bandana uses two primary techniques to address this limitation: first,
it stores embedding vectors that are likely to be read together in the same
physical location, using hypergraph partitioning, and second, it decides the
number of embedding vectors to cache in DRAM by simulating dozens of small
caches. These techniques allow Bandana to increase the effective read bandwidth
of NVM by 2-3x and thereby significantly reduce the total cost of ownership.
