Flashlight: Enabling Innovation in Tools for Machine Learning
  As the computational requirements for machine learning systems and the size
and complexity of machine learning frameworks increases, essential framework
innovation has become challenging. While computational needs have driven recent
compiler, networking, and hardware advancements, utilization of those
advancements by machine learning tools is occurring at a slower pace. This is
in part due to the difficulties involved in prototyping new computational
paradigms with existing frameworks. Large frameworks prioritize machine
learning researchers and practitioners as end users and pay comparatively
little attention to systems researchers who can push frameworks forward -- we
argue that both are equally important stakeholders. We introduce Flashlight, an
open-source library built to spur innovation in machine learning tools and
systems by prioritizing open, modular, customizable internals and
state-of-the-art, research-ready models and training setups across a variety of
domains. Flashlight allows systems researchers to rapidly prototype and
experiment with novel ideas in machine learning computation and has low
overhead, competing with and often outperforming other popular machine learning
frameworks. We see Flashlight as a tool enabling research that can benefit
widely used libraries downstream and bring machine learning and systems
researchers closer together.
