CodeReef: an open platform for portable MLOps, reusable automation
  actions and reproducible benchmarking
  We present CodeReef - an open platform to share all the components necessary
to enable cross-platform MLOps (MLSysOps), i.e. automating the deployment of ML
models across diverse systems in the most efficient way. We also introduce the
CodeReef solution - a way to package and share models as non-virtualized,
portable, customizable and reproducible archive files. Such ML packages include
JSON meta description of models with all dependencies, Python APIs, CLI actions
and portable workflows necessary to automatically build, benchmark, test and
customize models across diverse platforms, AI frameworks, libraries, compilers
and datasets. We demonstrate several CodeReef solutions to automatically build,
run and measure object detection based on SSD-Mobilenets, TensorFlow and COCO
dataset from the latest MLPerf inference benchmark across a wide range of
platforms from Raspberry Pi, Android phones and IoT devices to data centers.
Our long-term goal is to help researchers share their new techniques as
production-ready packages along with research papers to participate in
collaborative and reproducible benchmarking, compare the different
ML/software/hardware stacks and select the most efficient ones on a Pareto
frontier using online CodeReef dashboards.
