Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021
  Challenge: Report
  As the quality of mobile cameras starts to play a crucial role in modern
smartphones, more and more attention is now being paid to ISP algorithms used
to improve various perceptual aspects of mobile photos. In this Mobile AI
challenge, the target was to develop an end-to-end deep learning-based image
signal processing (ISP) pipeline that can replace classical hand-crafted ISPs
and achieve nearly real-time performance on smartphone NPUs. For this, the
participants were provided with a novel learned ISP dataset consisting of
RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and
a professional 102-megapixel medium format camera. The runtime of all models
was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI
processing unit capable of accelerating both floating-point and quantized
neural networks. The proposed solutions are fully compatible with the above NPU
and are capable of processing Full HD photos under 60-100 milliseconds while
achieving high fidelity results. A detailed description of all models developed
in this challenge is provided in this paper.
