Towards an Intelligent Microscope: adaptively learned illumination for
  optimal sample classification
  Recent machine learning techniques have dramatically changed how we process
digital images. However, the way in which we capture images is still largely
driven by human intuition and experience. This restriction is in part due to
the many available degrees of freedom that alter the image acquisition process
(lens focus, exposure, filtering, etc). Here we focus on one such degree of
freedom - illumination within a microscope - which can drastically alter
information captured by the image sensor. We present a reinforcement learning
system that adaptively explores optimal patterns to illuminate specimens for
immediate classification. The agent uses a recurrent latent space to encode a
large set of variably-illuminated samples and illumination patterns. We train
our agent using a reward that balances classification confidence with image
acquisition cost. By synthesizing knowledge over multiple snapshots, the agent
can classify on the basis of all previous images with higher accuracy than from
naively illuminated images, thus demonstrating a smarter way to physically
capture task-specific information.
