Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement
  Learning Painting Agent
  The objective of most Reinforcement Learning painting agents is to minimize
the loss between a target image and the paint canvas. Human painter artistry
emphasizes important features of the target image rather than simply
reproducing it (DiPaola 2007). Using adversarial or L2 losses in the RL
painting models, although its final output is generally a work of finesse,
produces a stroke sequence that is vastly different from that which a human
would produce since the model does not have knowledge about the abstract
features in the target image. In order to increase the human-like planning of
the model without the use of expensive human data, we introduce a new loss
function for use with the model's reward function: Content Masked Loss. In the
context of robot painting, Content Masked Loss employs an object detection
model to extract features which are used to assign higher weight to regions of
the canvas that a human would find important for recognizing content. The
results, based on 332 human evaluators, show that the digital paintings
produced by our Content Masked model show detectable subject matter earlier in
the stroke sequence than existing methods without compromising on the quality
of the final painting. Our code is available at
https://github.com/pschaldenbrand/ContentMaskedLoss.
