Deep 3D-to-2D Watermarking: Embedding Messages in 3D Meshes and
  Extracting Them from 2D Renderings
  Digital watermarking is widely used for copyright protection. Traditional 3D
watermarking approaches or commercial software are typically designed to embed
messages into 3D meshes, and later retrieve the messages directly from
distorted/undistorted watermarked 3D meshes. However, in many cases, users only
have access to rendered 2D images instead of 3D meshes. Unfortunately,
retrieving messages from 2D renderings of 3D meshes is still challenging and
underexplored. We introduce a novel end-to-end learning framework to solve this
problem through: 1) an encoder to covertly embed messages in both mesh geometry
and textures; 2) a differentiable renderer to render watermarked 3D objects
from different camera angles and under varied lighting conditions; 3) a decoder
to recover the messages from 2D rendered images. From our experiments, we show
that our model can learn to embed information visually imperceptible to humans,
and to retrieve the embedded information from 2D renderings that undergo 3D
distortions. In addition, we demonstrate that our method can also work with
other renderers, such as ray tracers and real-time renderers with and without
fine-tuning.
