Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D
  Shape Synthesis
  We introduce DMTet, a deep 3D conditional generative model that can
synthesize high-resolution 3D shapes using simple user guides such as coarse
voxels. It marries the merits of implicit and explicit 3D representations by
leveraging a novel hybrid 3D representation. Compared to the current implicit
approaches, which are trained to regress the signed distance values, DMTet
directly optimizes for the reconstructed surface, which enables us to
synthesize finer geometric details with fewer artifacts. Unlike deep 3D
generative models that directly generate explicit representations such as
meshes, our model can synthesize shapes with arbitrary topology. The core of
DMTet includes a deformable tetrahedral grid that encodes a discretized signed
distance function and a differentiable marching tetrahedra layer that converts
the implicit signed distance representation to the explicit surface mesh
representation. This combination allows joint optimization of the surface
geometry and topology as well as generation of the hierarchy of subdivisions
using reconstruction and adversarial losses defined explicitly on the surface
mesh. Our approach significantly outperforms existing work on conditional shape
synthesis from coarse voxel inputs, trained on a dataset of complex 3D animal
shapes. Project page: https://nv-tlabs.github.io/DMTet/.
