Procedural 3D Terrain Generation using Generative Adversarial Networks
  Procedural 3D Terrain generation has become a necessity in open world games,
as it can provide unlimited content, through a functionally infinite number of
different areas, for players to explore. In our approach, we use Generative
Adversarial Networks (GAN) to yield realistic 3D environments based on the
distribution of remotely sensed images of landscapes, captured by satellites or
drones. Our task consists of synthesizing a random but plausible RGB satellite
image and generating a corresponding Height Map in the form of a 3D point cloud
that will serve as an appropriate mesh of the landscape. For the first step, we
utilize a GAN trained with satellite images that manages to learn the
distribution of the dataset, creating novel satellite images. For the second
part, we need a one-to-one mapping from RGB images to Digital Elevation Models
(DEM). We deploy a Conditional Generative Adversarial network (CGAN), which is
the state-of-the-art approach to image-to-image translation, to generate a
plausible height map for every randomly generated image of the first model.
Combining the generated DEM and RGB image, we are able to construct 3D scenery
consisting of a plausible height distribution and colorization, in relation to
the remotely sensed landscapes provided during training.
