Timbre Transfer with Variational Auto Encoding and Cycle-Consistent
  Adversarial Networks
  This research project investigates the application of deep learning to timbre
transfer, where the timbre of a source audio can be converted to the timbre of
a target audio with minimal loss in quality. The adopted approach combines
Variational Autoencoders with Generative Adversarial Networks to construct
meaningful representations of the source audio and produce realistic
generations of the target audio and is applied to the Flickr 8k Audio dataset
for transferring the vocal timbre between speakers and the URMP dataset for
transferring the musical timbre between instruments. Furthermore, variations of
the adopted approach are trained, and generalised performance is compared using
the metrics SSIM (Structural Similarity Index) and FAD (Frech\'et Audio
Distance). It was found that a many-to-many approach supersedes a one-to-one
approach in terms of reconstructive capabilities, and that the adoption of a
basic over a bottleneck residual block design is more suitable for enriching
content information about a latent space. It was also found that the decision
on whether cyclic loss takes on a variational autoencoder or vanilla
autoencoder approach does not have a significant impact on reconstructive and
adversarial translation aspects of the model.
