Local and non-local dependency learning and emergence of rule-like
  representations in speech data by Deep Convolutional Generative Adversarial
  Networks
  This paper argues that training GANs on local and non-local dependencies in
speech data offers insights into how deep neural networks discretize continuous
data and how symbolic-like rule-based morphophonological processes emerge in a
deep convolutional architecture. Acquisition of speech has recently been
modeled as a dependency between latent space and data generated by GANs in
Begu\v{s} (2020b; arXiv:2006.03965), who models learning of a simple local
allophonic distribution. We extend this approach to test learning of local and
non-local phonological processes that include approximations of morphological
processes. We further parallel outputs of the model to results of a behavioral
experiment where human subjects are trained on the data used for training the
GAN network. Four main conclusions emerge: (i) the networks provide useful
information for computational models of speech acquisition even if trained on a
comparatively small dataset of an artificial grammar learning experiment; (ii)
local processes are easier to learn than non-local processes, which matches
both behavioral data in human subjects and typology in the world's languages.
This paper also proposes (iii) how we can actively observe the network's
progress in learning and explore the effect of training steps on learning
representations by keeping latent space constant across different training
steps. Finally, this paper shows that (iv) the network learns to encode the
presence of a prefix with a single latent variable; by interpolating this
variable, we can actively observe the operation of a non-local phonological
process. The proposed technique for retrieving learning representations has
general implications for our understanding of how GANs discretize continuous
speech data and suggests that rule-like generalizations in the training data
are represented as an interaction between variables in the network's latent
space.
