Semantic Noise Modeling for Better Representation Learning
  Latent representation learned from multi-layered neural networks via
hierarchical feature abstraction enables recent success of deep learning. Under
the deep learning framework, generalization performance highly depends on the
learned latent representation which is obtained from an appropriate training
scenario with a task-specific objective on a designed network model. In this
work, we propose a novel latent space modeling method to learn better latent
representation. We designed a neural network model based on the assumption that
good base representation can be attained by maximizing the total correlation
between the input, latent, and output variables. From the base model, we
introduce a semantic noise modeling method which enables class-conditional
perturbation on latent space to enhance the representational power of learned
latent feature. During training, latent vector representation can be
stochastically perturbed by a modeled class-conditional additive noise while
maintaining its original semantic feature. It implicitly brings the effect of
semantic augmentation on the latent space. The proposed model can be easily
learned by back-propagation with common gradient-based optimization algorithms.
Experimental results show that the proposed method helps to achieve performance
benefits against various previous approaches. We also provide the empirical
analyses for the proposed class-conditional perturbation process including
t-SNE visualization.
