{'fraction_non_alphanumeric': 0.012903225806451613, 'fraction_numerical': 0.0, 'mean_word_length': 5.313953488372093, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Document Modeling with Implicit Approximate Posterior Distributions

This paper proposes a Bayesian probabilistic document model whose variational inference is achieved by using an implicit approximate posterior distribution. The proposed model generates a set of documents as follows. First, we draw a noise vector from the multivariate standard normal distribution. Second, we use the noise vector as an input to a neural network to obtain a parameter vector of multinomial distribution. Finally, we draw word counts from the multinomial distribution to generate a document. This generative story is similar to that of NVDM. However, we use an implicit approximate posterior distribution in the variational Bayesian inference for our model. The inference for NVDM is achieved with VAE, which does not use implicit distribution for approximating the posterior. Our main contributions are to provide an example of Bayesian document model whose posterior can be effectively approximated by an implicit distribution and to show that our model is comparable to LDA in terms of perplexity.