{'fraction_non_alphanumeric': 0.020202020202020204, 'fraction_numerical': 0.0, 'mean_word_length': 5.871287128712871, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reinforcement learning, particle filters and the EM algorithm

We consider a parameter estimation problem for a Hidden Markov Model in the framework of particle filters. Using constructs from reinforcement learning for variance reduction in particle filters, a simulation based scheme is developed for estimating the partially observed log-likelihood function. A Kiefer-Wolfowitz like stochastic approximation scheme maximizes this function over the unknown parameter. The two procedures are performed on two different time scales, emulating the alternating `expectation' and `maximization' operations of the EM algorithm. Numerical experiments are presented in support of the proposed scheme.