{'fraction_non_alphanumeric': 0.023376623376623377, 'fraction_numerical': 0.0025974025974025974, 'mean_word_length': 5.4581005586592175, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
APInf: Adaptive Policy Inference Based on Hierarchical Framework

Deep reinforcement learning has made adequate progress in addressing complex visual tasks. However, policy inference in relatively large neural networks is costly. Inspired by the fact that some states in a specific task can easily make decisions, we propose an APInf framework that can achieve low policy inference costs with quality guarantees. Two-stage training is proposed: First, to accelerate inference at easy states, sub-policy networks ordered by increasing capacity are generated; Second, to keep the quality of policy, master-policy is designed to choose a suitable sub-policy network subjected to state difficulty dynamically. We also improve the efficiency of master-policy inference by sharing the convolution layer between sub-policy networks and master-policy and then train master-policy under an extended MDP. Extensive experiments conducted in gym show that the adaptive inference framework reduces the FLOPs by 41.2% while maintaining similar quality. Furthermore, APInf outperforms the existing dynamic network structure in terms of policy quality and inference cost.