{'fraction_non_alphanumeric': 0.0183366077275704, 'fraction_numerical': 0.0, 'mean_word_length': 5.821428571428571, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Communication-Efficient Cooperative Contextual Bandit and Its Application to Wi-Fi BSS Selection

In this study, we extended a contextual bandit algorithm, LinUCB, to facilitate cooperative learning of an optimal strategy with intermittent information sharing. We then applied the algorithm to a Wi-Fi basic service set (BSS) selection problem. The BSS selection problem, in which a mobile user selects a BSS that provides maximal throughput based on information observed, still remains a topic of debate. Reinforcement learning, specifically the multi-armed bandit algorithm, enables mobile users to learn an optimal strategy for selecting a good BSS in their environments. This paper proposes a cooperative contextual bandit algorithm, called Cooperative LinUCB (CoopLinUCB), to address the BSS selection problem. Conventional cooperative bandit algorithms require to share experiences such as context and payoffs for every action, and the information sharing increases communication costs. The proposed algorithm enables mobile users to learn strategies using a limited amount of information that is shared intermittently. The learned strategies are then guaranteed to be equivalent to the strategies that are updated using user experience information. Simulation evaluation based on measured throughput and received signal strength indication fingerprint demonstrates that CoopLinUCB-based BSS selection learns a BSS selection strategy faster and reduces the cumulative regret compared to BSS selection without cooperation.