{'fraction_non_alphanumeric': 0.007658643326039387, 'fraction_numerical': 0.0, 'mean_word_length': 5.310344827586207, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Nonlinear credit assignment for musical sequences

Reinforcement learning is an exciting possibility for generating music because of its ability to learn without explicit examples and to produce more than one response in a given state. We use reinforcement learning in the second phase of a jazz improvisor that learns to interactively play jazz with a human. The reinforcement signal is based on rules for improvisation. Because of time delays between note played and subsequent reinforcement, a critic adjusts the reinforcement signal. We describe this system and then examine the ability of a temporal difference critic to predict reinforcement for three different sequential musical phenomena. A nonlinear network with a linear TD output unit and context traces on input is able to successfully predict reinforcement values for these sequences and shows promise for use in musical reinforcement learning tasks.