{'fraction_non_alphanumeric': 0.02648171500630517, 'fraction_numerical': 0.01773329129886507, 'mean_word_length': 4.363060016906171, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Top Score in Axelrod Tournament

The focus of the project will be an examination of obtaining the highest score in the Axelrod Tournament. The initial design of the highest score in the Axelrod Tournament consisted of looking at the Cooperation rates of the top strategies currently in the Axelrod Library. After creating an initial Finite State Machine strategy that utilized the Cooperation Rates of the top players; our ten-state FSM finished within the top 35 players out of 220 in the short run time strategies in the Axelrod Library. After a quick evolutionary algorithm, 50 generations, our original ten-state FSM was changed into an eight-state FSM, which finished within the top 5 of the short run time strategies in the Axelrod Library. This eight-state FSM was then evolved again using a full evolutionary algortihm process, which lasted 500 generations, where the eight-state FSM evolved into a another eight-state FSM which finishes first in the Axelrod Library among the short run time strategies and the full Axelrod Tournament. From that final FSM, two of the eight states are inaccessible, so the final FSM strategy is a six-state FSM that finishes first in the Full Axelrod Tournament, against the short run time strategies as well as the long run time strategies.


Introduction
The objective of this project was to obtain the highest score in the Axelrod Tournament. There have been a number of strategies created from various disciplines to compete to obtain the highest score. Throughout the last several decades, there have been numerous strategies developed in order to compete in the tournament, as well as different strategy types [1]. This paper will demonstrate how our team was able to achieve the highest score in the Axelrod Tournament.
We have developed a Finite State Machine strategy that has six states that has the highest score in the Axelrod Tournament. Our six-state strategy was developed from a ten-state strategy that was written out by hand, SecondPrac, that ten-state strategy was then evolved using the evolutionary algorithm into an eight-state Finite State Machine, Fourthprac. This eight-state FSM finished in the top 5 of the short run time strategies before another evolutionary process that led it to the top score. The final eight-state Finite State Machine that was produced had two inaccessible states, so the final strategy was a six-state FSM called EvolvedFSM6.

Axelrod Tournament
The Axelrod Tournament has over 220 strategies ranging from Finite State Machines, to Artifical Neural Networks, to LookerUp Tables, as well as other strategies designed to play the Prisoner's Dilemma [1]. The top scores in the Axelrod Tournament are evolved versions of the LookerUp Table, the Evolved Neural Network, as well as an Evolved 16-state Finite State Machine. Each one of those strategies has been evolved, or trained to be able to maximize its score. There are other strategies that make up the rest of the field, to be able to compare how different strategies fair against one another. Many of the strategies that are in the library are deterministic, they follow a set of pre-determined guidelines to determine which moves will be played in a given situation. All of the strategies that are currently in the Axelrod Library can be found at the Axelrod Library's Github page [2].
When initially running the Axelrod Tournament with all of the short run time strategies in the Library, EvolvedLookerUp2 2 2 finishes first, followed by Evolved FSM 16, and Evolved HMM 5. Each of these strategies have been evolved and their scores can be seen in Table 1. The top strategies are all different forms of evolved strategies in the Axelrod Tournament. After settling on the Four Memory One moves, the next important step was to look at Cooperation Rates from those moves. The Cooperation Rates for top players in the Tournament were similar to each other, so we simply decided to use that. According to [3], the top 80 players will always Cooperate with each other. There needed to be a way to ensure that our strategy would Always Cooperate with the other top strategies so we looked at the Cooperation Rates of those top 80 players that are currently in the Axelrod Library. This led to our initial design and subsequent strategies.

First Attempt
The first attempt at making a Finite State Machine was to make one with a 100% Cooperation Rate after a [C, C] move and a 50% Cooperation Rate for the other three moves, [C, D], [D, C], and [D, D]. The initial thought process for this strategy was to always Cooperate after a [C, C] move, which led to a type of loop from State 1 to State 2 back to State 1. Then from the other states, there was always an attempt to alternate between Cooperating and Defecting after [C, D], [D, C], and [D, D] moves. The reason to create an FSM with these specified Cooperation Rates was to get a sense of how the Cooperation Rates based on the Memory One Moves can affect a strategy's score in the Axelrod Tournament.
This led to an eight-state FSM called FirstPrac. Figure 1 shows a modified Finite State Diagram for the first attempt in the Axelrod Tournament. The eight states were created in order to try to maintain 50% Cooperation rates after the other Memory One Moves. If there was a Cooperation after [C, D], the next time there was a [C, D], there would be a Defect move played. This led to the eight state FSM. state and move were connected to one another. The first move was to initially Cooperate and move to State 1. Then from State 1, if the opponent Cooperated, the player would Cooperate and move to State 2. From State 1, if the opponent Defected, the player would Cooperate and move to State 3. The rest of the diagram follows this system.
When running the FirstPrac strategy against the short run time strategies, it finished in 107 th place and had Cooperation Rates of 100%, 73.6%, 100%, and 0%. This looks like that the strategy stayed in the same few states and was never in a scenario where it would Cooperate after a [D, D] move. Figure 2 is another FSM diagram for FirstPrac, this diagram is a more common diagram. It is a diagram that is used to describe the Finite State Machines in [3].

Second Attempt
The second attempt to create an FSM strategy used the Cooperation Rates from the top 80 players. According to [3], the top players always Cooperate with each other, so this new strategy had to be able to Cooperate with the other top players. The way to make sure our strategy always Cooperated with the top players was to take the average Cooperation Rates from the top 80 players. After running an Axelrod Tournament, a csv file with the Cooperation Rates after With this information, we rounded those percentages so we looked to create an FSM, with Cooperation Rates of 100%, 40%, 50%, and 20%. This led to the creation of the second strategy, a ten-state FSM, called SecondPrac. Since there were more complex percentages in SecondPrac than there was in FirstPrac, there needed to be more states in order to try to maintain the Cooperation Rates from the top 80 players.    Figure 4 shows the Finite State Machine Diagram that corresponded to the SecondPrac strategy.

ThirdPrac
After failing to modify the SecondPrac strategy that finished 34 th successfully, a new strategy, ThirdPrac was created. ThirdPrac was created to try to maintain the same percentages from the top 80 strategies, but the states were completly altered. This resulted in an 8-state FSM. This FSM finished 58th and had Cooperation Rates of 100% after a [C, C] move, 44.1% after a [C, D] move, 91.9% after a [D, C] move, and 0% after a [D, D] move. It was at this point, to go back to SecondPrac and evolve it.

FourthPrac
Initially when trying to evolve SecondPrac using 500 generations, which is the default setting for the Axelrod Tournament, there would be an error that would occur around 40-60 generations on my computer. So I decided to evolve it using 50 generations, with a population size of 40, .1 mutation rate, a bottleneck of 10, 1 process, 10 repetitions, 20 turns, 0 noise, 4 Moran processes, and 10 states. This evolution worked and the FourthPrac strategy was created. This was a 10-ten state FSM that finished in the top 5 of the Axelrod Library in the Short Run Time tournament. It had the following Cooperation Rates, 92.  Over the course of the evolution from SecondPrac to FourthPrac, there were a number of states as well as moves that were changed. Out of the 20 possible moves, 12 of them stayed 1 C the same, for example, the first move [1, C, 2, C] were the same for both strategies. The first six moves stay the exact same, as the first change in strategy comes after the Fourth state. In SecondPrac, if the opponent Cooperates while the player is in State 4, the strategy will go to State 5 and Cooperate. In FourthPrac, under those same conditions, the strategy will go to State 5 and Defect. In total, there were three instances, where the strategy only changed a move, from Cooperate to Defect or Defect to Cooperate. There were two instances where the state only changed. And there were three instances where both state and move were changed.

Final Strategy
From the FourthPrac strategy, Dr. Fryer evolved it using 500 generations and a 8-state FSM was created Evolved FSM 8. This resulted in the top score of the Tournament.   Since EvolvedFSM8 starts at State 5, we will look at the possible states to get to. State 5 has two possibilities, stay in State 5 or go to State 7. There are now possible two states in the FSM. From State 7, there are two possibilities, go to State 4 or go to State 6, which leaves us with four states. The two possibilities from State 4 are to go to State 5 or go back to State 7. These are two previous states that have been identified. There are still four states. From State 6, there are two possibilities, go to State 3 or to State 8. Now there are six possible states. The possibilities for State 3 are to stay in State 3 or go to State 8. Those two possibilities are already in the FSM, so there are no new states being created. State 8 will either go to State 3 or will go back to State 4, and both of these states have been identified. This shows that there is no need to go be in State 1 or State 2 because that will never occur in this FSM since our initial state is State 5. So the 8-state FSM can be rewritten as a 6-state FSM.
We have done an initial dive into looking at when our evolved strategy became the top strategy in the Axelrod Tournament. It seemed to occur after the 351 st evolutionary generation. The 351 st evolutionary generation is also important because it is when the strategy went from a 7-state FSM to a 6-state FSM. On the parameter file, where each evolution generation is stored, there was a significant jump in the 351 st generation. In Generation 350, the average score was 2.871814908, and on Generation 351, the average score was 2.885947477. This was a difference in 0.014132569 points, which is significant. the Axelrod Tournament, they both place high. Generation 350 places 3 rd , while Generation 351 takes the top score. It must be at this specific iteration where the score becomes the highest.

Long Run Strategies
From all the previous tournaments we have run using the Axelrod Library, we mainly focused on the Short Run Time strategies. One of the main reasons for doing that was my computer was not capable of running a tournament that included the Long Run Time strategies. We assumed that our strategy had the highest score, but we could not know for sure until it ran against the Full Tournament library, which included the Long Run Time Strategies. Table 8 shows the Full Tournament which includes the Long Run Time Strategies before adding EvolvedFSM6.
Dr. Fryer ran the EvolvedFSM6 on his computer to generate these results for our strategy in the Axelrod Tournament. Table 9 refers to the Tournament with all the strategies, including the Long Run Time Strategies in the Axelrod Library. EvolvedFSM6 will be the top score against all the different strategies in the Axelrod Tournament.

Conclusion
We have shown that a six state Finite State Machine can be the top score in the Axelrod Tournament. By initially having a strategy that tries to follow the Cooperation Rates of the Top Strategies, a strategy can place fairly high in the tournament. The strategy can then be evolved in order to maximize a top score.
With further research and time, we want to be able to increase our current score to see what is the highest score possible in the Axelrod Tournament can be. Starting off with a high score prior to evolving the score played a major role in advancing our strategy.