{'fraction_non_alphanumeric': 0.015732846122614698, 'fraction_numerical': 0.005988631749898498, 'mean_word_length': 4.735157159487777, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Predicting emotion in music through audio pattern analysis

Music plays a big part in the development of humanity and has inspired so many people to do great things. Which is why a study was conducted focusing on music and its relation to human emotions. It tackles on how an emotional response is generated from an audio pattern through the use of a model constructed by the researchers. The work done is aimed to generate ratings of the emotion conveyed in a song represented as valence and arousal. Audio feature extraction and Artificial Neural Networks played big parts in the development of the research in creating the regression model and predicting the human emotion being emphasized in an audio pattern. The findings of the research will be used to give meaning to audio patterns and how their structure and composition leads to different emotional states of man.


Introduction
Researchers look for different kinds of correlations between music and the feelings people experience. Although ideas have been realized, there is still room to improve in implementations to reach the desired output: emotion in audio. The application of different combinations of machine learning algorithms with the features have yet to be tested and still remain uncertain on which is the best option. This research attempted to determine whether or not using a neural network with regression would yield a better result in determining valence and arousal values as compared to other methods. Valence is defined as the measure of how positive or negative an emotion is while arousal is how calmly or exciting the emotion is felt. In regression, output will be determined using R 2 score, which is a measure in statistics that deals with how close the data points are to the regression line. The research is aimed to construct a model that would predict valence and arousal R 2 scores given an audio sample. Manipulating audio patterns to derive human emotion has proven to be a difficult task. Some methods of classification have been considered to give emotions but only to a certain extent. The study is only limited to the use of the machine learning algorithm Artificial Neural Networks. Furthermore, the type of AI that has been developed in the research will be of a narrow kind for it only outputs mood ratings. The emotion regression was done using the Thayer's model of emotion and the annotations of each song from the dataset.

Review of Related Literature
In doing research, it is important to find references and studies about the topic being researched. Previous studies can provide valuable input for researchers especially in terms of methodology. Recently, Neural Networks have shown promising results in complex machine learning which is why the researchers

Feature Selection
The study [1] focused on identifying which set of features, when combined, yielded the best prediction results for valence and arousal. This study had a regression approach towards the problem. They used features extracted by Essentia and Marsyas which are both python libraries used for retrieving audio information such as Low-level, Rhythm and Tonal types. The study concluded that the use of several groups of audio features ended up giving better results compared to fewer groups.

SVM Approach
This study [2] focused more on the concept of Music Emotion Recognition (MER). They approached the problem using regression to predict the values arousal and valence scores directly. The study wished to evaluate the performance of a regression approach to predicting arousal and valence. They used an SVM as a regressor, which in turn yielded them an R 2 score of 58.3% for arousal and 28.1% for valence as their best result.

Fuzzy Approach
This study [3] focuses on the use of fuzzy logic for music emotion classification. Seeing the subjective nature of how humans perceive music to be, using fuzzy logic allows for a less binary approach to emotion analysis. Fuzzy classifiers allow for a value to be assigned based on how strong an emotion may be perceived. This approach manages to deal with the situation wherein people share different opinions regarding a certain song as well as being able to detect the variations of emotions found in said music.

Lyrical Analysis
The study [4] makes use of a bi-modal method of audio emotion classification as it deals with both audio and lyrical features, using these as basis for classifying audio to emotion. Selected machine learning algorithms are implemented in classifying to compare the each of their performances on the data.

Neural Network Architectures
The article [5] provides information on artificial neural networks. Neural networks have different kinds of applications and design architectures. The article mentioned proves to be insightful for researches involving neural networks giving an in depth understanding of the concept. In Thayer's model, there are four quadrants with different ranges of valence and arousal. Table 1 shows a sample of songs in the dataset and their corresponding placement in the plane.
The emotion ratings were scaled down to -0.2 to 0.2 to have a zero-centred axis and give lenience to error for the model in predicting values. This gives the neural network more room to train and react on the weight changes as the model is subjected to the data. In figure 1, it is clear that the data is condensed at the centre, in the positive and the negative axes. Basing on the Thayer's Model, songs that have mostly negative valence with positive arousal and songs that have positive valence with negative arousal are    : (1802, 193). The labels of each audio file are likewise stored in the same fashion with a shape: (1802,). These are the selected audio features extracted based on a study on classifying bird songs [6]. The features extracted are already proportioned by LibROSA so that the length of the audio would not affect the shape of the features obtained. Table 2 shows the features extracted with LibROSA. Another group of audio features was also taken into consideration from a certain research [2] that gathered information on which audio features contributed greatly to the prediction of valence and arousal. To get the features, the researchers used Essentia to extract the features in table 3 and also shaped them in the same fashion as the audio features extracted from LibROSA apart from the number of features to be inputted to the model

Neural Network Construction and Training
Two neural network configurations were created using the libraries: Tensorflow and Keras. One neural network having 1-1-1 NN layer layout (Input, Hidden, and Output) and the other having 1-6-1. The type of neural networks developed were feed-forward; one complex and the other simple. A feed-forward neural network is a kind of neural network where the nodes do not form a cycle and only move in one direction, which is forward and through the nodes. An example of a feed forward neural network is shown in figure 2. The 1-1-1 architecture was to see if a simple NN architecture would be sufficient enough to predict valence and arousal through the use of the different audio features obtained. Otherwise, the researchers would turn to the use of a more complex feed-forward structure.

Evaluating Models
The models were subjected to a 10-fold cross validation where the dataset was shuffled and per iteration was subjected to training and prediction. Each iteration reported on different regression metrics available from Sci-Kit Learn. This researched used explained variance score, mean absolute error, mean squared error, median absolute error and r 2 score for the verification

Analysis of Results
Using the 38 features from Essentia library yielded dissatisfying results when paired with the labels during the train and test process. The 193 features from LibROSA had better results compared to the ones from Essentia but were also ineffective in prediction. So the researchers incrementally added data rows for the train and test process to see if the model would produce better results as the number of data rows increased or decreased. The scaling of the values did not greatly affect the r-squared scores but lessened the mean absolute error. Table 4 displays some of the expected emotion ratings and the predictions made by the model. Several number of nodes were tested alongside it but the result from the simple feed forward network that had one hidden layer was not successful compared to the complex network. The best result obtained was from the neural network configuration that had 6 hidden layers each having 500 nodes with rectified linear unit activators and an output node with a linear activation, so the researchers used these values for further testing. Table 5 indicates the best results attained from the model.

Conclusion
The neural network architecture developed is experimental and needs refinement. A different type of neural network may prove to be more efficient than a feed-forward network for this kind of problem. A recurrent neural network works well with speech recognition and would be a better option in proceeding with emotion prediction of audio. The improvement of data would be greatly welcomed as time progresses music continues to evolve. More data rows and different kinds of datasets would better develop the field and lead to better and more accurate models. As of now, the data related to audio sentiment is limited and researchers struggle to get accurate data that would be viable for usage.
It is valid to predict emotion in music because music affects humanity and it is beneficial to know which types of music negatively or positively affects a person. Creating a model that can deduce an audio pattern's emotional rating provides a buffer for people to decide whether or not to listen or create music based on the rating.