{'fraction_non_alphanumeric': 0.031121946159033145, 'fraction_numerical': 0.011255770527516987, 'mean_word_length': 4.976441413515189, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multiscale Fractal Analysis on EEG Signals for Music-Induced Emotion Recognition

Emotion Recognition from EEG signals has long been researched as it can assist numerous medical and rehabilitative applications. However, their complex and noisy structure has proven to be a serious barrier for traditional modeling methods. In this paper, we employ multifractal analysis to examine the behavior of EEG signals in terms of presence of fluctuations and the degree of fragmentation along their major frequency bands, for the task of emotion recognition. In order to extract emotion-related features we utilize two novel algorithms for EEG analysis, based on Multiscale Fractal Dimension and Multifractal Detrended Fluctuation Analysis. The proposed feature extraction methods perform efficiently, surpassing some widely used baseline features on the competitive DEAP dataset, indicating that multifractal analysis could serve as basis for the development of robust models for affective state recognition.


I. INTRODUCTION
Machine Learning has made overwhelming progress in modeling rational intelligence and the way humans perceive and act upon their environment. In tasks such as Object Recognition [1] and Sequence Prediction [2], supervised machine learning systems have accomplished to surpass, in many cases, the human brain capabilities. However, there are still many challenges in approaching emotion-driven intelligence, although it constitutes a fundamental aspect of human's perception and decision-making processes. The reason for this is that emotions are highly subjective, and thus really difficult to be labeled when expressed. A large number of studies attempt to include speech [3], [4], text [5], as well as facial expressions [6] in building emotion recognition systems. Nevertheless, there is a growing interest in emotion tagging through physiological signals [7], since those are induced without our active interference and thus depict more clearly the actual affective state. Such methods have been popular in designing Human-Computer or Brain-Computer Interfaces (BCI) that aid humans and adapt to personalized preferences. Further, physiological signals have been used for medical purposes, among others for the detection of epilepsy [8] and depression [9].
Among a variety of physiological signals, special consideration is given to brain data. The electroencephalogram (EEG) is the most widely researched signal of its kind and has been highly effective in detecting affective states. A variety of time [10], frequency [11] and joint [12] domain features have been extracted from EEG. Particular attention has been given to channel connectivity features, such as mutual information [13] and differential asymmetry [14], reporting the highest scores  [15], [16].
However, processing EEG signals and extracting useful features remain core challenges, since EEG, like most biological signals, is chaotic, nonlinear and incorporates a large amount of noise, both from the recording equipment and interfering physiological processes [17]. Because of the nature of such signals [18] several nonlinear fractal methods have been proposed, one of them being the Higuchi Fractal Dimension [19], which has been used extensively in emotion recognition as an analysis tool [20], [21]. Due to their complexity though, such signals do not always share the same structure over every time scale, hence the fractal characteristics may vary and change dynamically or accordingly to the examined scale. For this reason, in this paper we propose the Multiscale Fractal Dimension [22] and Multifractal Detrended Fluctuation Analysis [23] to examine the EEG signals and determine emotional information buried in their fragmented structure. In order to demonstrate the efficiency of the proposed multifractal features, we modify the BCI workflow [24] to include our Feature Engineering algorithms (Fig. 1), obtaining competitive results that surpass some widely used baseline methods.
The rest of the paper is organized as follows: Sec. 2 provides a detailed description of the multifractal algorithms used in this study. In Sec. 3, we analyze the structure of EEG signals in terms of stationarity and fragmentation, which are important factors in fractal algorithms. Section 4 describes the feature extraction methodology, whereas in Sec. 5 we describe the the context of our experiments and discuss their outcomes. Finally, in Sec. 6 we conclude our analysis and propose further directions for future work on the field.

II. MULTIFRACTAL METHODS
In signal analysis and geometry, the Fractal Dimension D is an index of complexity and fragmentation, comparing how the details of a signal's pattern change when measured at different scales. The fractal dimension of 1D signals may vary between 1 and 2, and the larger the D is, the larger the amount of fragmentation of the signal. Alternatively, one could consider the Hurst Exponent H of a signal to analyse the global properties of its fluctuations. Given a self-similar signal, e.g. fractional brownian motion, the fractal dimension is derived from the Hurst exponent through H = 2 − D.

A. Multiscale Fractal Dimension
Maragos [22] developed an efficient algorithm to determine the Multiscale Fractal Dimension (MFD) of a signal by measuring the multiscale length of a curve with disks of varying radius via morphological coverings. The cover is created using 2D morphological set dilations of the signal graph F by multiscale versions sB = {sb : b ∈ B} of a unit-scale convex symmetric set B, s ≥ 0 the scale: Then, the cover area A B (s) = area(F ⊕ sB) is computed and the fractal dimension D is yielded by: It has been shown [22] that the above limit will not change if we approximate A B (s) with 1D nonlinear convolutions instead of 2D set operations, which enables its efficient calculation. In practice, D can be estimated by a least-squares line to find the slope of log[A B (s)] versus log(s) since assuming the power law A B (s) ≈ s 2−D as s → 0. We therefore compute the slope of the data over a small scale window of w scales that move along the scale axis s {s, s+1, ..., s+w}, creating a profile of local MFDs D(s, t) at each time t (fractogram). The local slope is now an estimate of 2 − D and from this the fractal dimension D can be easily derived.

B. Detrended Fluctuation Analysis
Detrended Fluctuation Analysis (DFA) [26] estimates the Hurst exponent H in time series data x[n] of length N by utilizing its cumulative sum y This profile is divided into N s non-overlapping windows y[k, n], k = 1, ..., N s of length s and for every window the local trend r[k, n] is obtained through linear regression. We denote y d [k, n] = y[k, n]−r[k, n] the detrended version of the k-th profile segment. Then, the RMS value of each detrended segment is computed and averaged across the segments: The result of the above operations is a vector of s values, one for each chosen scale. The relationship between F (s) and s is described by the power law F (s) ∝ s H , which determines H. Multifractal DFA (MFDFA) [23] is essentially a generalization of DFA, where the computation of F (s) includes q moments: As a result, a separate line is computed for every value of the factor q, with q = 2 being the reduction to classical DFA. MFDFA could prove especially useful in cases where the scaling exponents and complexities are dependent on the scale, or change dynamically, in the context of time series.

A. Dataset
We work on the competitive, but widely used DEAP Dataset [25], including data from 32 subjects in their already preprocessed form. Each subject is exposed to forty 60-seconds long music videos as stimuli, while having their EEG recorded, along with other physiological signals. After watching each video, the subject was instructed to rate their induced emotion in 5 dimensions: valence (pleasantness), arousal (excitation), dominance, liking and familiarity to the stimulus. We solely experiment with valence and arousal, as they form a complete emotion space [25]. The rating ranges from 1 (weakest) to 9 (strongest). The EEG signals were recorded at a sampling rate of 512 Hz and downsampled to 128 Hz. The 10-20 placement system was followed, using 32 electrodes.

B. Stationarity & Hurst Exponent Estimation
Physiological signals like the EEG are widely researched as noisy and non-stationary signals and commonly demand heavy pre-processing. The observed structure is partly due to external stimuli or other physiological operations and mainly indicates the complexity and the states of neural assemblies during brain functioning [17]. In our experiments, it is crucial to determine the stationarity of the signals in order to correctly interpret their fractal properties. We apply the Augmented Dickey-Fuller (ADF) Test to a randomly sampled set of EEG signals and, to our surprise, we derive evidence of strict stationarity. Specifically, the examined signals appeared non-stationary only at very low scales, up to windows of 100 samples or 0.8 seconds. The same holds when we test the signal profiles, i.e. the cumulative sums. However, a few signals exhibit nonstationarities at their major frequency bands. In order to find the source of this stationarity, we reproduced the preprocessing applied in [25] to a sample raw waveform. This included downsampling to 128 Hz, eye-artefact removal, filtering at 4-45Hz and averaging to the common reference channel. After this procedure, it was found that the cause of stationarity was the performed bandpass filtering.
Although fractal methods are used for analyzing time series that appear to have long-memory correlations and nonstationary dynamics, they are not restricted to such. Scale-free stationary processes can be viewed as fractional Gaussian noise (fGn), while their increments typically construct nonstationary processes in the form of fractional Brownian motion (fBm), of the same Hurst exponent. Thus, the exponent estimation is crucial in characterizing EEG signals for multifractal analysis [27] and can be determined by monofractal DFA. If the estimated exponent is less than H = 1, then it characterizes a stationary process, which can be modeled as fGn with that exponent. Otherwise, it is assumed to be produced by a nonstationary fBm process with an exponent of H − 1.
The EEG signals of the DEAP dataset provide a very low Hurst Exponent value that approaches 0, while their profiles and separate bands provide an increased DFA-estimated exponent, still though below 0.2 at most cases. The results however alter when we examine the profiles of the filtered bands, particularly alpha and beta, in which the exponent estimation shows a steady increase. These values confirm the evidence from the conducted ADF Test that EEG signals are negatively correlated and their fluctuations are smaller in larger time windows, which is the typical behavior of fGn processes having Hurst exponents below 0.5.

A. Baseline Features
A set of widely used baseline features is extracted for comparative reasons and to assess the combined efficiency of the proposed feature set. These features are the Power Spectral Density (PSD) and the Higuchi Fractal Dimension (HFD) [19]. PSD is computed across the entire signals through the Welch method, resulting in 64 features per signal. The Higuchi Fractal Dimension is extracted using PyEEG [29] and produces a scalar feature. To derive a feature vector here, we first split each signal into windows of 15 seconds (1920 samples) with 50% overlap and then the HFD is determined for each one of the 7 windows, resulting in a 7D vector.

B. MFD Features
Since MFD is mainly used for short-time analysis [30], we again split each signal into 15 sec. windows with 50% overlap. The proposed feature set includes 30 linearly sampled features extracted out of each window's MFD. The respective features of each window are then summarized using 3 statistical metrics: mean, median and standard deviation. In this way, we get a final 90D feature vector incorporating the signal's temporal variance. The signals are analyzed at discrete scales of s = 1, ..., 274 samples, thus the maximum scale is at s = 1/7 of the signals' length. The fractogram of a sample signal along with the variance of its 7 windows are shown in Fig. 2. The EEG fractograms reveal a highly fragmented structure and a high fractal dimension D > 1.5. This finding is consistent with the low Hurst Exponent we got from monofractal DFA.

C. MFDFA Features
We additionally acquire 30 features from processing the last half of each EEG waveform through the computationally expensive MFDFA. We select 10 scales ranging from 30 to 500 samples and 16 q-moment values ranging from −5 to 5. The resulting representation is a set of 16 linear-like graphs of 10 values, as shown in Fig 3. 16 Hurst Exponent values are determined through linear regression, one for each moment. The mass exponent t is then derived through t(q) = qH(q)−1. A monofractal signal with constant H would produce a linear graph, the EEG instead produces a curve that we utilize to produce the signal's multifractal spectrum D: where n = 1, ..., 15, q excludes the largest moment value, and h(q) is the singularity exponent. The resulting curve, determined by 15 h(q) and 15 D(q) values, represents the MFDFA feature set.
V. EXPERIMENTAL EVALUATION Experimental Protocol: We evaluate the features extracted from the multifractal analysis on the emotion recognition task.  The experimental protocol can be divided into two categories: Subject Dependent, in which a classifier is trained and tested on trials of a single participant, with the final score being the average per-subject score, and Subject Independent, where a classifier is trained on several participants and tested against unseen trials. We shall mention that lower scores are typically reported [24] for the latter, since EEG is highly subjectsensitive, thus we anticipate such behavior also in our models.
In this work, we make use of a single classifier unifying features from all available EEG channels. The model consists of a Standard Scaler, that standardizes training features by removing their mean and scaling them to unit variance, and a Support Vector Machine (SVM) with an RBF kernel. Experiments consider single labels, i.e., valence or arousal, in binary format by setting the threshold for binarization in the median score 5. We perform 5-fold cross validation on stratified splits of the available data: approximately 56.5% of all samples are of high valence and 59% of high arousal annotations.
Comparison to Baselines: The classification results for all features at the 2 distinct settings are summarized in Table 1 and 2. We notice the accuracy difference between subject dependent and independent tasks, supporting the claim that brain responses inherit mainly subjective characteristics. The EEG PSD is shown to be efficient in the subject-dependent setting, where the raw signal modality achieves 64.2% in valence and 65.2% in arousal. Interestingly, these scores significantly drop in the subject-independent setting, where the PSD emerges as the least efficient feature set, achieving only chance-level scores in arousal, 6% below the top recorded accuracy of MFD. We can therefore assume that the withinsubject variability is concentrated more on separate spectral characteristics of each participant and therefore, fractal analysis is more robust across subjects. both experiments, surpassing chance levels and the baseline features in most cases. In contrast to spectral features that are sensitive to valence, these features prove efficient mainly in recognizing the arousal state, in which they achieve around 5% higher scores. At the subject-dependent experiment particularly, MFD of the alpha band and MFDFA at the raw signal yield 66.9% and 66.2% respectively, whereas their highest subject-independent accuracy hits 63%. Our results are in accordance with those reported in [31] for PSD and HFD, while the top scores obtained by MFD and MFDFA surpass most of the ones reported there. On the other hand, at subjectindependent classification, multifractal features perform comparably to those discussed at [32] for DEAP, although we recognize the additional difficulty of eliminating all of the trials of a tested participant from training. Aggregated features: Multifractal features, and MFD in particular, clearly outperform the Higuchi baseline in arousal and perform comparably in valence, indicating that the multiscale variability of the EEG can capture latent emotional information. Furthermore, the two kinds of fractal dimensions provide even better scores in arousal when combined. As shown in Table 3, in both subject dependent and independent settings we record higher accuracy than the one we obtained from the individual features in the ablation study, mainly when testing raw signals or the alpha band. The differences are significant in the subject independent setting, whereas even the top scores obtained previously are improved. The model can now predict arousal at 67% and 64% at subject dependent and independent experiments respectively.

Multifractal methods show indeed strong performance in
Other than that, it seems that the selection of a single feature type could be adequate for affective state recognition, since we do not observe substantial improvement for the "Combined" features, in which we measure the aggregated performance of the three bands and the raw signal. We then compare the two asymmetrical sets of channels, where the left hemisphere is more efficient in terms of multifractal analysis, both in subject dependent and independent setting, performing 2% better on average. In order to assess the asymmetrical performance, we also experimented on aggregated trials, the results however did not meet the scores obtained individually. It is evident though that higher accuracy is acquired when we consider the difference or quotient of those features, instead of just concatenating them. Finally, except for MFD and HFD, aggregated sets between the mentioned feature types do not provide a statistically significant improvement in recognition. We shall only mention that some valence scores including PSD features surpass some of those we report in the ablation study.

VI. CONCLUSION
In this paper we analyzed the multiscale fractal structure of EEG and proposed a feature extraction method utilizing two multifractal algorithms for emotion recognition. The proposed features perform strongly against the baselines, particularly in the challenging subject-independent setting and in arousal recognition, indicating that arousal is correlated with the fragmented structure of the EEG. Further improvements are achieved when the fractal dimension features are aggregated, while the efficiency of the alpha frequency band is underlined in all experiments. Our analysis showed that multifractality and the anti-correlation properties should be considered when processing EEG signals. Further work on EEG emotion analysis should consider feature extraction algorithms for determining asymmetrical multifractal properties, whereas an interesting direction would be the examination of the correlation between brain signals and their stimuli.