{'fraction_non_alphanumeric': 0.035137239634027644, 'fraction_numerical': 0.014891960288105898, 'mean_word_length': 4.9150834772596435, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Offline EEG-Based Driver Drowsiness Estimation Using Enhanced Batch-Mode Active Learning (EBMAL) for Regression

There are many important regression problems in real-world brain-computer interface (BCI) applications, e.g., driver drowsiness estimation from EEG signals. This paper considers offline analysis: given a pool of unlabeled EEG epochs recorded during driving, how do we optimally select a small number of them to label so that an accurate regression model can be built from them to label the rest? Active learning is a promising solution to this problem, but interestingly, to our best knowledge, it has not been used for regression problems in BCI so far. This paper proposes a novel enhanced batch-mode active learning (EBMAL) approach for regression, which improves upon a baseline active learning algorithm by increasing the reliability, representativeness and diversity of the selected samples to achieve better regression performance. We validate its effectiveness using driver drowsiness estimation from EEG signals. However, EBMAL is a general approach that can also be applied to many other offline regression problems beyond BCI.


I. INTRODUCTION
EEG-based brain computer interfaces (BCIs) [20], [26], [30], [37], [40] have started to find real-world applications. However, usually a pilot session is required for each new subject to calibrate the BCI system, which negatively impacts its utility. It is very important to minimize this calibration effort, i.e., to achieve the best learning (classification, regression, or ranking) performance using as little subject-specific calibration data as possible.
There have been many approaches to reduce the BCI calibration effort. They can roughly be categorized into three groups: 1) methods to extract more robust and representative features, e.g., deep learning [18], [27], Riemannian geometry [3], etc.; 2) methods to make use of axillary data from similar/relevant tasks, e.g., transfer learning/domain adaptation [41], [45], [46], multi-task learning [2], etc.; and, 3) methods to optimize the calibration experiment design to generate or label more informative training data, e.g., active learning (AL) [21], [28], active class selection [47], etc.. It is interesting to note that these three groups are not mutually exclusive; in fact, methods in different groups can be combined for even better calibration performance. For example, active class selection and transfer learning were combined in [43] to reduce the calibration effort in a virtual reality Stroop task, AL and transfer learning were combined in [42] for a visually evoked potential oddball task, and AL and domain adaptation were combined in [44] to reduce the calibration effort when switching between different EEG headsets.
This paper focuses on the third group, more specifically, AL to reduce offline BCI calibration effort, which considers the following problem: give a pool of unlabeled EEG epochs, how to optimally select a small number of them to label so that the learning (classification or regression) performance can be maximized?
Considerable research has been done in this direction for classification problems in BCI [8], [21], [28], [29], [42], [44], [49], but to our best knowledge, AL has not been used for regression problems in BCI. In fact, compared with the extensive literature on AL for classification problems [34], AL for regression in general is significantly under-studied, not only for BCI. However, there are many interesting and challenging regression problems in BCI, e.g., driver drowsiness estimation from EEG signals [22], [23], [38], [41]. This is very important because according to the U.S. National Highway Traffic Safety Administration (NHTSA) [36], 2.5% of fatal motor vehicle crashes between 2005 and 2009 (on average 886 annually in the U.S.) and 2.5% of fatalities (on average 1,004 annually in the U.S.) involved drowsy driving. In our previous research we have focused on online driver drowsiness estimation from EEG signals [41]. This paper considers offline analysis: given a pool of unlabeled EEG epochs recorded during driving, how do we optimally select a few to label so that an accurate regression model can be built from them to label the rest of the epoches? This paper proposes a novel enhanced batch-mode active learning (EBMAL) approach for regression, which improves upon a baseline AL algorithm by increasing the reliability, representativeness and diversity of the selected samples to achieve better calibration performance. We use driver drowsiness estimation from EEG signals as an example to show that it significantly outperforms a baseline random sampling approach and two other AL approaches. However, our approach can also be applied to many other offline regression problems beyond BCI, e.g., estimating the continuous values of arousal, valence and dominance from speech signals [48] in affective computing.
The remainder of this paper is organized as follows: Section II-D introduces two baseline AL approaches and the proposed EBMAL approach to enhance them. Section III describes the experiment setup and compares the performance of EBMAL with several other approaches. Section IV draws conclusions.

II. ENHANCED BATCH-MODE ACTIVE LEARNING (EBMAL)
Our proposed EBMAL approach can be augmented to many existing AL algorithms to improve their performance. In this section we introduce two popular AL for regression approaches, point out their limitations, and show how they can be improved by EBMAL.
More specifically, assume in a regression problem there are N unlabeled samples {x n } N n=1 , the committee consists of P regression models, and the pth model's prediction for the nth unlabeled sample is y p n . Then, for each unlabeled sample, the QBC approach first computes the variance of the P individual predictions, i.e. [5], and then selects the top a few samples which have the maximal variance to label.

B. AL for Regression by Expected Model Change Maximization (EMCM)
Expected model change maximization (EMCM) is also a very popular AL approach for classification [7], [32]- [34], ranking [14], and regression [6] problems. Cai et al. [6] proposed an EMCM approach for both linear and nonlinear regression. In this subsection we introduce their linear approach, as only linear regression is considered in this paper.
Like in QBC, EMCM in [6] also uses bootstrap to construct P linear regression models. Assume the pth model's prediction for the nth unlabeled sample x n is y p n . Then, for each unlabeled sample, it computes whereȳ n is again computed by (2). EMCM finally selects the top a few samples which have the maximal g(x n ) to label.

C. Limitations of the QBC and EMCM Approaches
The above QBC and EMCM approaches, which will be called baseline AL approaches subsequently, have several limitations: 1) Usually the first batch of the samples for labeling are randomly selected, because the regression models cannot be constructed at the very beginning when no labeled data are available. However, there can still be better initialization approaches to select more reliable and representative seedling samples, without using any label information. 2) Sometimes the selected samples may be outliers, and hence labeling them not only waste the labeling effort, but may also deteriorates the regression performance. The baseline QBC and EMCM approaches do not have a mechanism to prevent outliers from being selected.
3) The baseline QBC and EMCM approaches consider each sample in the same batch independently, and no action is taken to reduce the redundancy among them, e.g., multiple selected samples in the same batch may be very close to each other, and hence using only one of them may be enough. The redundancy can be reduced by increasing the diversity of the samples in the same batch.

D. EBMAL
In response to the above three limitations, we propose EBMAL in Algorithm 1, which employs the following three intuitive heuristics to improve the reliability, representativeness and diversity of samples selected by a baseline QBC or EMCM approach.
First, to select more reliable and representative seedling samples in the first batch, we perform k-means clustering on all unlabeled samples, where k equals the batch size. We then compute the number of samples in each cluster to check if any cluster has a size no bigger than a certain empirical threshold, e.g., max(1, 0.02N ). If so, then the samples in that cluster are very likely to be outliers, and hence they are marked and restrained from being selected. We then perform k-means clustering again on the remaining unlabeled samples and repeat the check, until the number of samples in every cluster passes the size threshold. Then, for each cluster, we identify the sample that is closest to its centroid and select it for labeling. In this way we have selected k samples in the initialization batch that are representative and diverse to label.
Second, to prevent potential outliers from being selected, we record all such samples from the initialization step and restrain them from consideration in all future iterations.
Third, in subsequent iterations after the initialization, instead of selecting directly the top k unlabeled samples from a baseline AL approach, we now pre-select the top 2k samples using the baseline AL approach, and then perform k-means clustering on them, where k again equals the batch size. This step partitions the 2k samples into k groups according to their mutual distances. Then, for each cluster, we select the most informative sample (according to the baseline AL approach) for labelling. This ensures that the selected samples in the same batch are well-separated from each other, i.e., diversity is maintained.

A. Experiment Setup
The experiment and data used in [41] was again used in this study. We recruited 16 healthy subjects with normal/correctedto-normal vision to participant in a sustained-attention driving experiment [9], [10], consisting of a real vehicle mounted on a motion platform with 6 DOF immersed in a 360-degree virtual-reality (VR) scene. The Institutional Review Board of the Taipei Veterans General Hospital approved the experimental protocol, and each participant read and signed an informed consent form before the experiment began. Each experiment lasted for about 60-90 minutes and was conducted in the afternoon when the circadian rhythm of sleepiness reached its peak. To induce drowsiness during driving, the VR scenes simulated monotonous driving at a fixed speed (100 km/h) on a straight and empty highway. During the experiment, random lane-departure events were applied every 5-10 seconds, and participants were instructed to steer the vehicle to compensate for them immediately. The response time was recorded and later converted to a drowsiness index. Participants' scalp EEG signals were recorded using a 500Hz 32-channel Neuroscan system (30-channel EEGs plus 2-channel earlobes), and their cognitive states and driving performance were also monitored via a surveillance video camera and the vehicle trajectory throughout the experiment.

B. Preprocessing and Feature Extraction
The preprocessing and feature extraction methods were almost identical to those in our previous research [41], except that herein we used principal component features instead of the theta band power features for better regression performance.
The 16 subjects had different lengths of experiment, because the disturbances were presented randomly every 5-10 seconds. Data from one subject was not correctly recorded, so we used only 15 subjects. To ensure fair comparison, we used only the first 3,600 seconds data for each subject. We defined a function [38], [41] to map the response time τ to a drowsiness index y ∈ [0, 1]: τ 0 = 1 was used in this paper, as in [41]. The drowsiness indices were then smoothed using a 90-second square movingaverage window to reduce variations. This does not reduce the sensitivity of the drowsiness index because the cycle lengths of drowsiness fluctuations are longer than 4 minutes [24]. We used EEGLAB [12] for EEG signal preprocessing. A 1-50 Hz band-pass filter was applied to remove high-frequency muscle artifacts, line-noise contamination and direct current drift. Next the EEG data were downsampled from 500 Hz to 250 Hz and re-referenced to averaged earlobes.
We tried to predict the drowsiness index for each subject every 10 seconds, which is called a sample point in this paper. All 30 EEG channels were used in feature extraction. We epoched 30-second EEG signals right before each sample point, and computed the average power spectral density (PSD) in the theta band (4-7.5 Hz) for each channel using Welch's method [39], as research [25] has shown that theta band spectrum is a strong indicator of drowsiness. The theta band powers for three selected channels and the corresponding drowsiness index for a typical subject are shown in Fig. 1(a). The correlation coefficients between the drowsiness index and CZ, T5 and CP5 theta band powers are 0.3005, 0.2706, and 0.3129, respectively, indicating considerable correlation. Next, we converted the 30 theta band powers to dBs. To remove noises or bad channel readings, we removed channels whose maximum dBs were larger than 20. We then normalized the dBs of each remaining channel to mean zero and standard deviation one, and extracted a few (usually around 10) leading principal components, which accounted for 95% of the variance. The projections of the dBs onto these principal components were then normalized to [0, 1] and used as our features. Three such features for the same subject in Fig. 1(a) are shown in Fig. 1(b). The correlation coefficients between the drowsiness index and the first three principal component scores are 0.2094, -0.6518 and 0.0169, respectively. Note that the maximum correlation is significantly improved by using the principal component features.

C. Algorithms
We compare the performances of five different sample selection strategies: 1) Baseline (BL), which randomly selects unlabeled samples for labelling. 2) QBC [5], which has been introduced in Section II-A. 3) Enhanced QBC (EQBC), which is the QBC above enhanced by the EBMAL. 4) EMCM for linear regression [6], which has been introduced in Section II-B. 5) Enhanced EMCM (EEMCM), which is the EMCM above enhanced by the EBMAL.
All five approaches build a linear ridge regression model from the labeled samples, as in [41]. The ridge parameter σ = 0.01 was used in all five algorithms.

D. Evaluation Process and Performance Measures
From the experiments we already knew the drowsiness indices for all ∼360 samples, obtained every 10 seconds from the first 3,600 seconds data. To evaluate the performances of different algorithms, for each subject, we first randomly selected 80% of the ∼360 samples as our pool 1 , and then identified five samples to label in each batch by different algorithms, built a ridge regression model, and computed the root mean squared error (RMSE) and correlation coefficient (CC) as performance measures. The maximum number of samples to be labeled was fixed to be 60, corresponding to 12 batches.
We ran this evaluation process 30 times, each time with a randomly chosen 80% population pool, to obtain statistically meaningful results.

E. Experimental Results
The average RMSEs and CCs for the five algorithms across the 15 subjects are shown in Fig. 2, and the RMSEs and CCs for the individual subjects are shown in Fig. 3. Observe that all methods give better RMSEs and CCs as m increases, which is intuitive. QBC and EMCM had very similar performance: both were comparable to or slightly worse than BL for small m, but as m increased, they started to outperform BL. Remarkably, with the help of EBMAL, both EQBC and EEMCM outperformed the other three approaches for all m, although the performance improvement of EQBC and EEMCM over QBC and EMCM diminished as m increased. 1 For a fixed pool, EQBC and EEMCM give a deterministic selection sequence because there is no randomness involved. So, we need to vary the pool in order to study the statistical properties of EQBC and EEMCM. We did not use the traditional bootstrap approach, i.e., sampling with replacement to obtain the same number of samples as the original pool, because bootstrap introduces duplicate samples in the new pool, which does not happen in practice (a subject cannot have completely identical EEG responses at two different time instants), and also worsens the performances of QBC and EMCM (they may select multiple identical samples to label in the same batch). To better visualize the performance differences among different algorithms, in Fig. 4 we plot the percentage performance improvement between different pairs of algorithms. Observe that although QBC and EMCM did not outperform BL for small m, the corresponding EQBC and EEMCM achieved the largest performance improvements over BL (and also QBC and EMCM) for small m, especially when m = 0. This indicates that the new initialization strategy in EBMAL is indeed effective.
We also performed non-parametric multiple comparison tests using Dunn's procedure [15], [16] to determine if the differences between different pair of algorithms were statistically significant, with a p-value correction using the False Discovery Rate method [4]. The p-values for RMSEs and CCs for different m are shown in Tables I and II, respectively, where the statistically significant ones are marked in bold.
Observe that QBC and EMCM had statistically significantly better RMSEs and CCs than BL for large m, but EQBC and EEMCM had statistically significantly better RMSEs and CCs for almost all m. The performance improvement of EQBC over QBC, and EEMCM over EMCM, was statistically significant for small m. It is also interesting to study if each of the three enhancements proposed in Section II-D are necessary, and if so, what their individual effect is. For this purpose, we constructed three modified versions of the EBMAL algorithms: EBAML1, which employs only the first enhancement on more representative initialization; EBAML2, which employs only the second enhancement on better outlier handling; and, EBMAL3, which employs only the third enhancement on diversity. We then applied them to EMCM (the resulting algorithms are called EEMCM1, EEMCM2, and EEMCM3, respectively) and compared their performances with the baseline EMCM and the complement EEMCM. The results, averaged over 30 runs and 15 subjects, are shown in Fig. 5. Observe that every enhancement outperformed the baseline EMCM. More specifically, the first enhancement on more representative initialization helped when m was very small, especially at zero; the second and third enhancements on outlier handling and diversity helped when m became larger. By combining the three enhancements, EEMCM achieved the best performance at both small and large m. This suggests that the three enhancements are complementary, and they are all essential to the improved performance of EBMAL. In summary, we can conclude that our proposed EBMAL approach can significantly enhance a baseline AL for regression approach, especially when the number of labeled samples is very small (including zero). The three enhancements in EBMAL all contribute to its superior performance.

IV. CONCLUSIONS
Reducing the calibration data requirement in BCI systems is very important for their real-world applications. In our previous research we have extensively studied this in both online and offline BCI classification problems [21], [28], [42], [44]- [46], and also online regression problems [41]. This paper has proposed a novel EBMAL approach for offline BCI regression problems, and used EEG-based driver drowsiness estimation as an example to validate its performance. EBMAL solves the following problems: given a pool of unlabeled samples, how do we optimally select a small number of them to label so that an accurate regression model can be built from them to label the rest? Our proposed approach improves upon a baseline AL algorithm by increasing the reliability, representativeness and diversity of the selected samples to achieve better regression performance. To our best knowledge, this is the first time that active learning is used for regression problems in BCI. However, EBMAL is general and it can also be applied to many other offline regression problems beyond BCI, e.g., estimating the continuous values of arousal, valence and dominance from speech signals [48] in affective computing.