{'fraction_non_alphanumeric': 0.02710843373493976, 'fraction_numerical': 0.007712668857247171, 'mean_word_length': 4.914439946018893, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The BeiHang Keystroke Dynamics Authentication System

Keystroke Dynamics is an important biometric solution for person authentication. Based upon keystroke dynamics, this paper designs an embedded password protection device, develops an online system, collects two public databases for promoting the research on keystroke authentication, exploits the Gabor filter bank to characterize keystroke dynamics, and provides benchmark results of three popular classification algorithms, one-class support vector machine, Gaussian classifier, and nearest neighbour classifier.


Introduction
Electronic Devices with internet access are an essential part of modern society.
Especially, Internet has greatly changed our lives, making our work and daily lives more convenient. However, it also brings us a big concern in information security. We depend so much on computers and Internet to store and process sensitive data, and it has become extremely necessary to secure them from intruders [1][2][3][4][5]. Our private information suffers from more risks than ever before.
To protect our information, we need to verify whether a user is legal using authentication and identification techniques. For user authentication and identification in computer based applications, there is a need for simple, low-cost and unobtrusive device. A user can be defined as a person who attempts to access information on the computer or internet in using the keyboard or touch screen. Currently, password is extensively and widely used to prevent user accounts from being intruded. However, too many methods can be used to decipher password, and once it is cracked, it will probably lead to a significant financial loss of the user would be caused. Such crimes on the Internet can cause a wide range of serious damages, and yet are difficult to be prevented. Therefore, to cope with such problems, we urgently need a more reliable way to protect our privacy.
Though the use of biometrics such as face, fingerprint and signature can improve the security, such techniques usually require additional tools to protect a device, which induces an extra cost in comparison with the password technique. The use of keystroke dynamics, which detects the typing pattern of an individual, can be an alternative for enhancing security without extra cost, as it can be obtained using the existing systems such as the standard keyboard. Keystroke dynamics is a kind of behavior feature [1,2]. It utilizes the rhythm and manner in which an individual types characters on a keyboard. The keystroke data contain the time of the key press and release (shown in Fig.1) from which two kinds of features are extracted, flight time and dwelling time. The flight time is defined as the time difference between one key release and the following key press. The dwelling time is the time difference between the press and release of one key. One of the major advantages of this biometric is that it is non-intrusive and can be applied covertly to enhance existing cyber-security systems.
It is also suitable to the touch screen of current mobile devices [1][2][3][4][5], which also contains two kinds of features: flight time and dwelling time. However, physical keyboards still remain the primary way for data entry in many electronic devices. Hence, in this paper we focus on keystroke dynamics on physical keyboards.
In 1980s, the National Science Foundation and the National Bureau of Standards in the United States conducted some studies concluding that typing patterns contain unique characteristics that can be identified. Researchers have explained the psychological basis behind the use of keystroke dynamics so as to provide researchers with a basic understanding of the various processes involved in typing [1][2][3][4][5]. Extensive studies show that a person's typing pattern is a behavioral characteristic that develops over a period of time and therefore cannot be shared, lost or forgotten. The patterns show sufficiently distinct information that can be used for identification and authentication [3]. Keystroke dynamics is still an on-going research topic as evidenced from recent publications [15][16][17]. Researchers have proposed many methods for keystroke dynamics [3][4][5][6][7][8][9][10][11][12][13]. The first research paper on keystroke dynamics [3] was published by Rand Corporation in 1980, which verifies that professional typists have distinguishable "styles" of typing as measured by patterns of expected times to type certain digraphs. In [4], Young and Hammon conducted an experiment to build a template from an individual's typing manner. Monrose and Rubin [5] constructed an identification system based on template matching and Bayesian likelihood models. Hu et al. [6] proposed a K-nearest neighbor based authentication method, which focuses on improving the efficiency while maintaining the performance as other methods. In [7], the researchers presented a method based on Hidden Markov Model, which achieves a reasonable performance. Some researchers applied neural networks to keystroke dynamics [8,9]. In [10], the authors developed a pressure-based user authentication system, where the discrete time signal is transformed into the frequency domain by using FFT. In [11][12][13], SVM was studied for keyboard dynamics, whose performance and efficiency are shown better than those with neural networks.
Similar to other biometrics techniques, public databases are very important to researches in the field of keystroke dynamics. However, to the best of our knowledge, there is no such database collected from a real commercialized system, while the benchmark databases in other biometric problems are generally ample. For example, the FERET and FRGC databases are commonly used in face recognition [18,19]. In palm print recognition, the PolyU dataset [14] is the open platform for comparative evaluation of different algorithms.
The aim of a public database is to allow different researchers to test their own algorithms based on the same dataset. In the same experimental environment, the comparisons between different algorithms are more reasonable. In [3], the work on the keystroke dynamics was based on long texts. Several people were asked to type the same paragraph of words. These experiments can show the uniqueness of keystroke behavior; however, they could not be used for practical applications. In this paper, we create two available databases to help researchers interested in keystroke dynamics so that better comparisons of methods and results can be performed. It should be noted that most of the previous experiments collected test samples in laboratory environments [13]. Therefore those datasets cannot represent a more general situation. So creating an open and comprehensive database from a commercialized system for keystroke dynamics is urgently needed. Meanwhile, password is predominantly used to protect people's accounts in many applications, and so an embedded system is in great demand for real applications. In this paper, we develop an electronic device which can be embedded with the password protection technique in the system. Previous works show that the classifier design is one of main topics in keystroke dynamics, and the one-class SVM is still the state-of-the-art algorithm [29]. In this paper, we focus more on feature extraction, and specifically investigate FFT, DCT and Gabor wavelet features to enhance the performance of the one-class SVM classifier.
The rest of the paper is organized as follows. In Section 2, we describe the proposed keystroke dynamics system and the framework of the embedded system used to create our databases, called BeiHang Keystroke Dynamics Databases. The details of the benchmark algorithms, the databases and experiments are presented in Sections 3 and 4, respectively.
Section 5 gives our experiment results and Section 6 concludes this paper with some discussions on future work.

The Proposed Keystroke Dynamics Systems
In this section, two keystroke dynamics systems are presented. One is designed for the Internet, which has already been commercilized in China. The details can be found from [26][27][28]. Another is an Embedded system, in which we build a single electronic device for keystroke dynamics authentification. For the research purpose in the field, four datasets are collected from the two systems.

The Keystroke Dynamics System for the Internet
There have been several methods used to capture keystroke information. The kernel (PC) based method is powerful and can obtain any information typed on a keyboard as it reaches the operating system. However, the synchronization problem is not well solved on multi-kernel computers, though it is effective and difficult to be detected by user-mode applications on single-kernel computers. Another widely used method is based on the HOOK keyboard APIs. It includes a series of functions which reveal the status of key press or release event. However, the HOOK function is generally based on a lot of APIs, which can lead to an increase in CPU usage. There are also other methods based on web browsing, but they are not secure in keystroke event detection. To deal with the above problems, we design an instance stream to capture the key press and release events. The proposed method is effective since the instance stream is complementary to the traditional HOOK function. The whole system consists of three parts: kernel level keyboard driver, preprocessing and authenticator module (Fig. 2). It uses the keyboard driver to collect user keyboard input information. The keyboard driver can efficiently capture keystroke dynamics information, and guarantee that the keystroke event is fully recorded. In our tests, the minimum response time of the keyboard driver is just 1/1000000 second, and the system achieves an excellent performance.
A commercialized system implementing the above method was deployed to different environments, such as Internet Cafe and laboratories. It involved a variety of individuals whose registration and log-in keystroke information was collected. Each user was asked to type his/her username once and his/her password 4 or 5 times in order to create a new account.
Some false data from the users' misuse of the system were included in the primary dataset. By using a filter, these false data were discarded and finally we created the BeiHang Keystroke Dynamics Database 1.

The Keystroke Dynamic Device for an Embedded System
An embedded system is designed for specific functions within a larger system, often with requirement of real-time computing. It is embedded as part of a complete device usually including hardware and mechanical parts. By contrast, a general-purpose computer, such as a personal computer (PC), is designed to be flexible and to meet a wide range of end-user needs.
A processor is an important unit in the embedded system hardware. It is the heart of the system. The key characteristic, however, is dedicated to handle a particular task. Since the embedded system is dedicated to specific tasks, design engineers can optimize it to reduce the size and cost of the product and increase the reliability and performance. Some embedded systems are mass-produced, benefiting from economies of scale. Complexity varies from low, with a single microcontroller chip, to very high with multiple units, peripherals and networks mounted inside a large chassis or enclosure.
The main processor of our embedded system is STC90C58AD type single-chip, which has 4kB of SDRAM and is fully competent for our task. Another main functional device is the keyboard. Generally, the keyboard used in a Single-Chip System (SCM) is specialized. These keyboards are costly, complicated and unreliable. Since the PS/2 keyboard widely used on PCs has several advantages such as low cost, compact and reliable, it is a perfect choice to use it in our SCM system. Currently, the mini-DIN 6pin is the PS/2 interface used widely on the PCs, as shown in Fig. 3. PS/2 can fulfill the slave device to the host and the host to the slave device communication. When the slave device sends data to the host device, it first checks the clock to confirm whether the clock is the high level. If so, the slave device transmits data.
Otherwise, the slave device has to wait until receiving the bus right of control. When the communication finishes from the slave device to the host device, the slave device changes the data state at the time that the clock is in high level and the host device reads the data state at the moment when the clock is on its failing edge.
The PS/2 keyboard employs the second scan code set, which has two different types: make code and break code. When a key is pressed or being pressed persistently, the keyboard sends make code to the host device, and at the time that the key is released, it sends the break code of the key to the host device. The illustration is shown in Fig. 4.

The BeiHang Keystroke Dynamics Databases
Pressing the key, sending the make code Releasing the key, sending the break code We have collected two databases by using the proposed embedded system and the online system respectively. It can be used by researchers to test their algorithms and eventually boost the development of keystroke dynamics. All the data in these databases are originally collected, without any manual modification.

Description of the Databases
Generally a password is represented by the following stream: ...,

Database Access
To download the databases for research purpose, one can visit mpl.buaa.edu.cn, or send an email to the corresponding author.

Benchmark Algorithms
The framework of our Keystroke Dynamic System is shown in Fig. 6. Feature extraction and classification algorithm are the main components and are discussed in detail in the following sections.

Feature Extractions
Suppose a password is represented by the following sequence: ..., , The above feature is also called the original feature. The number of the registration samples collected in the training procedure is 4 or 5..
In signal processing and object recognition, the transformed features are extensively studied [22][23][24][25], such as Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT), and Gabor wavelets. This paper investigates these three transformation methods to further enhance the performance of keystroke dynamics systems using the original feature directly.
Different from most of previous works focusing on classifier design, this paper design better features for performance improvement.
The Fast Fourier Transform (FFT) is first discussed by Cooley and Tukey [25]. SVM for one-class classification has been widely investigated. Researchers proposed a method of adapting the SVM methodology to the one-class classification problem in [20,21].
Essentially, after transforming the features via a kernel, they treat the origin as the only member of the second class. By using relaxation parameters, they can separate the image of the one class from the origin. Then the standard two-class SVM techniques are employed. As shown in Eq.(7),they phrased the problem in the following way: Suppose that a dataset has a probability distribution P in the feature space. Find a "simple" subset S of the feature space such that the probability that a test point from P lies outside S is bounded by some a priori specified value. It can be summarized as mapping the data into a feature space using an appropriate kernel function, and then trying to separate the mapped vectors from the origin with maximum margin (see Fig. 7).
where ' x ， i x are input vectors.
Gaussian classification is another common methodology in object recognition. In a Gaussian model, a set of data is characterized by the mean and covariance for each class within the data along the dimensions. New data points can then be classified by measuring their distances from each centroid and assigning the class of the closest centroid to it. In probability theory, the Gaussian distribution, is a continuous probability distribution that is often used as a first approximation to describe real-valued random variables that tend to cluster around a single mean value [1]. Suppose that the keystroke data follow the Gaussian distribution, and therefore we can build a Gaussian classifier using the training data. Then we can use the model for authentication. The Gaussian distribution is where x is a test sample and  and  are the covariance matrix and mean vector of the Gaussian model respectively. Noted that  is assumed to be a diagonal matrix to enhance computation efficiency.

Fusion of Features and Classifiers
Combination of different expert decisions is extensively studied in previous twenty years.
Combination methods can be grouped by the level at which they operate. The simplest way is in the feature level, where different kinds of features are concatenated into an extended feature vector. This combination inherits the advantages of different features, and any classifier is easily used with them to build the final classification model. Combination can also be done in the level of decision or output score, which is called classifier-level combination. It is a quite popular way as the score is generally considered as a new kind of feature. This paper investigates both methods for performance improvement.  (2) and (6),

Experiments
In this section, we present benchmark experimental results for some classification and feature extraction algorithms on the BeiHang Keystroke Dynamics Databases. The extensive experiments include the evaluation of different features, classifiers, and their fusions. We also show that specific rhythms for different individuals can lead to high performance, which can be used in practical applications, such as password protection.

Evaluation Criteria
In the experiments, we use the False Positive Rate and the True Positive Rate for evaluation metrics. The former is the percentage of intruders who can enter the accounts by imitating the typing manner of genuine users. The latter is the percentage of genuine users who can successfully log into the system with the right keystroke manner. By changing the threshold in the classification procedure, we obtain a series of True Positive Rates and False Positive Rates, and then we use these results to draw a ROC curve. The ROC curve is used for evaluation of various algorithms including the OC-SVM classifier, Gaussian classifier, and NN classifier with the original feature, Gabor, DCT, and FFT features, etc.. We also provide the Equal Error Rate (EER) for further evaluation of different methods. EER is the percentage where the False Positive Rate equals the False Negative Rate.

A. Comparative experiments in terms of the ROC curves
(1) The ROC curves with the original feature and OC-SVM on different datasets The comparative experiments on different datasets are first conducted with the original feature defined in Equation (2) and the OC-SVM classifier. OC-SVM uses the RBF kernel function, and the algorithm is form LIBSVM [21]. In the experiments, the training set is used in registration process, while the test set is for performance evaluation. We first use the original feature to get the baseline results. Dataset 1A in Fig. 8 represents  Since Database 1 has been well discussed in our previous work [29], all the ROC curves shown here are mainly conducted on BeiHang Keystroke Dynamics Database 2. As shown in  To see what extent the Gabor, FFT and DCT features affect the performance, we conduct the experiments to observe performance variations. Fig. 11 shows that the performance is significantly improved with the Gabor feature on Dataset A of database 2 in terms of the ROC curve, while the FFT and DCT features are slightly worse than the original feature. The improvements can also be observed from Fig. 13 and Fig. 14. The curve of the OriGabor feature rises faster nearby the origin. The Gabor feature and the OriGabor feature perform much better than the original one. It can also be observed that the OriFFT feature achieves a better performance than the original feature, even when the single FFT feature is worse than the original one.     (12) are applied in classification. Fig. 15 illustrates that the classifier-level combination performs better than the original feature. These results reveal that the more features are extracted from the raw input signal, the more effective information is reserved.
(4) Comparative experiments with customization of keystroke rhythm In this section, we perform an experiment on different datasets to show how the specific rhythm of keystroke dynamics affects the final performance, which provides a feasible way for commercial applications. On Dataset B of Database 2, we customize the rhythm of keystroke, that is to say, the dwelling and flight times are different for different input characters, which can be seen as a specific feature for different individuals. As shown in Fig. 16 and Fig.17, the performance on Dataset B of Database 2 is significantly better than those on Dataset A, which is not customized on the keystroke process.

B. Comparative experiments in terms of the Equal Error Rate (EER)
To further compare the performance of different features and classifiers, we show the experimental results in terms of Equal Error Rate (EER) in Tables 2-5. Part of the results contains those from our previous paper [29]. The experimental results in Table.2 show that the Gaussian classifier with the original feature achieves much better performance than the OC-SVM and NN classifiers, while the NN classifier is worst. It is indicated in Table 3 that on Dataset B of Database 2, the Gaussian classifier greatly outperforms the other classifiers.
However, the results on Datasets A and B of database 1 as shown in Tables 4-5 demonstrate that OC-SVM is better than the Gaussian classifier.