{'fraction_non_alphanumeric': 0.026639975361872497, 'fraction_numerical': 0.0055820757622420695, 'mean_word_length': 5.429950495049505, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Adaptive abnormal flow detection method for new energy stations based on HHT algorithm

With the development of new energy technology, new energy stations are becoming more intelligent and data-based, and cyber-attacks on new energy stations are increasing year by year. In response to the continuous threats brought by malicious traffic to the network of new energy stations, this paper researches on the traffic anomaly detection technology based on network communication characteristics. An adaptive abnormal traffic detection method for new energy stations based on HHT algorithm is proposed, which improves the efficiency of identifying abnormal network traffic and more accurately identifies network attacks against new energy stations. It is verified through experiments that compared with mainstream classifiers, the method studied in this paper can achieve adaptive detection while adaptively determining the threshold, and the detection accuracy can reach 95%, the false alarm rate is lower than other methods, it can provide more accurate identification results for new energy field station network cyber-attacks detection.


Introduction
New energy refers to various energy other than traditional energy, mainly including wind energy, solar energy, biomass energy, etc., which is the general trend of future grid development. New energy power generation has good development prospects and practical value. As more and more new energy plants and stations are constructed and connected to the grid, the network security risks of new energy plants and stations have become increasingly prominent.
The security problems of new energy plants and stations may pose a threat to the security of the entire power grid. On the one hand, attacks from the network of new energy plants and stations may penetrate upwards, seriously threatening the security and stability of the entire grid-side dispatch control system. For example, if a new energy plant is attacked, it may cause a large-scale off-grid accident of the unit, which will eventually affect the stability of the grid system voltage and frequency; on the other hand, the network of the new energy plant has firewalls, encryption machines, isolation devices, etc. The necessary security protection capabilities, but its monitoring system, control system and networked information system are also very likely to be penetrated and attacked from the vertical boundary of the network, resulting in unpredictable two-way security consequences. Therefore, realtime security threat detection and security protection at the vertical network boundary of new energy plants and stations is an inevitable and urgent need; in response to this need, research on cyberspace security monitoring frameworks, anomaly detection algorithms, and security based on large-scale traffic real-time analysis Monitoring and attack early warning systems are bound to become an important trend in the development of power production network security technology.

Background and Existing abnormal traffic detection technology
In the past few decades, we have seen a series of cyber-attacks on CNI and SCADA. The 2010 computer worm Stuxnet attack caused damage to one-fifth of the centrifuges near Iran's nuclear facilities [1]. In 2011, five global energy and oil companies were attacked, including social engineering, Trojan horses, and Windows-based attacks [1]. In 2012, a malware named Flame was found to have been running on many websites in the Middle East and North Africa for at least two years [1], [3].
Miller and Rowe's analysis showed [1] that the number of cyber-attacks against CNI has been increasing over time. At the same time, the number of SCADA-related security incidents is also steadily increasing. In 2010, the Industrial Security Incident Library (RISI) recorded 161 incidents, and approximately 10 new incidents were added every quarter [4]. In 2013, the RISI company database had recorded 240 events from 2001 to the end of 2012 [5]. In addition, on the basis of extensive research on the network security status of a large number of SCADA systems, interviews with a large number of experts confirmed that the cyber threats in SCADA systems are escalating and they are "real and expanded" [6].
In view of the significant impact of the electric energy system on national security, in order to improve the network security of critical infrastructures, and for the energy field network security, in 2014, the US Department of Energy proposed a general network security capability maturity model in the US Department of Homeland Security. On the basis of (Cybersecurity Capability Matural Model, C2M2) [7], an ES-C2M2 (Electricity Subsector Cybersecurity Capability Matural Model) model specifically for the electric power field [8] is proposed.
Abnormal traffic detection technology, In the context of big data, machine learning algorithms are widely used in network security fields such as traffic anomaly detection, intrusion detection, and Trojan horse detection. At the same time, deep learning, integrated learning and other technologies are gradually being used in the security field. In terms of abnormal traffic detection, Tian Wang [9] [11] Et al. proposed a hybrid deep learning algorithm for abnormal network traffic detection. Chen Jilei et al. [12] proposed a deep learning model based on a sparse autoencoder, which improved the detection rate of intrusion detection algorithms and reduced the false alarm rate. However, when the amount of data is small, the detection effect is not ideal; Zhao Ying et al. [13] applied the Markov model to network traffic classification, and improved the semi-supervised learning network traffic classification method through the method of similarity calculation. Wu Jinlong et al. [14] used the random forest method for Trojan detection, which greatly improved the accuracy of Trojan detection, but did not demonstrate the detection of other abnormal traffic; Chen Lulu et al. [15] used gradient projection The negative matrix factorization method reconstructs the multi-dimensional entropy matrix, which has a good effect on the recognition of continuous abnormal traffic such as DDOS, but the effect of general anomaly detection is relatively unsatisfactory; Paweł [16]

HTT algorithm introduction and steps
The core of the HHT algorithm in the field of anomaly detection is two algorithmic processes: (1) Empirical Mode Decomposition (EMD) is performed on the time series of the original flow characteristic index value, and the original signal is decomposed into Eigen mode functions of different frequencies (Signal component) and a residual trend item; (2) Perform Hilbert Transform on the signal component, and use the concept of frequency spectrum to show the characteristics of the signal, and find anomalies in the way of energy spectrogram. The following describes the algorithm process: <1> Empirical mode decomposition(EMD) Assuming that the original signal is x(t), m(t) is the mean function of the upper and lower envelopes, let s(t)=x(t), h(t) is the intermediate variable of signal decomposition, and c(t) is IMF function. The signal decomposition process is as follows: <1-1> Find all the maximum and minimum points of the function x(t), construct the upper and lower envelopes through the cubic spline interpolation function, and calculate the mean function m(t); <1-2> Subtract the mean function m(t) from the function x(t) to get h(t), that is, h(t)=x(t)-m(t)； <1-3> Judge whether h(t) meets the IMF conditions, if not, set x(t)=h(t), repeat steps 1, 2, and 3 for the x(t) function, otherwise go to step 4. ； Represents the i-th eigenmode function， ) (t r n Represents residual trend items。 Taking a certain indicator "TCP flow size" as an example, the TCP flow size signal can be decomposed into multiple components, as shown in the figure below (the first sub-picture EEMD_TCP_ORIGINAL_PART3 is the original signal) ： Where P stands for Cauchy principal value, The analytical signal z(t) is defined as: term，called Hilbert Energy Spectrogram, as shown below:

Figure 2. Hilbert Energy Spectrogram
In the figure, the brighter the colour, the higher the energy, and the greater the probability that the energy-concentrated part indicates the flow is abnormal, so the abnormal time of the original signal can be determined from the time when the energy is abnormally concentrated.
Compared with traditional wavelet analysis and other methods, the advantages of HHT method are as follows： <1> HHT can analyze nonlinear and non-stationary signals In traditional data processing methods, there are strict requirements on signal processing. The more popular processing methods include Fourier transform and wavelet transform, among which wavelet transform can theoretically process nonlinear and non-stationary signals, but in practice Like the Fourier transform processing in the algorithm, it can only be a linear non-stationary signal. HHT can handle any non-linear and non-stationary signal, which has great advantages in signal analysis.
<2> HHT is adaptive In the process of signal processing by HHT, components can be generated adaptively, while most other algorithms need to set preconditions, such as Fourier trigonometric function, wavelet basis, etc., which are uncertain. However, HHT does not need to set function conditions in advance, and decomposes the signal adaptively according to the characteristics of the signal to obtain components with actual physical meaning, which is of great significance for adaptive anomaly detection.
<3> HHT is not restricted by Heisenberg's uncertainty principle According to Heisenberg's uncertainty principle, it can be known that certain physical quantities of a microscopic particle cannot have a definite value at the same time, that is, the more certain one quantity is, the greater the uncertainty of the other quantity. HHT is not restricted by Heisenberg's uncertainty principle, so that it can achieve high accuracy at the same time as time and frequency, while traditional signal processing methods are restricted by this. These methods cannot achieve the best accuracy at the same time in terms of time and frequency in principle.
Due to the above advantages, the HHT algorithm has been highly valued and applied in the field of anomaly detection. Several research results have also shown its advantages in the field of anomaly detection over signal analysis, data mining, neural networks and other methods. Based on this theory,

Adaptive anomaly detection technology for new energy stations based on HHT algorithm
This paper proposes an adaptive anomaly detection technology based on network traffic feature extraction and HHT algorithm. The anomaly detection technology can be divided into three levels: the first level is the expression of characteristic indicators, namely: real-time collection of large-scale network traffic, and through the frequency domain decomposition of various characteristic indicators, cycle calculation, variance analysis, energy analysis and other processes, Multi-dimensionally accurately portray the situational characteristics of network traffic. The second level is the expression of abnormal situation events, that is, through abnormal detection of characteristic indicators, microscopic abnormalities of the situation are found and abnormal events are triggered. The third level is event correlation and combined output, that is, correlate micro-level abnormal situation events to trigger higher-level macro-level abnormal situation alarms and attack alarms. And how to use signal processing methods to dig out hidden and possible abnormal signals from the time series of characteristic indicators, and to detect and warn them to realize abnormal discovery of unknown hidden situations.
The overall anomaly detection technology roadmap for the above 3 levels is shown in the figure below： As can be seen from the figure, the quantitative expression of network traffic security situation is divided into three steps, namely: (1) traffic feature collection and situation feature characterization; (2) situation feature-oriented adaptive learning and anomaly analysis; (3) abnormality Situation detection and warning. The following are described separately: (1)Flow collection and situation feature index extraction First, use traffic probes to capture and analyse traffic 7x24 hours to obtain situational characteristic indicators in multiple dimensions, and realize online real-time collection, logging, and storage of characteristic indicators. Every 1-minute (or a specified period) for the entire network, each subnet and a single IP host generate an analysis record, which contains the current values of all characteristic indicators. The project team has designed more than 40 characteristic indicators for network traffic in the early stage, and all of them are collected, stored and visualized in real time. Based on the above-mentioned basic processes such as flow collection, flow classification, and flow tracking, this project proposes a multi-dimensional situation feature extraction technology. The purpose is to design and collect multi-dimensional feature indicators in the flow, and describe the operating situation of the network space and network entities (host, host, Users, etc.).
According to the traffic of the whole network and the traffic of a single network entity, this paper proposes 6 categories and more than 40 dimensions of situation characteristic indicators, and respectively designs corresponding online mining algorithms to obtain the above-mentioned dimensional indicators from the traffic. The six categories are described as follows:  Traffic statistics feature indicators: This category of indicators provides a means to describe the summary statistical features of network macro or micro entity traffic. Can be used to detect cyber-physical attacks and large-scale active attacks.  Traffic morphology characteristic indicators: designed to characterize the traffic morphology of customer networks, including scale, user composition, application behavior patterns, etc. It can be used to detect large-scale active attacks, internal attacks, and cyberphysical attacks.  Characteristic indicators of encrypted communication behaviors: designed to provide a description and verification basis for encrypted sessions, encryption machines, and protocol compliance in encrypted networks. Can be used to find insider attack threats against encryption machines.  General behavior index: It is designed to describe the general behavior of the entire network, IP subnet, and host/user, and can be used to describe the situation of basic network services.
Can be used to detect active/inside attack threats.  Network spatial structure type characteristic index: It aims to characterize the network form, mainly for the whole network routing characteristics, server group and key backbone link routing characteristics. It can be used to detect cyber-physical attacks and specific types of active attacks.  Application access behaviour characteristic indicators: designed to characterize the application behaviour of users and application servers. Can be used to detect specific types of internal user attacks. The characteristic indicators of the above multiple dimensions can be automatically uploaded to a variety of big data platforms to provide storage and subsequent analysis, and online real-time monitoring can also be provided. The project will mine various indicators related to anomalies from the network traffic and form a log record every N minutes. The preliminary design of relevant indicators is as follows (the indicator system will be further expanded during the investigation of new energy plants): The number of suspected retransmissions of the same type of payload PROTOCOL_MATCH_MAX Number of protocol compliance detection alarm messages (2) Self-adaptive learning and abnormal analysis for situation characteristics On the basis of flow collection and situation feature extraction, an online learning algorithm is designed to learn the index data obtained by flow feature collection. The key technology of this step is adaptive learning and anomaly analysis technology. That is: how to adaptively learn, identify the normal state and normal pattern in the network, form a continuously updateable learning result, and apply the learning result for subsequent abnormality analysis and detection. The output of this step is the learning knowledge base of a single situation indicator and the abnormal analysis result of the current indicator value.
Compared with the traditional wavelet signal decomposition method, the advantages of the HHT method in this paper include: (1) It can analyse nonlinear and non-stationary signals. In traditional data processing methods, there are strict requirements on signal processing. The more popular processing methods include Fourier transform and wavelet transform, among which wavelet transform can theoretically process nonlinear and non-stationary signals, but in practice Like the Fourier transform processing in the algorithm, it can only be a linear non-stationary signal. (2) It is adaptive. In the process of HHT processing signals, components can be generated adaptively, and Fourier and wavelet transforms need to set preconditions. For example, the function base set by Fourier is a trigonometric function, and wavelet transforms select wavelets that meet certain conditions. Basis, and the result of wavelet transform will vary with the selected wavelet basis transform, which has uncertainty. However, HHT does not need to set the function conditions in advance, and decomposes the signal adaptively according to the characteristics of the signal to obtain components with actual physical meaning, which is of great significance for analysing the signal. (3) HHT is not restricted by Heisenberg's uncertainty principle. (4) The superiority of frequency solving method.
(3)Network traffic abnormal situation warning On the basis of the above learning and anomaly analysis, online alarms of abnormal situations are carried out. The online abnormal alarm output is divided into three categories: (a)The absolute value of a single characteristic index is abnormal. Once the current sampling value of any characteristic index breaks through the threshold (the upper limit of the learning range), an alarm record will be generated. This project intends to output alarms for all more than 40 situation characteristic indicators.
(b)A single characteristic indicator suddenly jumps to the alarm. The jump scale of the previous sample value and the current sample value of any characteristic index, once the jump threshold (upper limit of the learning range) is broken, an alarm record will be generated. This type of alarm also includes 40 characteristic indicators. According to the associated knowledge base of multiple features, the common anomaly of multiple features often represents an intelligible anomaly with a clear physical meaning (for example, DDOS attacks, etc., defined as "understandable anomalies"). Therefore, if The current sampling values or sudden jump scales of multiple characteristic indicators meet the requirements of an item in the associated knowledge base, and the association of multiple indicators can trigger a higher-level "understandable abnormality" alarm.

Algorithm steps
Based on this algorithm, this paper proposes a flow modelling method based on multi-dimensional quantitative features of digital signals. The method includes the following steps: <1>First, collect the actual flow of the target environment of the new energy plant online; <2>Secondly, carry out multi-dimensional feature extraction of traffic (extract more than 40 dimensions of traffic feature indicators such as statistical, morphological, encrypted communication, general behaviour, spatial structure, and behaviour), and analyse the network space Real-time quantification of the traffic situation; <3> Then, the Empirical Mode Decomposition (EEMD) method is used to decompose each characteristic index in the frequency domain to obtain signal components of different frequencies, and calculate the signal strength, period, variance, energy and other characteristic values of different components; <4>Finally, the multiple characteristic values of the above-mentioned multiple dimensional characteristic indicators together constitute the flow characteristic characterization indicator system, which is the flow model of the new energy plant.
The advantages of the modelling method in this article are: <1>Comprehensiveness, using multiple dimensional feature indicators to describe the characteristics of traffic more comprehensively, while traditional modelling methods only describe simple indicators such as traffic size, and can only complete low-dimensional descriptions; <2>Adaptability, the modelling process does not need to select a model or adjust parameters, and almost all existing models need to manually select models and adjust parameters, and do not have full adaptive modelling capabilities; <3>Real-time, the flow modelling process and flow collection process are completed in real time simultaneously, without offline analysis and training process of various data sets; <4>Practicality. The goal of this flow model is to accurately describe the flow characteristics and provide an accurate reference for anomaly detection, while traditional flow models are mostly for the purpose of flow mechanism research, flow fitting, and flow prediction, and are practical in anomaly detection Not strong.
In this paper, the real-time quantification process of network space traffic situation is realized by the method of network boundary traffic collection and characteristic characterization index system. This method continuously mines multiple dimensional feature indicators in real time from network traffic, draws network space and its traffic characteristics in real time. On the one hand, it can ensure the real-time or quasi-real-time monitoring, early warning, and emergency response requirements of the network situation, and on the other hand a small amount of log scale enables fine-grained characterization of network traffic characteristics. It can provides a high-quality basic information source for subsequent traffic anomaly analysis ,detection and security warnings. Compared with traditional methods, it has obvious advantages in real-time, depiction accuracy, data scale, and data quality.

Analysis of results
The experiment in this paper collects the flow of the vertical network boundary of the new energy plant for a week, and uses part of the DARPA2000 intrusion detection data set provided by Lincoln Laboratory, which contains samples of different types of attacks and abnormal traffic. At the same time, it is compared with the wavelet analysis method to verify the effectiveness of this method.
Taking a certain indicator "TCP flow size" as an example, the TCP flow size signal can be decomposed into multiple components, as shown in the following figure (Figure 1 EEMD_TCP_ORIGINAL_PART3 is the original signal), and Figure 2 and Figure 3 are the slaves of the original traffic after EEMD decomposition. 11 flow components from high frequency to low frequency, r(12) is the residual quantity.  Select the detection window as the component period, that is, the detection window size of IMF4 is 27min, and the detection window size of D4 is 22min. Calculate the variance of the components obtained by the EEMD method and the wavelet transform method, and perform probability statistics to fit the probability distribution. The fitting method adopts the commonly used curve close to the variance distribution, and the fitting function is relatively simple Gaussian fitting. According to the obtained fitting curve, the point with the largest curvature of the curve is used as the dividing point, and the variance corresponding to the dividing point is calculated. The variance is used as the threshold. If it is greater than the threshold, it is considered that there is abnormal traffic, otherwise it is considered that the traffic in the detection window belongs to normal traffic. After many tests and comparisons, the test results are shown in the following 30% It can be seen from the chart that the method studied in this paper can achieve 95% detection accuracy when the threshold is adaptively determined, and the false alarm rate is lower than wavelet.

In conclusion
This paper deeply identifies the security threats of new energy plants, analyzes the threat characteristics of the vertical network boundary of new energy plants, establishes the active monitoring and early warning capabilities of the network-related vertical network boundary attack threats of new energy plants, and real-time monitors and identifies the network from the plant side It can further threaten the security of the power grid dispatching control system, prevent network information security attacks against the new energy power plant monitoring system, prevent power security accidents caused by power monitoring system security incidents, and ensure the security of the new energy power plant and the power grid as the source of the power grid Stable operation. The adaptive abnormal traffic detection technology for new energy farms and stations based on the HHT algorithm proposed in this paper improves the efficiency of identifying abnormal network traffic, and more accurately identifies cyber-attacks against new energy farms and new energy farms. Site's cyber-attacks. The experimental results show that the method studied in this paper can achieve adaptive detection while adaptively determining the threshold, and the detection accuracy can reach 95%, and the false alarm rate is lower than that of wavelet, which can be used for new energy fields. Network detection provides more accurate recognition results.