{'fraction_non_alphanumeric': 0.03307944217536888, 'fraction_numerical': 0.010229237958567906, 'mean_word_length': 4.6798328108672935, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Overview of Data Mining Techniques Applied to Power Systems

The growth of available data in the electric power industry motivates the adoption of data mining techniques. However, the companies in this area still face several difficulties to benefit from data mining. One of the reasons is that mining power systems data is an interdisciplinary task. Typically, electrical and computer engineers (or scientists) need to work together in order to achieve breakthroughs, interfacing power systems and data mining at a mature level of cooperation. Another reason is the lack of freely available and standardized benchmarks. Because of that, most previous research in this area used proprietary datasets, which makes difficult to compare algorithms and reproduce results. This chapter has two mains goals and, consequently, is divided in two parts. In the first part, the goal is to present a brief overview on ho w data mining techniques have been used in power systems. There are several works, such as (Mori, 2002), that introduce data mining techniques to people with background in power systems. In contrast, this text assumes previous knowledge of data mining, describes some fundamental concepts of power systems and illustrates the kind of problems that the electric industry tries to solve with data mining. The second part of the work presents a thor ough investigation of a specific problem: classifying time series that represent short-circuit faults in transmission lines. Studies show that these faults are responsible for 70% of the disturbances and cascading blackouts (Kezunovic & Zhang, 2007). Besides, there is a large and growing number of publications about this problem. Two types of fault classification systems are discussed: on-line and post-fault. On-line fault classification must be performed on a very short time span, with the analysis segment (or frame) being located approximately at the instant the fault begins. Post-fault classification can be performed off-line and its input consists of a multivariate time series with variable length (duration). Post-fault is a sequence classification problem, while in on-line classification the input is a fixed-length vector. Both fault classification systems (and most data mining applications) require a preprocessing or front end stage that converts the raw

comprehensive labeled dataset. Such datasets are key components to allow reproducing research results in different sites, which is crucial given the large number of parameters to be tuned in a fault classification system. Therefore, in order to promote reproducible research, this work also provides detailed information about the adopted parameters. The chapter is organized as follows. In Section 2 a brief description of data mining applications in power systems is provided. The section also introduces basic concepts of power systems. For a more detailed treatment, the reader is refereed to (Casazza & Delea, 2005). The second part begins in Section 3, which poses the fault classification problem and discusses solutions. Section 4 presents simulation results and is followed by the conclusions.

Power systems for data miners 2.1. Concepts of interest
A typical power system is represented in Figure 1 and can be divided into three parts: generation, transmission and distribution. The distribution system delivers power to the end users (loads). Most systems adopt three phases (A, B and C), using three conductors to carry sinusoidal voltage waveforms that have an offset in time equivalent to 120 degrees. While the customers need low voltage values (hundreds of Volts), the transmission system typically uses much higher values for efficiency. The transformers are responsible for the up and down conversions of voltage values and are located in different parts of the system. Under normal conditions, the voltage waveform x(t) has a pre-established frequency (e.g., 60 or 50 Hz). Knowing the nominal value of the amplitude (e.g., 500 kV), it is convenient to normalize x(t) by this value and report the amplitude in p.u. (per unity). In this case, the ideal waveform could be expressed by x(t) = cos(2 π f t + θ), where f is the frequency and θ the initial angle. Figure 2(a) illustrates a segment of the ideal voltage waveform for a frequency f = 60 Hz and θ = 0 radians. Figure 2(b) depicts simultaneously all three voltage waveforms in a segment containing a fault recorded by an oscillograph recording equipment: a short circuit between the conductor corresponding to phase B and ground (G). It can be seen that, besides phase B, the other two phases are also disturbed. Such faults belong to a category of events that is called transients because they tend to disappear after proper operation of the system to recover normal conditions, as occurs after approximately 0.05 seconds in Figure 2(b). In order to properly operate it, a power system contains several data acquisition equipments. For example, some of these equipments register the status of logical (boolean) variables at each minute, while others store waveforms digitized at relatively high sampling www.intechopen.com rates (e.g., 5 kHz). In many cases it suffices to monitor the root mean-square (RMS) value of each waveform estimated at each second. Figure 3 illustrates the information about the voltage amplitude that is provided by the RMS estimation. Fig. 2. a) Example of ideal normalized voltage waveform of one phase. b) All three phases with short-circuit between phase B and ground, as registered by an oscillograph equipment. Fig. 3. Example of the RMS waveforms superimposed to the voltage waveforms in a fault between phase B and ground. It can be seen that the voltage in phase B drops to a value ower than 1 p.u., while phases A and C achieve values above the nominal.

www.intechopen.com
In summary, most power systems data can be considered as multivariate time series, but the sampling frequencies may differ significantly and the variables are eventually categorical (not numerical). Roughly, one can organize the data originated in power systems into three categories: a. Raw waveforms (voltages and currents) sampled at relatively high sampling frequencies; b. Pre-processed waveforms (e.g., RMS) typically sampled at low sampling frequencies; c. Status variables (e.g., if a relay is opened or closed) typically sampled at low sampling frequencies. Due to its higher volume of information, the first category data is sometimes organized in specific databases, such as the oscillographic database, which stores all data from oscillographs. The other two categories are sometimes organized in the so-called Supervisory Control And Data Acquisition (SCADA) systems (Boyer, 1999). SCADAs are complex systems that periodically store several thousands of heterogeneous variables and are an important source of information for data mining. For example, automatically mining cause-effect relationships in SCADA data is an incipient but promising activity. Some power systems are affected by events that repeatedly cause troubles but their causes remain undetected. However, most of the times it is necessary to organize a data warehouse in order to be able to mine data from a SCADA, and fewer works use such data when compared to the first category. Data mining can also alleviate another problem in power systems: when a disturbance is detected, a large amount of messages and alarms are generated. Protection equipments are responsible for detecting a problem, and act appropriately, isolating the defective part of system, for example. Part of this operation is automatic, but some tasks depend on a specialist. The amount of information regarding the problem cannot be excessive but should be enough for making decisions. Data mining techniques can be used to filter alarms and messages and provide the important information to the operator. Failures in the performance of protection equipments, remote terminal, communications link and acquisition of date online, and variations in the voltage levels after of the occurrence disturbance, are factors that difficult the assessment and diagnosis in real time initial cause of power off. Another problem of interest to the electric power industry is load forecasting, in which the goal is to predict the demand for power in specific regions. This can be cast as a conventional regression problem. Power quality is another area that can benefit from data mining. Here the goal is to help characterizing how close to the ideal (nominal) parameter values the system is operating. Small and large deviations are categorized by detection and classification modules. The location where a power quality event happened is also of interest. The next section describes tasks and techniques used in data mining.

Review of data mining applications in power systems
This section briefly describes typical applications of data mining in electrical power systems via a collection of 18 papers. Table 1 lists the technique, task and application area. Among the several applications listed in Table 1, the second part of this work concentrates in fault classification, as discussed in the next section.

Reference
Technique Task Application Area Problem Ramos, 2008

Classification of time series representing faults
As mentioned, most transmission systems use three phases: A, B and C. Hence, a shortcircuit between phases A and B will be identified as "AB". Considering the possibility of a short-circuit to "ground" (G), the task is to classify a time series into one among ten possibilities: AG, BG, CG,AB, AC, BC, ABC, ABG, ACG and BCG. The ABC and ABCG faults are typically not distinguished because in well-balanced circuits (or ATP simulations) there is no current flow through the ground (Anderson, 1995). Algorithms to solve this classification problem are used by digital fault recorders (DFRs), distance relays and other equipments (Luo & Kezunovic, 2005). The signal capturing equipments are typically located at both endpoints of transmission line. Most of them are capable of digitizing both voltage and current waveforms. It is assumed that a trigger circuit detects an anomaly and stores only the interval of interest: the fault and a pre-determined number of samples before and after the fault. The trigger is out of the scope of the present work and the simulations assumed a perfect trigger algorithm, with the fault endpoints being directly obtained from the simulator. The next subsection describes the front end, the stage that is responsible for providing a suitable parametric representation of the time series. At some points the notation may look abusive, but there are many degrees of freedom when dealing with time series and a precise notation is necessary to avoid obscure points.

Front end
Each fault is a variable-duration multivariate time series. Then n-th fault n X in dataset (oscillography records, for example) is represented by a , is a multidimensional sample represented by a vector of Q elements. For example, if we consider voltage and current waveforms of phases A, B and C, then 6 = Q in the experiments. In some situations, it is possible to obtain synchronized samples from both endpoints of a given line. In these cases the sample is an augmented vector with twice the dimension of the single endpoint scenario. For the previous example, the sample dimension for double endpoint measures would be 12 = Q . A front end converts samples into features for further processing. An example of a modern front end algorithm is the wavelets decomposition (Vertelli & Kovacevic, 1995). Independent of the adopted parametric representation, a single sample typically does not carry enough information to allow performing reasonable decisions. Hence, it is useful to consider that a front end converts the matrix X in to a matrix Z with dimension N K × , as depicted in Figure 4 (the processing is performed on Z , not X ), where K is the number of features and N the number of feature vectors. A front end is called raw when it outputs features that correspond to values of the original samples, without any processing other than organizing the samples into a matrix Z . In the framed raw front end, this organization is obtained through an intermediate representation where N is the number of frames. The frames can overlap in time such that the frame shift S , i.e., the number of samples between two consecutive frames, is less than the frame length. Hence, the number of frames for a fault n X is: where ⎣⎦ . is the flooring function. www.intechopen.com The frames F (matrices) are conveniently organized as vectors of dimension QL K = , and It should be noticed that, if L S = (no overlap) and a frame is a concatenation of samples ( 2) the matrices X and Ẑ would coincide, i.e., X Z = . For example, in (Kezunovic & Zhang, 2007) the frames are composed by the concatenation of raw samples and vectors Z have dimension 198 = K . In more details, for example, If 6 = Q (currents and voltages), a concatenated raw front end could obtain frames F of dimension 5 6 × by concatenating to each central sample its four neighbours, two at the left and two at the right. In this case, assuming a fault with 10 = T samples and In this case, Ẑ and Z would have dimensions 10 6 × and 2 30 × , respectively. Figure 5 illustrates the segmentation in vectors z of features for one fault (ABG) with 4 frames. In this example, and this leads to three vectors z , each of dimension 18 = K . As an alternative to the raw front end, the wavelet transform provides information via a multi-resolution analysis (MRA) (Vertelli & Kovacevic, 1995). When adopting this front end special care needs to be exercised to fully describe the processing, given their large number of degrees of freedom. It is assumed a γ-level dyadic wavelet decomposition, which has γ stages of filtering and decimation (Vertelli & Kovacevic, 1995) and transforms each of the Q waveforms into γ+1 waveforms. More specifically, the q-th waveform is decomposed into approximation q a and details . For simplicity, the dependence on q is omitted hereafter. Some works in the literature use only one of the details or calculate the average power of the coefficients (Morais et al., 2007). In contrast, the framed wavelet front end keeps all the coefficients by taking in account that for 1 > γ they have different sampling frequencies and organizing them as matrix Z . For that, instead of using a single L , the user specifies a value min L for the waveforms with lowest s f ( a and γ d ) and a large value is another user-defined parameter.

The values are organized in a Frame
The number of frames for this organization of a wavelet decomposition is where T a is the number of elements in a.
The notation is flexible enough to easily describe several wavelet front ends, such as the concatenated wavelet (wavelet-concat, for which min S min L = ) and wavelet-energy described in (Morais et al, 2007). Many recent works adopt the wavelet front end ((Saibal, 2008); ; (Costa, 2006); (Chia-Hun & Chia-Hao, 2006); (Pang & Ding, 2008)). However, some of these works do not compare the wavelet with other (and eventually) simpler front ends. In Section 4, some results are presented with a simple RMS-based front end. It consists in taking the minimum RMS value of each phase during the whole fault duration. As a first step, a normalization is adopted (Morais et al., 2007) to to represent the voltage values in p.u. Because the feature vectors have dimension 3, it is relatively simple to visualize them. Figure 6 illustrates 1,000 Fig. 6. Vector of features obtained with a simple RMS front end, which represents each fault by a three-elements vector. The color and shape indicate the fault category according to the legend. of these vectors obtained via different simulations (more details in Section 4) with the color indicating the kind of fault. It can be seen, for example, that the monophasic faults (AG, BG e CG) and the triphasic fault (ABC) can be distinguished from the biphasics faults. Moreover, the biphasic faults that do not involve the ground are relatively similar to those biphasic faults involving ground.

On-line and post-fault classification
Fault classification systems can be divided into two types. The first one aims at performing a decision (classification) for each feature vector z or, equivalently, a frame F (giving that z is just a representation F as a vector). This is typically the goal in on-line scenarios, at the level of, e.g., a protection relay (Kezunovic & Zhang, 2007). Alternatively, the decision can be made at a supervisory center in a post-fault stage. The latter case makes a decision having available the whole matrix Z of variable dimension K×N n , where n distinguishes the individual faults, which having distinct durations in the general case. The on-line and postfault systems try to solve problems that can be cast as conventional classification (Witten & Frank, 2005) and sequence classification (Ming & Sleep, 2005) problems, respectively. On-line fault classification must be performed on a very short time span with the frame located in the beginning of the fault. It is often based on a frame corresponding to half or one cycle of the sinusoidal signal (typically of 60 or 50 Hz The accuracy of the system ) (Z G can be evaluated according to the misclassification rate and it is clearly dependent on the accuracy of the classifier ) (z F . The misclassification rates are s E and f E , for the post-fault (sequence) and on-line (frame) modules, respectively. In the case of post-fault systems, in spite of s E being the actual figure of merit, it is sometimes useful to also calculate E f . However, one should note that estimating f E takes in account all frames that compose a fault (frames in the beginning, middle and end of the fault). In online applications, such as relaying, taking a decision in the beginning of the fault is the most important. In order to take this situation in account, this work defines o E as the misclassification rate obtained when one considers only the first frame of the fault. The next section presents simulation results for fault classification.

Simulation results
The experiments used the UFPAFaults4 dataset, which can be downloaded from www.laps.ufpa.br/freedatasets/UfpaFaults. The UFPAFaults4 dataset is composed by 5,500 faults, organized into five sets of 100, 200, . . ., 1000 faults each. The division into these sets is to facilitate obtaining sample complexity curves (Vapnik, 1999). The sample complexity indicates how many training examples are required to train the classifier. It can be evaluated by observing how the performance varies with the number of training examples. Each fault in the dataset corresponds to three voltage and three current waveforms stored as binary files with an associated text (ASCII) files, which stores a description of the fault (its endpoints, label, etc.). The waveform samples are stored as real numbers represented as the primitive type float in Java (big-endian, 32-bits, IEEE-754 numbers). The faults are generated with the software AMAZONTP (Pires et al., 2005). Some parameters for the simulations are randomly generated. The values of all four resistances were obtained as i.i.d. samples draw from a uniform probability density function (pdf) U(0.1; 10), with support from 0.1 to 10 Ohms. The begin and duration (both in seconds) of the fault were draw from U(0.1; 0.9) and U(0.07; 0.5), respectively. The location was draw from U(2; 98) (percentage of the total line length). Eleven types of faults (AG, BG, CG, AB, AC, BC, ABC, ABG, ACG, BCG, ABCG) were generated using a uniform distribution.

www.intechopen.com
The voltage and current waveforms generated by the ATP simulations had a sampling period equal to 0.25 microseconds, corresponding to a sampling frequency 40 = s f kHz. It is possible to obtain versions with smaller values for s f by decimating the original waveforms. This operation requires low-pass filtering to avoid aliasing. Details about decimation and filtering can be found in digital signal processing textbooks, e.g. (Oppenheim, 1989).

Normalization
The elements of feature vectors z may have very different dynamic ranges (e.g., voltage in kV and currents in Amperes). This can cause the learning algorithms to perform poorly. Therefore, as a pre-processing stage, it is important to apply a normalization process. There are many algorithms for normalization of time series. This work adopted the so-called allfault (Morais et al., 2007), which takes in account all duration of the waveforms for getting the maximum and minimum amplitudes of each phase, and the converting the values to the range [-1, 1]. A distinct normalization factor is calculated for each of the Q waveforms.

Model selection
Often, the best performance of a learning algorithm on a particular dataset can only be achieved by tedious parameter tuning. This task is called model selection and corresponds, for example, to choosing parameters such as the number of neurons in the hidden layer for a neural network. A popular strategy for model selection is cross-validation (Witten & Frank, 2005). This is a computationally intensive approach, but avoids tuning the parameters by repeatedly evaluating the classifier using the test set. The test set should be used only once, after model selection, such that the error rate on this test set is a good indicator of the generalization capability of the learning algorithm. When dealing with frames extracted from sequences, it should be noted that, in conventional classification, the examples are assumed to be i.i.d. "samples" from an unknown but fixed distribution ( ) are extracted from contiguous frames of the same sequence given the relatively high similarity among them. Hence, in practice it is important to use crossvalidation properly, to avoid overtting due to a training set with similar vectors extracted from the same waveform. This work performed model selection via a validation set, disjoint to both training and test sets. A grid (Cartesian product) of model parameters is created and the point (set of parameters) that leads to the smallest error in the validation set is selected. For each coordinate, the user specifies the minimum and maximum values, the number of values and chooses between a linear or logarithmic spacing for the values.

Results
The simulations in this work relied on Weka (Witten & Frank, 2005), which has many learning algorithms. Specifically, the work used decision trees (J4.8, which is a Java version of C4.5 (Witten & Frank, 2005)), multilayer artificial neural network (ANN) trained with backpropagation, naïve Bayes and K-nearest neighbor (KNN). The choice of these classifiers was based in the fact that they are popular representatives of different learning paradigms (probabilistic, lazy, etc.). The parameters obtained by model selection for each classifier are summarized in Table 2. The KNN used the squared-error as distance measure and K = 1 neighbors. The naïve Bayes used Gaussian pdfs and does not have parameters to be tuned. For the ANN, H is the number of neurons in the hidden layer, N the maximum the number of epochs, L the learning rate and M the momentum (Witten & Frank, 2005). For J4.8, C is the confidence and M the minimum number of examples in a leaf.  Table 2. Summary of parameters for the front ends and two classifiers.
The results for frame-based classification using the concatenated raw front end are shown in Figure 7. The best results were obtained by the ANN, followed by the J4.8 classifier. The best frame length was 9 = L . It is interesting to note that for 1 = L , ANN achieved an error rate more than three times the one achieved by J4.8, which could be due to problems in convergence. Fig. 7. Error rate f E for several classifiers and frame lengths L using the concatenated raw front end.
The results obtained with RMS front end ( Figure 8 ) were inferior to the ones in Figure 7, for the raw front ends. But it is interesting to note that the described RMS front end, which uses only three numbers obtained reasonable results. Besides, the interpretation of decision trees trained with the RMS parameters provides a good insight about the problem. Figure 9 shows an example. In this case, the participation of a phase in the short circuit can be inferred by a relatively low minimum RMS value. For example, if this minimum value is above the threshold estimated from the data, then the corresponding phase should not be involved in the short-circuit. . b) Difference E f -E S between the erro rates for frame-by-frame and sequence classification.   The figure shows the mean and standard deviation. It can be seen that the error has a more erratic behavior than in Figure 11. Figures 11 and 12 show, respectively, the results for sample complexity of frame-by-frame E f and one-frame E 0 classification with 9 = = S L . Model selection was used for each value of M, given that the best parameters for the classifier typically depend on the number of training examples (Rifkin & Klautau, 2004) . Comparing Figures 11 and 12 one can conclude that more examples are needed to train a classifier that observes only the first frame of the fault and its misclassification rate E 0 is typically higher than E f under the simulated conditions.

Conclusions
This work presented an overview of data mining techniques used in power systems. Among several data mining tasks, fault classification is popular especially because it is relatively easy to generate artificial data using simulators such as ATP. Other applications of data mining will potentially impact the electric power industry, but this will require data warehouses to cope with preprocessing and organizing heterogeneous and large datasets. Even within fault classification, the research methodology needs improvement for an easier conversion of academic results into effective products. One important issue is the robustness of proposed algorithms to distinct power system constituent elements, such as the transmission line lengths. Several algorithms are tested with only one simulated scenario. This work shows that a very simple RMS front end, which represents each fault by only three values, can lead to misclassification rates under 1% in controlled conditions. Hence, it is important to improve benchmarks with publicly available datasets, such as the UFPAFaults4, and use them to properly evaluate new approaches. This chapter also presented results comparing different figures of merit for evaluating fault classification systems. It was shown that post-fault classifiers, which can take the whole fault segment in account to make a decision, achieve smaller error rates than classifiers based on a fixed-length (and short) frame. In fact, neural networks precisely classified all test examples (zero error) in some configurations. Another aspect that was emphasized is that, as vastly discussed in the machine learning literature, the number of examples to train a classifier depends on the learning algorithm and the domain (dataset). www.intechopen.com