{'fraction_non_alphanumeric': 0.04004022305512387, 'fraction_numerical': 0.018877411097906572, 'mean_word_length': 4.548820694902359, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Repetitive Transients Extraction Algorithm for Detecting Bearing Faults

This paper addresses the problem of noise reduction with simultaneous components extrac- tion in vibration signals for faults diagnosis of bearing. The observed vibration signal is modeled as a summation of two components contaminated by noise, and each component composes of repetitive transients. To extract the two components simultaneously, an approach by solving an optimization problem is proposed in this paper. The problem adopts convex sparsity-based regularization scheme for decomposition, and non-convex regularization is used to further promote the sparsity but preserving the global convexity. A synthetic example is presented to illustrate the performance of the proposed approach for repetitive feature extraction. The performance and effectiveness of the proposed method are further demonstrated by applying to compound faults and single fault diagnosis of a locomotive bearing. The results show the proposed approach can effectively extract the features of outer and inner race defects.


Introduction
Rolling bearings are one of the most prevalent components in rotating machines and reciprocating machines [35]. Vibration-based fault detection has become the preferred technique for bearing fault diagnosis [44,18]. Bearing vibrations are usually modeled as cyclostationary signals (or groupsparse periodic) [9,10,4]. When a localized defect occurs on the bearing, repetitive transients will be generated due to the passing of rollers over the defect [26,38]. These transients have repetitive structure and are usually submerged in background noise. Many denoising methods have been introduced to extract fault features for the purpose of detecting faults in machines, such as wavelet transform [43,12], singular value decomposition (SVD) [27], time-frequency analysis methods [21,23], empirical mode decomposition (EMD) [30], methods for blind extraction of a cyclostationary signal and spectral kurtosis (SK) [9,10,5]. If compound faults exist, then the observed vibration signals are rather complex and it is difficult to identify each fault using traditional signal processing methods.
A number of approaches have been developed for the multiple fault or compound fault diagnosis.
Principal component analysis (PCA) based method has been used to classify faults [29,31,33]. Support vector machine (SVM) based methods have been used for multi-fault diagnosis [46,1] as well. Some other techniques such as neural network and independent component analysis (ICA), have been introduced to assist fault detection and classification [42,8,7]. Some of these methods require large data collection as a training set and further require an off-line training phase; and some methods rely on careful selection of features (e.g., wavelet packet sub-bands).
Adopting sparsity in the field of fault detection was initially illustrated in Ref. [45], where basis pursuit denoising (BPD) [14] was used to exploit sparse features in various domains to detect faults. Some recent works consider other sparse representations for fault diagnosis [44,16,25,17]. The fault frequencies of potential bearing fault features can be simply obtained using the geometry of the components in many cases or directly obtained from the user operation manual. Many works have considered the fault frequencies as priori information [44,32,40,34,24].
In this work, a method using the temporal periodicity (namely fault characteristic or fundamental frequency) directly in the time domain is proposed to extract fault features while simultaneously denoising. The proposed method is based on convex optimization using non-convex regularization.
Specifically, this paper aims to address the problem of extracting compound features caused by faults in vibration signals, where the features exhibit repetitive group sparsity.
In particular, the observed signal is modeled as where w denotes additive white Gaussian noise (AWGN), and x 1 and x 2 are both repetitively group-sparse signals with periods T 1 and T 2 respectively. Note that here the repetitively groupsparse signal means the useful features (group-sparse) appear repetitively. In other words, the features have group-sparse property and appear repetitively. To avoid confusion, here the "period" is referred to as the cycle for cyclostationary signal in Ref. [2]. In the case of compound faults detection of bearings, useful features x 1 and x 2 also satisfy the following two conditions.
1. The periods T 1 and T 2 are different.
2. Each period is not close to an integer multiple of the other.
Note that, in real applications, the fault features are not strictly group-sparse periodic. In other words, the real fault frequencies have a variation from the calculated frequencies (up to 1-2%), i.e., there are not strict periods T 1 and T 2 [35]. In such cases, the proposed approach can still work owing to the overlapping of group structure, which will be shown in the following sections.
The proposed approach is not an improvement of the cutting edge techniques developed in the cyclostationary framework, instead, it is an alternative to these techniques. A work closely related to the proposed approach is the periodic overlapping group sparsity (POGS) problem [24], which assumes only one repetitive group-sparse component is present in the vibration signal. Notice that although the observed signal is modeled with two components, the proposed method also works when there exist only one fault component. Therefore, the proposed method generalizes POGS, and is useful when there exists multiple components.
Another related work is group-sparse signal denoising (GSSD), which is also known as overlapping group sparsity (OGS) with non-convex regularization [13], where mathematical derivations and proofs have been given in detail to show that non-convex regularization can be used to promote group-sparsity, while maintaining convexity of the problem as a whole.
The method proposed in this paper also uses the concept of morphological component analysis (MCA) [39], which is a method to decompose signals based on sparse representations. In contrast to MCA, the proposed method does not utilize any transform (e.g., Fourier or wavelet transform), i.e., the sparse features are in the signal domain. Moreover, non-convex regularization is used to strongly induce sparsity while maintaining convexity of the proposed problem formulation. The proposed approach reduces to the OGS method without prior knowledge, i.e., we can utilize the sparsity-based OGS approach if we do not have prior knowledge of the characteristic frequencies.

Notation
In this paper, the elements of a vector x are denoted as x n or [x] n . The norms of x are defined as A function of x determined by parameter a is denoted as f (x; a), and to distinguish it from a function with two ordered arguments, e.g., f (x, y).

Review of majorization-minimization
In this paper, the majorization-minimization (MM) approach is used to derive a fast-converging algorithm. This subsection briefly describes the MM approach for minimizing a convex cost function. The MM is an approach to simplify a complicated optimization problem into a sequence of simpler ones [22]. More specifically, consider an optimization problem Using MM, the problem can be solved iteratively by where F M : R N × R N → R is an upper bound (majorizer) of the objective function F , satisfying Note that the majorizer F M (u, v) touches F (u) for u = v, as shown in (5). Figure 1 illustrates the majorizer (red line) of a penalty function (blue line), which will be described in the following subsection. The proof of convergence for MM has been given in Ref. [28,Chapter 10]. More details about the MM procedure can be found in [28,22] and references therein.

Non-convex penalty functions
Non-convex penalty functions can promote sparsity more strongly than convex penalty functions [36,37]. This subsection briefly describes the non-convex penalty functions which will be used in the proposed approach. The smoothed non-convex penalty function φ : R → R + is used in this work. Table 1 gives several examples of the functions defined by φ (u; a) := φ( u 2 + ; a), > 0, where φ is a non-smooth penalty function satisfying the following properties: 1. φ(u; a) is continuous on R.
Note that, for both φ and φ , the parameter a ≥ 0 controls the concavity of the function.
The parameter controls the smoothness of the functions. As a special case, when = 0, φ (u; 0) = φ(u), then the penalty function is non-differentiable at 0. In practice, is specified very small, e.g. 10 −10 , so that the function is differentiable. Figure 1 gives two specific examples of φ .
A majorizer of penalty function φ is given by where ψ(v; a) is listed in the third column of Table 1. Note that ψ(v; a) > 0 for all v ∈ R. Also note that φ M (u, v) is quadratic in u. In Ref. [19], a detailed proof has been given to show that when φ satisfies the above properties, the majorizer (7) satisfies the condition (5) for φ .

Review of POGS
The periodic overlapping group sparsity (POGS) problem [24] considers the signal model where x is a repetitive group-sparse signal, and w is additive white Gaussian noise. The POGS method extracts x by solving a convex problem where Φ : R N → R is defined as The function Φ is a regularization term that promotes repetitive group-sparsity. Moreover, b ∈ {0, 1} K is a binary weight array designated according to the period. To simplify following derivation, denote that where the penalty function is strictly convex as shown in Figure 1(a).

Problem formulation
To extract two repetitive group-sparse components, an optimization problem is formulated as The function R is an overlapping group sparsity (OGS) regularization function with group size K 0 1 . There are two more regularizers in (12) promoting the repetitive group-sparsity of x 1 and x 2 respectively, and formulated in (11).
Moreover, in contrast to MCA, which has two regularizers, problem (12) has three. The regularization term R is introduced because according to the signal model, the summation of the two components is also sparse. Note that, (x 1 + x 2 ) might be sparse when x 1 and x 2 are not, but in (12), the regularizers with Φ does force x 1 and x 2 to be sparse.

Convexity of the objective function
The idea of using non-covex regularization with "maximizing concavity" in a convex problem has been illustrated in [36], wherein the quadratic formulation of data fidelity term can be used to compensate the non-convexity in the regularization so that the objective function is still convex.
In this work, we adopt this idea for an extraction algorithm with simultaneous denoising. Entirely there are three regularizers in (12), and we allow one of them to be non-convex to promote the global sparsity more strongly. Moreover, the specific condition to assure the convexity of problem (12) is derived as the following proposition.
Proposition 1. Suppose the parameterized penalty function φ is defined by formula (6) and λ 0 > 0. If then the objective function P : (12) is strictly convex.
A proof of the above proposition is given in Appendix A.

Algorithm derivation
In this section, an algorithm is derived to solve (12) based on MM. The majorizer of R in (13) is and written explicitly as where C(z 1 , z 2 ) is a constant only dependent on z 1 and z 2 . In (16), r 0 is a function r 0 : Z × R N → R given by The majorizer of function Φ has been derived in Ref. [24,Section 3.3]. Here, it can be rewritten Repeat: Until convergence Return: x 1 , x 2 using a slightly different notation, that Φ M : where r : Z × R N → R is defined as Using the above results, the majorizer of the objective function P in (12) can be written as where the function R M : (13), wherein the derivation in detail is given in Appendix B.
Note that in (20), an extra term 1 2 (x 1 − z 1 ) − (x 2 − z 2 ) 2 2 is introduced, which does not break the property of majorizer, but helps to cancel the term x T 1 x 2 in the data fidelity term. Then P M (x 1 , x 2 , z 1 , z 2 ) can be written as where denotes element-wise multiplication, and the coefficients p 1 , p 2 , q 1 , q 2 ∈ R N can be written explicitly as Note that r 1 and r 2 are functions defined by (19) dependent on the binary weighting arrays b 1 and b 2 in (14) respectively.
Consequently, using MM, the problem (12) is solved iteratively by and the results of x 1 and x 2 in each iteration can be written separately as (24b) Table 2 gives the specific steps of the proposed algorithm RTEA. Note that, in this algorithm, the computation of r 0 , r 1 and r 2 can be implemented directly with convolution, and the rest steps are element-wised independent, where practically multi-threading computation can be directly adopted.
The systematic structure of the proposed RTEA for fault detection of rolling element bearings is presented in Figure 2. Since this algorithm is derived using MM procedure, the convergence can be guaranteed and converges to the optimal minimizer in this case. The detailed proof of MM for convex problems can be found in Ref. [28,Chapter 10]. 5 Synthetic data example where 1 ≤ J ≤ 10 is a random integer, and for each j, A j is a random amplitude, ω j is a random frequency, and θ j is a random phase. The sequences of transients are shown in Figure 3(a) and   shows the convergence behavior of the proposed algorithm.
As a comparison, considering another problem, based strictly on MCA, whose objective function is arg min Note that (26) has one regularization term less than (12), where the global sparsity is not promoted by function R.
Although non-convex penalties can help to promote sparsity, this will break the convexity of problem (26), so that a global optimal solution is not assured. Moreover, experiment results find that convex formulation (12) obtains as good a performance as the non-convex formulation (26).
As a consequence, the formulation (12) is preferred, because there is no need to sacrifice convexity.
In addition, the proposed method with non-convex formulation is also evaluated, where a 1   and/or a 2 are greater than zero. In this case, losing the convexity, the resulting sparsity can be even further promoted, then the formulation (12) is still preferred to the MCA formulation (26).
Further comparisons to some denoising methods are also presented. Firstly, wavelet-based denoising method is adopted to the test signal, more specifically, a 6-scale undecimated wavelet transform [15] using Haar wavelet filter. For denoising, hard-thresholding is applied and the threshold value is chosen by 3σ-rule for each subband. As shown in Figure 7 Secondly, the non-convex regularized fused lasso (FL) approach proposed in [6] is adopted, which is an improved version of conventional FL [41], allowing one regularizer to be non-convex  and preserving the global convexity. The result is shown in Figure 7(b). Thirdly, group-sparse signal denoising (GSSD) also known as OGS with non-convex regularization [13] is adopted. More specifically, 'atan' penalty with group size to be 3 samples is used, and the regularization parameter is chosen to optimize the RMSE. The result is shown in Figure 7(c). Both of the above methods also have worse recoveries in terms of RMSE. In addition, these methods are all denoising methods only, they cannot decompose the signal into two distinct sequences of transients, each exhibiting its own period. In Section 4.1 of [24], the schemes to set the binary weight array b (14) and the regularization parameter λ for POGS problem have been discussed in detail. Moreover, a look-up table has been given as a guide to choose the regularization parameter, where using the given multiplier in [24, Table 3], the regularization parameter can be chosen by λ = βσ w , where σ w is the deviation of the additive noise. Here, the table is quoted with a slight change of notation.

Parameter selection
As an extension of POGS with simultaneously decomposition, this parameter selection scheme can be used with a modification wherein the weights are shared among the three regularizers. In this case, the regularization parameters {λ 0 , λ 1 , λ 2 } are suggested to be set by where 0 < η < 1 is a parameter to balance sparsity of the sum (x 1 + x 2 ) and sparsity of the individual signal component x i .
To run the method, firstly the binary weight arrays b 1 , b 2 need to be set by (14) using the priori periods. Then β i can be determined by Table 3 using N i1 and M i for i = 1, 2. Note that it is necessary to chose K 0 = min{N 11 , N 21 } to induce group-sparsity. Hence in practice K 0 is not necessary to be chosen, and so as β 0 .
In practice, the parameters are set to be K 0 = N 11 = N 21 , and M 1 = M 2 = 4. The regularization parameters can be straight-forwardly determined by Table 3 and (27).
Setting parameter η. The parameter 0 < η < 1 in (27) is to balance the sparsity, as mentioned above. As special cases: 1. If η → 1, then λ 1 = λ 2 → 0, and the approach promotes sparsity of x 1 + x 2 , but leads to 2. If η → 0, then the problem (12) reduces to (26), which is a conventional MCA problem. It is able to seperate the components, but promotes sparsity weakly.
To obtain both of the benefits, through experiments, parameter η is suggested to be set around 0.5. 6 Engineering Examples  In this section, the proposed RTEA is applied to analyze vibration signals collected from a rolling element bearing with compound defects on a locomotive, using a SONY EX data acquisition system operating at a sampling rate of f s = 12.8 kHz. The locomotive bearing with faults in inner and out races is shown in Figure 9. The bearing (552732QT) parameters are given in Table 4.

Compound faults detection
In this example, the vibration signals were collected at a constant shaft speed of 360 r/min. Thus, based on the geometric parameters and rotational frequency, the ball-pass frequency of the outer race is about 43.3 Hz, and that of the inner race is about 58.7 Hz.
In practice, the regularization parameters in (12) can be estimated from the noise level, or the deviation of the vibration signal without any fault under an approximately identical experiment environment. Moreover, when the healthy data is not available, the 'noise' level can still be determined by the formulaσ = MAD(y)/0.6745 (28) which is a conventional estimator of noise level used for wavelet-based denoising [20], where MAD is the median absolute deviation defined as MAD(y) := median(|y n − median(y)|).
In this example, the formula (28) is used to estimate σ directly from the observation data, and then that value is used to choose the regularization parameters from Table 3. Figure 10 shows the results using RTEA, where x 1 is the transient sequence generated by the fault in the outer race, and x 2 is the transient sequence generated by the fault in the inner race.
During the recording, the outer race is stable and the inner race is rotating. Hence, the amplitude of transients in x 2 (inner race fault) exhibit modulating effect.
To further reveal the characteristic frequencies, the Hilbert envelope spectrums of the extracted components are shown in Figure 11 envelope spectrum is also presented to indicate the characteristic frequencies more clearly. Figure 11 shows that the characteristic frequencies of outer and inner race are at about 45 and 60 Hz, and their harmonics are revealed by the peaks of the smoothed profiles. Thus, the fault features of the two defects are clearly extracted by the proposed approach.
As a comparison, we include the Welch's estimate of the cyclic spectral coherence for the analysis of cyclostationary signals [2,3,4]. The result is illustrated in Figure 12. Note that, the signal length is 6400, the window length is 32, the signal is divided into 14 overlapping blocks and the block overlap is 21 with a Hanning window, which is suggested in Ref. [2]. When observing the coherence, the cyclic frequency of the outer race (approximate 43.3 Hz) and its harmonic components can be identified. However, the characteristic frequencies of inner race 58.7 Hz and its harmonic components cannot be observed from Figure 12. This may be caused by the fact that the component of inner race in this case is weak.

Example 3: Single fault detection
In this example, the proposed method also works when the bearing has only one fault. Figure 13 shows the measured data and the results from the bearing in Figure 9, but there was only one defect on the outer race. The acceleration signals were collected at a constant shaft speed of 481 r/min. transients. This implies that the status of the inner race is healthy.

Conclusion
This paper proposes a novel approach for the extraction of repetitive transients with group-sparse structure in vibration signals, and for detecting faults in rolling element bearings. To simultaneously extract both sparse components and perform denoising, a repetitive group-sparsity based optimization problem is formulated. To solve the problem, a computationally efficient iterative algorithm, termed repetitive transients extraction algorithm(RTEA) is derived. The non-convex penalty function is used to strongly promote sparsity, and a condition is given so that the objective function is strictly convex. Moreover, for practical problems, an approach to select regularization and non-convexity parameters is provided.

Appendices A Proof of Proposition 1
Proof. The first two terms of the objective function in (12) can be rewritten as F (u) is strictly convex. Moreover, it is immediate that when a = 0, function (11) is convex. As a consequence, an equivalent problem of (12) is considered, namely which satisfies the convexity condition of equality constrained problem in [11], and which implies that: when (12) satisfies (15), it is a convex problem.