{'fraction_non_alphanumeric': 0.03400014655235583, 'fraction_numerical': 0.014655235582911996, 'mean_word_length': 4.720033528918693, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Hybrid methods of Brandt’s generalised likelihood ratio and short-term energy for Malay word speech segmentation

Received Jan 21, 2019 Revised Apr 20, 2019 Accepted May 5, 2019 Speech segmentation is an important part for speech recognition, synthesizing and coding. Statistical based approach detects segmentation points via computing spectral distortion of the signal without prior knowledge of the acoustic information proved to be able to give good match, less omission but lot of insertion. In this study the segmentation is done both manually and automatically using Malay words in traditional Malay poetry. This study proposed a hybrid method of Brandt’s generalized likelihood ratio (GLR) and short-term energy algorithm. The Brandt’s algorithm tries to estimate the abrupt change in energy to determine the segmentation points. A total of five Pantun are used in read mode and spoken by one male student in a noise free room. Experiments are conducted to see the the accuracy, insertion, and omission of the segmentation points. Experimental results show on average 80% accuracy with 0.2 second time tolerance for automatic segmentation with the algorithm having no knowledge of the acoustic characteristics.

Speech segmentation can be defined as the process of finding the limits (with specific characteristic) in natural spoken language between words, syllables or phonemes. [4,5]. The main objective of Speech segmentation is to serve other speech analysis problems such as speech synthesis, data training for speech recognizers, or to fabricate and label prosodic databases. Therefore, it can be viewed as a vital sub-issue for various fields in speech analysis and research. [6,7]. The traditional approach handling this issue is by manual segmentation of speech, which is generally performed by specialized phoneticians. However, this method is based on listening and visual judgment on required boundaries which makes it inconsistent and time consuming. [8,9]. Another method which is considered very convenient is an automatic segmentation. The speech can be automatically segmented into sub word units which are defined acoustically. [10] In Automatic Speech Recognition ASR systems, segmentation can be performed: (i) At the system training stage, when segmentation is applied to the training set recordings. (ii) At the recognition stage [5].

SEGMENTATION TECHNIQUES
Several well-established segmentation techniques have been proposed by previous researchers, such as in [10], that audio segmentation is performed using segment features. The proposed technique uses a loglinear segment model to determine the segmentation of the input audio stream [11]. First, the audio data is processed with a speaker independent acoustic model [12]. The decoding process will hypothesis the locations of sentence start and end. The resulting segments are also clustered and used in Constrained maximum likelihood linear regression (CMLLR) feature transformations and maximum likelihood linear regression (MLLR) mean transformations. The experimental results in [10] shows that the framework is applicable for various segments, boundary features, and for different change point detection methods.
The Hidden Markov Model (HMM) is one of the highly-used segmentation techniques. A refined HMM algorithm was tested for segmenting a Chinese corpus [11]. The method is carried out in 3 steps: 1. Obtain initial segmentation marks using HMM with forced alignment. 2. Create a super vector for each boundary of this database by placing acoustic vector near the boundary. The pseudo-triphone formed from the boundary are classified using a classification and regression tree (CART) where the pseudo-triphone are clustered into smaller number of classes. Then each leaf node on the CART is used to train a Gaussian Mixture Model (GMM). 3. For each labelled sentence, attempt to refine the boundary of each segment. Using the HMM boundary obtained above, compute the likelihood of this frame contains the actual boundary. The optimal boundary is assumed to be the frame that has maximum likelihood of the GMM model associated with the CART leaf node for the pseudo-triphone. Experimental results in [5] shows that the refined HMM is more accurate than the standard HMM segmentation.
The Brandt's generalized likelihood ratio (GLR) method aims to detect discontinuities in homogenous segment of the speech signal models using statistics to detect sequentially abrupt changes in the parameter of the model [11][12][13][14]. The signal Yn, is decribed using an autoregressive model M, such that where en is a zero-mean noise with variance σ 2 . Assume the audio signal is windowed as in Figure 1. Dn(r) = -(n -r)Logσ2 2 -rLogσ1 2 + NLogσ0 2 (2) and D0 is the predefined threshold. This research will focus on the use of Brandt's GLR algorithm on segmenting Malay words. The Malay language is one of many Austronesian languages which also includes languages such as Pilipino and Tagalog [15]. It is officially used in Malaysia and Singapore. Some aspects of the language are influenced by English and some words are directly borrowed from English itself. ASR system for Malay is challenging due to various regional dialects used and the occasional use of English words in sentences. Structures for the Malay syllables exits in the form of V, CV, CVC, CCV, CCVC, and CCCVC [16]. C's are consonant and V's are vowels. Syllable borders have two significantly different energy clusters which are visually noticeable however when pronounced continuously and closely together, false abrupt changes might occur depending on the speaker's utterance style [17]. For read mode, the patterns are quite obvious and easily detected as only minimal amount of noise is present but in spontaneous mode, the presence of background noise, talking pace, and interference from other speaker may cause difficulty during segmentation [18].

METHODOLOGY
The framework of this research is shown in Figure 2. Speech signal data in the form of Malay Poems or known as Pantun is used to test the Brandt's GLR. The Pantun represents read mode audio data where audio recording is done in controlled environments and the speaker controls his/her manner of speech to ensure clear pronunciation and fixed reading pace [19,20]. The Poems (Pantuns) are framed per sentence. No windowing method is used due to the Brandt's GLR only working on detecting energy difference in time domain. Further processes are explained in detail in the following sections.

Data collection
As mentioned earlier, the data used during testing are in the form of traditional Malay Poems (Pantun) read by a male speaker in a noise free room. Instances of modern poems, as well as other types of poems, can be found in all sorts of printed and electronic documents including books, newspapers, magazines, and websites [21]. An instance of modern Malay poems can represent a complete poem or a poem portion. Five (5) poems (pantuns) of 10 sentences long each are first manually segmented. Segmentation both manually and automatically is done on frames consisting of one sentence long cut from the Pantuns. Manual segmentation is performed using the wavesurfer program as shown in Figure 3.
Segmentations are done on word by word by word basis visually by observing the waveform while listening to the audio as well. Referring to Figure 3, the energy difference between words are visually obvious since the data is in read mode and there are clear silences in between words.  Figure 3. Manual segmentation using on the sentence "baik-baik jaga pedoman" using the wavesurfer program. Words segmented are "baik", "baik", "jaga", "pedoman"

Brandt's GLR algorithm
Noise filtering is not required since little to no noise is present in the data. The data is first framed into individual frames of one (1) sentence each [22][23][24]. Then the Brandt's GLR ratio is calculated for each sample as follow, using the windows in Figure 4. 1. The covariance of W1 and W2 are calculated using brute force calculations. W1 will start at r = 1 and will grow until n and the variance values of values for both W1 and W2 are calculated. 2. The variation of Dn is then calculated using equation (1) and graph Dn is plot. The segmentation points are then acquired by observing the highest ratio calculated by the Brandt's GLR. Resulting segmentation points from the Brandt's GLR method are compared to the reference segmented points from the manual segmentation that was done using the wavesurfer program by us. The measurement criterion is adapted from [25]. Let K = {K1, K2,,……,An} and R = {R1, R2,…….,Rn} be the segmented points in seconds obtained from the Brandt's GLR and manual segmentation respectively. For each Kj, the corresponding point Rkj is determined by the time instance closes to that of Kj. Thus a sequence Rk = {Rk1, Rk2,….., Rkn} is build to compare both segmentations.
Omission can be detected as when point in Rk is not in Kj and insertion when points in Kj is not in Rk. Number of similar points in both Rk and Kj are calculated as Match, m = (m/p * 100) where p is the number of points in R [26,27]. Accuracy is calculated using, accuracy = ((m/p+n) * 100) which is affected by the number of insertions. The Brandt's GLR method will be evaluated in terms of the number of omission and insertions, matches, and accuracy [28].

Short-term energy algorithm
The energy parameter has been used in speech segmentation since the 1970's [29]. This algorithm was adopted and modified to better locate the beginning and ending of speech points for the isolated spoken Malay utterances and will be discussed in detail. This is a two-step search algorithm where the absolute energy (AE) for a coarse search is first used [19]. The speech signal was first divided into 50% overlapping frames of 10ms and then passed through a rectangular window [30,31]. The AE was computed by summing the absolute magnitudes of speech samples in each frame as shown in (3).
where, w(m) rectangular window, N length frame duration ending at n=m and m peech samples overlapping at 10 ms.
The mean and standard deviation of the AE measure is first computed during the first 50ms of the speech, assuming there is only background noise in that interval [22]. This information was further used to compute the peak energy (IMX) for the entire interval in each speech sample and the silence energy (IMN) [23,24]. Subsequently the IMX and IMN were used to set two energy thresholds: upper threshold (Tu) and lower threshold (Tl) according to (4).
The upper threshold (Tu) will be computed as in (5) and (6).
. 0 (6) where, WL word length, I is index of all frames, having E(i)>Tl Therefore, upper level for average energy is set to 0.25 based on experimental findings in case of high noise [25].

RESULTS AND ANALYSIS
In the experiments, the Brand's GLR is applied on periodic frames of 0.4 seconds. The algorithm was tested on frames of 0.2 seconds but was find to create high amount of insertion thus lowering the accuracy of the segmentation. At 0.2 seconds, Brandt's GLR produces twice the amount of segmentation points compared to the reference segmentation points as shown in Figure 5.
Each pantun is read in a controlled rhythm where each of the words in each sentence is read approximately 0.5 seconds apart from each other. Therefore 0.4 second framing is relatively effective for this type of segmentation. Pantun five (5) shows the worst accuracy as a lot of the words are made of prefixes such as dihati and membujang. Insertion occurs in between the prefix and the word hence lowering the accuracy of it. All of the other pantuns manage to be segmented with 80% accuracy with a 0.2 second tolerance.
Nine out of 20 of the data managed to be 100% segmented and overall result is presented in Table 1. And the 5 th sentence from pantun two (2) achieved 100% segmentation within 0.1 second time tolerance. In that sentence, "suka hati kumbang yang terbang", none of the words contains prefixes and suffixes, and contains no more than two syllable per words. Suffixes and prefixes can sometimes be captured as new words. For example, in the 3 rd sentence of pantun five (5), the prefix "membu" in "membujang" was captured as a separate word. 0.4 second frames are chosen as it manages to segment words that are two syllables long without over segmenting. This however will cause over-segmentation in words that are three syllables or more which is commonly due to the presence of prefixes or suffixes.   Figure 6 shows the reference segmentation of the word "membujang" and Figure 7 shows how the algorithm did it. The word "membujang" was read 0.6 seconds long, which was captured by two separate GLR frames thus causing over segmentation.  Figure 6. Waveform for the sentence "sudah lama hidup membujang", the red line shows the start of the word "membujang" and the blue line marks the end of the prefix "mem" Figure 7. Automatic segmentation of the sentence "sudah lama hidup membujang". Red line shows segmentation points

CONCLUSION
Four out of five of the pantun managed to be segmented with 80% accuracy. However, it is to be noted that all the data is in read mode and was recited in a controlled rhyme thus making the segmentation process a lot simpler than if to be done on spontaneous speech where there will be multiple speakers which all speak at different pace. Salam recommended in [7] to use higher order of autoregressive model to purposely cause over segmentation and to remove the insertions using Neural Network. This might also help with segmentation of spontaneous data. To test this would be our future goal.