{'fraction_non_alphanumeric': 0.02519007053219749, 'fraction_numerical': 0.005587615645323807, 'mean_word_length': 4.83979673709548, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
AhoTransf: A Tool for Multiband Excitation Based Speech Analysis and Modification

In this paper we present AhoTransf, a tool that enables analysis, visualization, modification and synthesis of speech. AhoTransf integrates a speech signal analysis model with a graphical user interface to allow visualization and modification of the parameters of the model. The synthesis capability allows hearing the modified signal thus providing a quick way to understand the perceptual effect of the changes in the parameters of the model. The speech analysis/synthesis algorithm is based in the Multiband Excitation technique, but uses a novel phase information representation the Relative Phase Shift (RPSs). With this representation, not only the amplitudes but also the phases of the harmonic components of the speech signal reveal their structured patterns in the visualization tool. AhoTransf is modularly conceived so that it can be used with different harmonic speech models.


Introduction
Speech models based in the separation of the periodic and the noise-like parts of the speech were early introduced in the speech processing panorama. The early work by McAulay and Quatiery (1986) with the sinusoidal modelling, where the signal was modelled by means of sinusoidal components located at the frequencies where the peaks of the spectrum were, was quickly followed by the harmonic systems (Griffin & Lim, 1988;Laroche, Stylianou, & Moulines, 1993;Stylianou, 1996). This harmonic constraint is appropriate for the speech signal and simplified the analysis and the synthesis, eliminating the need of peak picking and peak tracking algorithms. However, modelling only the harmonic part of the signal leaves out quite a lot of information, so harmonic models were complemented with a noise-like component. This noisy component has been defined in different ways: some proposals (Laroche, Stylianou & Moulines 1993;Stylianou, 1996) assume that the noise is above a certain frequency (harmonic plus noise family, HNM); others overlap the harmonic and the noise-like parts along part or all of the spectrum (Stylianou, 1996;Erro, Moreno & Bonafonte, 2007) (harmonic plus stochastic family); and finally others interleave periodic and noisy components in harmonic bands, (Griffin & Lim, 1988;Dutoit & Leich, 1993) (multiband excitation family, MBE). The model implemented in the tool described in this paper falls into this last category. When these models were first proposed (late eighties and early nineties) they meant an important leap towards voice quality, because they allowed high quality coding and thus good synthetic voice quality. Being fully parametric, they solved the problem of concatenation mismatches and allowed easy pitch and duration modifications of the signals. They also permitted low bit rate high quality coding. The main downside was their complexity and the heavy computational requirements of the analysis stage. The arrival of the unit selection techniques for synthesis, which produced higher naturalness and required comparably less computational effort, slowed down the development of these methods. Nevertheless, more recently HNM models have gained more and more interest, as more and more research effort is being oriented towards the area of voice transformation and voice conversion. Sure enough, the parametric nature of these models allows not only pitch and duration transformations but also spectral manipulation, and it has been reported that strong modifications can be done to the signal while keeping a certain degree of naturalness (Stylianou, 1996). Our interest in this area derives also from its application to voice transformation in general, and we have developed several HNM models, seeking the higher possible level of naturalness for speech. We have built up a Harmonic plus Noise model based on the Multiband Excitation techniques, but with specific phase control techniques (Saratxaga et al., 2009) developed by us. The resulting system appears suitable for voice transformation: it is robust, it is pitch asynchronous, it has good quality, it is fully parametric, and the parameters are quite straightforward, so as they can be easily manipulated. To gain a better understanding of the relationship between the parameters of such a model and the perceptual characteristics of the speech we have developed the AhoTransf tool. This tool shows the different parameters of the model in spectrogram-like displays and allows modifying any of these parameters. It integrates a re-synthesis algorithm so the user can hear the effect of the modifications. In the next section, the model is outlined in three parts: one describing the analysis stage, another the synthesis one and the last one explaining pitch and duration modifications. Then, the functionality of AhoTransf is described in detail and finally, a conclusion section closes the paper.

HNM-MBE model
The proposed harmonic plus noise multiband excitation model (HNM-MBE) is based in the vocoder developed by Griffin and Lim (1988), with several modifications related to the analysis and representation of the phase of the harmonics. In this model the speech signal is decomposed into two components, a harmonic one h(t) and a noisy one n(t): The MBE model considers that the whole spectrum is divided into equally wide bands centred around the pitch harmonic frequencies and each of these bands is classified as harmonic or noisy, depending on the Power Spectral Distribution (PSD) of the signal within the band. In this way, we get two components, harmonic and noisy, each of them having energy in different but interspersed frequency bands. The modelled signal can be expressed by: Where k denotes the band, K(t) is the total number of bands at time t (which depends on the pitch value at that moment) and h k (t) and n k (t) stand for the harmonic and noisy models of the k-th band. <A|B> operator (A or B) implies a selection between the two arguments. The harmonic bands are modelled by means of a sinusoid at the harmonic frequency, while noisy bands are modelled by a band-pass white noise. The harmonic part can thus be written as: where N is the number of bands, the A k are the amplitudes of the spectral envelope, φ k is the instantaneous phase, f o is the pitch or fundamental frequency and θ k is the initial phase of the sinusoid. The noise-like part can be better defined in the frequency domain, where its banded structure is clearly exposed: where B k are the amplitudes of the noise spectral envelope in each band, BW is the bandwidth of a band and W(ω) is the Fourier transform of a sufficiently long white noise signal fragment.

HNM-MBE analysis
The analysis starts with the calculation of the fundamental frequency. A cepstrum-based pitch determination algorithm (CDP) is used for that purpose (Luengo et al., 2007). The analysis is pitch asynchronous so the frame rate can be freely chosen (8-10ms). The speech signal is windowed by means of a Hann window. The window is three pitch periods long, so as to assure a good resolution in the frequency domain where the analysis will be done. The MBE model assumes that the spectrum of the speech signal is divided into bands centred on the pitch and its harmonics. The power spectrum is represented by an envelope with one value per band, and two of these envelopes are calculated for every analysis frame: one using the harmonic model and the other using the noise model.

Spectral envelopes calculation
The values of the amplitudes in every band are calculated by minimizing the energy of the modelling error of the windowed frame (Griffin & Lim, 1988): where S w is the windowed frame of the signal, and Ŝ w the corresponding modelled synthetic signal. This error is minimized when the coefficients are: where a k and b k are the lower and upper limits of each frequency band, and E w is the Fourier transform of the windowed synthetic excitation signal: sum of harmonic sinusoids in the case of the harmonic model, and normalized white noise in the case of the noise model. For the harmonic model, the Fourier Transform of a synthetic windowed excitation signal E w (ω) is obtained for each frame.
where w han (t) is the aforementioned Hann window. The Fourier transform of the signal frame, S w (ω), is also computed and the coefficients are calculated for every band. It is worth noting that the coefficients A k are real numbers. No complex calculation is done in this analysis. The phase of the sinusoidal components will be obtained otherwise, as it is explained in the next section. For the noise model, the expression used to calculate the envelopes is the same as (6), but the synthetic excitation signal is much simpler: the Fourier transform of the windowed normalized white Gaussian noise equals one across the bands. Therefore, expression (6) becomes:

Phase calculation
Unlike the traditional MBE model, where the instantaneous phases of the harmonic components are obtained resolving a complex version of equation (6), in our model these phases are extracted from the spectrum of the signal. Moreover, in our model we do not use the instantaneous phases but instead the Relative Phase Shifts (RPS's) are used (Saratxaga et al., 2009). The RPS's are the difference between the initial phase shift of every harmonic sinusoid with respect to the first harmonic (F0). They can be calculated from the instantaneous phase of the harmonics using the expression: where θ k is the RPS, φ k the instantaneous phase of the kth harmonic, φ 1 the instantaneous phase of the fundamental frequency harmonic and t a the instant chosen for the analysis. The result of this formula is wrapped to values inside the [-π, π] interval. The RPS's exhibit some desirable properties for the phase representation. The differences of the initial phase shifts of the sinusoidal components determine the actual waveform shape of the signal. Therefore, the RPS's are constant while the waveform shape keeps stable. Furthermore, the RPS's reveal a structured pattern in the phase information of the voiced segments, which is not clear at all in the instantaneous phase representation as it is depicted in fig. 1. Fig. 1 shows the different phase information for a voiced signal containing five vowels /aeiou/ ( fig. 1.c). Fig. 1.a shows the evolution of the usual instantaneous phase both in frequency (vertical axis) and in time (horizontal one), where no structure can be appreciated. Fig 1.b shows the evolution of the RPS's representation for the same signal, where the subjacent phase structure of every vowel is exposed. As mentioned before, the instantaneous phases are taken from the phases of the windowed signal spectrum at the harmonic frequencies. The spectrum is calculated for every frame by means of an FFT. Afterwards, the instantaneous phases at the frequencies of every harmonic are taken and their phase difference with respect to F0 is computed applying expression (9). For the F0 itself, its instantaneous phase is kept in order to allow a synchronous reconstruction of the original signal.

2.1.3.
Voiced/unvoiced band decision Till this point, we have two independent and complete models of the signal spectrum, one harmonic and the other noise-like. The final stage of the analysis involves deciding whether each band should be represented by the harmonic or by the noise component. The band modelling error is used as input for the decision. As stated in (Griffin & Lim, 1988) the error expression (5) is biased towards the longer periods, for the longer the period is, the more densely the spectrum is sampled, consequently reducing the value of the error. An unbiased expression of the error, proposed in the same paper, is used: where P is the period of the pitch and w[n] are the samples of the window.

Figure 1. Instantaneous phase vs. RPS phasegrams
This expression gives a normalized error independent of the pitch and of the actual energy of the frame. Expression (10) is calculated using both harmonic and noise models for Ŝ w and the band is classified as voiced or unvoiced by comparing the errors produced by each model. A weight can also be used to bias the decision towards one or the other model. In our implementation, the voiced decision (i.e. harmonic component) has been favoured, because it produces perceptually clearer resynthesis.

HNM-MBE synthesis
The synthesis from the data obtained in the MBE analysis is carried out in two independent processes for the harmonic and the noise components. Both of them are added at the end of the frame generation process.

Synthesis of the harmonic component
The synthesis of the harmonic part requires the pitch, the harmonic coefficients, the V/UV band decision, the phase differences and the instantaneous phase of F0 to be accomplished. Each frame is synthesized taking into account the initial parameters (i) and the final ones, which correspond to the next analysis frame (i+1) to ensure continuity. Between these parameters, linear interpolation is used to obtain the amplitudes, RPS's and frequencies for every sample. When a band is voiced (i.e. modelled by a harmonic sinusoid) at the beginning of the frame and becomes unvoiced at the end, or vice versa, the final (or initial) amplitude is set to zero so that the harmonic component fades (or appears) smoothly. As the final parameters will become the initial ones of the next synthesis frame, continuity is ensured. The expression of the harmonic component for frame i is:  [n]). The procedure is thoroughly explained in (Saratxaga et al., 2009).

Synthesis of the noise component
The noise component is synthesized by means of a FFT filter. A synthetic spectrum for the white noise is generated first, long enough to minimize the windowing distortion. This length is variable and depends on the required frequency resolution (that is to say, the number of bands) and on the length of the signal to be generated. Interframe discontinuities are seldom perceptible in noisy signals. Thus, a simple average is done between the noise coefficients of the initial and final analysis frames and they are kept constant within the frame. In an analogous but inverse way to the harmonic part, if a band starts as unvoiced but ends as voiced (or on the contrary, becomes unvoiced) the corresponding unvoiced coefficient is set to zero. The rules above are applied for each band, and a spectral envelope is obtained. Then, it is applied to the synthetic noise spectrum and the inverse Fourier transform is calculated.
where E(ω) is the envelope and W(ω) the synthetic spectrum. The last step of the synthesis process is the addition of the harmonic and noise signals to get the complete frame, which is concatenated to the previously generated output signal.

Pitch and duration modifications
Changing most of the parameters of the model (amplitudes, phases or banding decisions) has an immediate impact on the signal spectrum. On the contrary changing the pitch or the duration of the signal should ideally leave the spectrum unaffected, but imply a different kind of parameter modification. Duration changes using RPS's are immediate. They just imply changing the number of synthesis samples per analysis frame according to the length modification factor, while the rest of the parameters remain unaffected. By the contrary, pitch changes have deeper effects, because modifying the pitch implies modifying the frequencies of all the harmonic components and thus the number of parameters. The problem is how to estimate the values of the parameters at the new frequencies of the harmonics departing from the original ones. The usual solution (Quatieri & McAulay, 1992) consists in considering the original parameters as points of a frequency envelope which is resampled at the new frequencies to obtain the new set of parameters. AhoTransf uses linear interpolation to obtain the new parameters and employs this technique both for amplitudes and for the RPS's.

AhoTransf
AhoTransf is a modular tool designed to visualize and modify the parameters of harmonic speech models. It also integrates speech analysis and resynthesis along with the GUI, thus allowing a straightforward manipulation of the speech signal. The tool has been developed using the HNM-MBE model, but other harmonic models can be integrated with little effort. The application is developed in Matlab and is organized around three core modules: the director module, the displaying module and the editing module. Around them the HNM-MBE analysis, synthesis and modification algorithms' implementations are used to get and process the data. This modular structure allows using the tools core modules not only with HNM-MBE parameters but also different harmonic models with minimal modifications. A diagram of the structure of the tool is shown in fig. 2.

Figure 2. Modular structure of AhoTransf
The director module captures the user commands and manages the invocation of the rest of the components and functions in order to fulfil them. We will now describe the functionalities of the display and editing modules as they gather the main functionalities of the tool.

Display Module
The display module is responsible of formatting the parameters of the model so that they can be easily interpreted by the user. The display module has a parametric and modular structure that allows an effortless reconfiguration to adapt it to other kind of speech models.

Figure 3. Visualization window
For the HNM-MBE model, the visualization window shows four panels. Three of the panels show representations of the amplitudes and phases of the harmonic part of the model, and the frame-by-frame voicing decision. The fourth one shows the signal waveform. In order to keep the window as simple as possible, the amplitudes of the noise part are not shown because they look very much like the amplitudes of the harmonic part since they model the same PSD.
The parameters of the model are displayed in spectrogram-like graphics, with time in the horizontal axis and frequency in the vertical one. This representation is not directly obtained from the parameters of the model. In fact, for every analysis frame the number of harmonic parameters is different as they are function of the pitch at the time of the analysis. So the parameters have to be scaled in frequency in order to get a meaningful representation.
The display module provides several visualization facilities such as time axis scrolling, selection and zooming (either individually by panel or combined for all the panels).
Regarding the synthesis possibilities, the user can choose to hear the whole of the signal or parts of it. He or she can also hear the original and the resynthesized signals. In this last case, the user can choose to hear separately the signals corresponding to the harmonic or the noise parts of the model.

Edition Module
The modification of the parameters to obtain different voice perceptual qualities is a complicated task, as they require non-uniform coordinated modification of whole groups of parameters. The edition module of AhoTransf allows simple but detailed modification of the amplitudes, phases, voiced/unvoiced decisions per band, pitch and overall duration of the signal. The edition window shows four panels with the harmonic amplitudes, phases, voiced/unvoiced decisions and the pitch of the signal. Modifications can be applied either to the whole signal or to a selected segment. For bidimensional parameters (i.e. those dependent on time and frequency) it is possible to limit the modification to a certain segment and certain frequencies. The selection of the parameters to be changed is easy, as it is done using the mouse. Zooming and hearing tools are available to help with the selection of the desired fragment. The editing possibilities are different depending on the parameter.
• Amplitudes (A k , B k ): Changes in this panel are applied to the amplitudes of both the harmonic and noise models. The amplitudes can be set to a certain value and can also be scaled by a frequency dependent factor thus allowing modifying the tilt of the spectrum. • Phases: They can be adjusted to a frequency dependent mathematical expression to test the perceptual influence of different phase structures. They can also be set to random values. • Voiced/Unvoiced decisions can be set per band. This feature allows producing pure harmonic or noisy versions of the original signal, and also studying the actual contribution of each component to the voice quality (harmonic-to-noise ratio, breathiness, maximum voicing frequency). • Pitch can be scaled, interpolated between two certain values or set to a certain value, thus allowing prosodic modifications. • Signal duration can be scaled by a factor.
Changes in the parameters are immediately visualized at the corresponding panel and finally the signal can be resynthesized using the modified data.

Conclusions
We have developed a graphic application for the visualization and edition of the parameters of our HNM-MBE model. AhoTransf is a modular application that can be customized to manage different sets of parameters, so it will be expanded to work with other models of the harmonic family. This application will be used for research purposes to test the perceptual effects of the changes in different parameters, as the GUI provides a quick and effortless way to check them. It will also be used for educational purposes to help explaining the harmonic models. Future work could be done to include different speech coding algorithms in order to compare their parameters and resynthesis quality.

Acknowledgements
The work presented in this paper has been partially funded by the Spanish Government under grant TEC2009-14094-C04-02 (BUCEADOR project) and by the Basque Government under grant IE09-262 (BERBATEK project).