{'fraction_non_alphanumeric': 0.021276176592265716, 'fraction_numerical': 0.013337010697188787, 'mean_word_length': 4.892280905397563, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Non-invasive classification of gas–liquid two-phase horizontal flow regimes using an ultrasonic Doppler sensor and a neural network

The identification of flow pattern is a key issue in multiphase flow which is encountered in the petrochemical industry. It is difficult to identify the gas–liquid flow regimes objectively with the gas–liquid two-phase flow. This paper presents the feasibility of a clamp-on instrument for an objective flow regime classification of two-phase flow using an ultrasonic Doppler sensor and an artificial neural network, which records and processes the ultrasonic signals reflected from the two-phase flow. Experimental data is obtained on a horizontal test rig with a total pipe length of 21 m and 5.08 cm internal diameter carrying air-water two-phase flow under slug, elongated bubble, stratified-wavy and, stratified flow regimes. Multilayer perceptron neural networks (MLPNNs) are used to develop the classification model. The classifier requires features as an input which is representative of the signals. Ultrasound signal features are extracted by applying both power spectral density (PSD) and discrete wavelet transform (DWT) methods to the flow signals. A classification scheme of ‘1-of-C coding method for classification’ was adopted to classify features extracted into one of four flow regime categories. To improve the performance of the flow regime classifier network, a second level neural network was incorporated by using the output of a first level networks feature as an input feature. The addition of the two network models provided a combined neural network model which has achieved a higher accuracy than single neural network models. Classification accuracies are evaluated in the form of both the PSD and DWT features. The success rates of the two models are: (1) using PSD features, the classifier missed 3 datasets out of 24 test datasets of the classification and scored 87.5% accuracy; (2) with the DWT features, the network misclassified only one data point and it was able to classify the flow patterns up to 95.8% accuracy. This approach has demonstrated the success of a clamp-on ultrasound sensor for flow regime classification that would be possible in industry practice. It is considerably more promising than other techniques as it uses a non-invasive and non-radioactive sensor.


Introduction
Multiphase flows are found in many industrial processes such as petroleum production, power generation, thermal engineering and nuclear reactors. The characteristics used to describe single phase flow, such as turbulence, velocity profile and the boundary layer, are not suitable for describing the nature of multiphase flows (Corneliussen et al 2005). Multiphase flows are cateogrised into flow regimes. These flows occur both in horizontal and vertical orientations. The flow regimes are developed based on the flow-line geometry and orientation, individual phase flow rates, and component transport properties (density, viscosity and surface tension (Rajan et al 1993, Thorn et al 2013. Identification of the flow regimes in multiphase flow is essential to both efficient operation of the multiphase flow systems and the determination of phase fractions (Arvoh et al 2012). To group flow regimes according to their topological similarities, several mech anisms of the flow regime classifiers or flow regime descriptors have been developed over the years. Typical flow regimes in the horizontal pipe flow are: slug, stratified, wavy, elongated bubble and annular flow patterns, and flow vertical gas-liquid flow regimes are: the bubbly, slug, churn and annular flow (Falcone et al 2009). The process of objective flow regime identification from the sensor signals of the flow requires the use of a pattern recognition technique.
The application of pressure fluctuations of the two-phase flow signals and statistical analyses for objective characterization was pioneered by Drahos and Cermák (1989). The two-phase flow signals from several pressure transducers have been analyzed for features extraction using power spectral density (PSD) for generating an input variable for the neural network (Xie et al 2004, Sun and Zhang 2008, Kv and Roy 2012. Other sensor signals have been used for flow regime classification using the statistical moment of the analysis, such as conductance probe (Hernández et al 2006), and radioactive images (Sunde et al 2005). It has been found that the pattern recognition of flow regimes using pressure signals is fast enough to be used for online flow regime identification (Xie et al 2004, Kv andRoy 2012). However, these transducers are invasive sensors. Hence there is a need for a noninvasive method of flow regime classification for two-phase flow such as ultrasound or gamma. In addition, the review of the methods of objective flow regime classification showed that the early methods used mechanistic models or empirical models. The flow patterns of the multiphase flow were identified using equations governing the physics of the fluid developed from the mechanistic models derived from the physics of the fluid. The process of identifying flow patterns using these models has the disadvantage that each flow regime has to be examined independently (Ozbayoglu and Ozbayoglu 2009).
The ultrasonic technique based on the principle of pulseecho intensity is widely used in gas/liquid interface detection and often, the location of the interface is determined by measuring the time of flight of the reflected wave (Chang and Morala 1990). Wada et al (2006) presented an ultrasonic method of two-phase flow pattern recognition based on the measurement of the instantaneous echo intensity profile along the ultrasonic beam. They comment that the echo intensity of the flow is measured by the integral of the difference in the energy of single phase flow and two-phase flow over the pipe diameter. The flow patterns from single phase flow to annular flow are identified by the statistical distribution of the echo intensity. Murai et al (2010) developed a pulse-echo ultrasonic technique to determine instantaneous liquid-gas interface detection. Chakraborty et al (2009) introduced a new ultrasonic method for measuring the void fraction of two-phase flow using an ultrasonic sensor and twin signal processing methods based on the time series analysis technique: symbolic dynamic filtering and analytical signal space partitioning for void fraction measurements and identification of flow regimes. An algorithm was built on the method of symbolic dynamic filtering to analyze the ultrasonic pulse echoes reflected off the bubbles. The pulse-echo technique of flow pattern identification is not a complete flow regime classification technique but an identification of the flow pattern (Jha et al 2013). However, the authors comment that more research work on the computational and experimental work is required before the method can be deployed for industrial use. Another issue is that, although the ultrasound method applied is not intrusive, the set up is invasive. Jha et al (2013) extended the work of Chakraborty et al (2009) and proposed that the concept of ultrasonic pulse echo should be implemented in a clamp-on set up together with symbolic dynamic filtering for industrial application.
Despite the feasibilities of using the pulse-echo ultrasound for flow regime identification, the flow regime identification is based on computational models. The computational methods for flow regime identification employ sets of non-linear equations, but often the equations are simplified. For practical applications, the simplified equations are not often used because it requires prior knowledge of several flow properties, such as pipe diameter and pipe thickness, which degrade in the cause of time (Meribout et al 2010). The pulse-echo ultrasound technique is limited in liquid flow velocity information due to the restriction on the maximum measurable velocity using pulse wave ultrasound by the Nyquist criterion (Evans and McDicken 2000).
Ultrasound Doppler flow sensors, which use a continuous wave of ultrasound signals, also have a great potential to achieve non-invasive flow velocity measurement. Techniques using continuous wave ultrasound already exist in the medical ultrasound system. The techniques use frequency shift representing the flow velocities to develop methods to predict multiphase flow regimes (Übeyli and Güler 2005). In multiphase flow measurement, Kouame et al (2003) present an application of continuous wave Doppler ultrasound (CWDU) velocity measurement to two-phase flow in pipes. They proposed the use of frequency resolution techniques to overcome the hindrance to the velocity profile measurement by the presence of coloured noise, which introduces a significant obstacle to the classical frequency estimators. Pulse echo ultrasound techniques for two-phase flow measurement have limited liquid velocity information due to the restriction on the maximum measurable velocity using pulse wave ultrasound by the Nyquist criterion (Evans and McDicken 2000). Also, characteristics of the reflected wave are being influenced by the shape and size of the interface about the ultrasound wave length (Murai et al 2010).
Artificial neural networks (ANNs) are often preferred over statistical methods of pattern recognition because of their fast responses and simplification (Mi et al 2001). Also, the ANNs have good performance on pattern recognition due to their efficiency and available learning algorithms (Jain et al 2000). Also, with regards to flow regime classification, ANN has advantages over other analytical tools such as the expert system and clustering. The expert system requires prior information on the flow regime which could affect its objectivity. Similarly, clustering may not affect performance accurately due to its poor handling of transitional data points (Hu et al 2011). Usually, the process of the neural network development is to train the network to recognise the measurement error in the training data and then the network is tested on another set data. If the trained network is accurate enough then it is implemented for online measurement for prediction error correction (Liu et al 2001). More importantly, the neural network would offer a non-linear mapping between the ultrasound input signals and the predicted flow regimes. So the use of ANN avoids the need for calibration of the multiphase flowmeter (Figueiredo et al 2016). Filletti and Seleghim et al (2010) developed a numerical simulation measurement of the interfacial area and volumetric fraction in two-phase flow using an acoustic signal and ANN to investigate the feasibility of the application of the ultrasound system for a clamp-on flow measuring system. They found that the trained ANN models were able to estimate the values of the volumetric fraction and the interfacial area. Similarly, Figueiredo et al (2016) employed an ultrasonic methodology based on pulse wave ultrasound transducers which operate on the principle of signal attenuation detection. The ultrasound signal attenuation was analysed and incorporated with ANN for flow pattern detection and void fraction measurement. They suggest that the flow regime identification in the 2″ pipe was limited to bubbly flow and slug flow only. The technology presented is appropriate for the detection of the gas volume fraction and flow regime determination in multiphase flow. However, there does not appear to be any consideration of the flow regimes except of the bubbly flow and slug flow. However, according to the authors' knowledge, studies on the two-phase flow regimes classification using a clamp-on CWDU have not been reported in the open literature.
The main aim of this research was to investigate the feasibility of a non-invasive method of flow regimes classification using an ultrasonic Doppler sensor and a neural network. A continuous wave ultrasound Doppler sensor employed in this study has recently been implemented for investigating the velocity characteristics of slug body and film in a two-phase gas-liquid slug flow. The results showed that the velocity characteristics of the slug flow obtained are in good agreement with other experimental methods (Fan et al 2013). The present approach records and processes ultrasonic Doppler signals on the flow and then features are extracted using both PSD and wavelet transforms methods. These features are the inputs for the ANN models which process it for the flow regime classification. The multilayer perceptron neural network (MLPNN) with three layers (namely, inputs, varying hidden layers and four output neurons) were developed to map the flow regimes. The four numerical outputs selected to represent the flow regimes are as follows: the elongated bubble, slug, stratified flow and stratified wavy flow (Kandaswamy et al 2004, Subasi 2005, Übeyli and Güler 2005. Despite the limitation of ANNs whereby they cannot perform accurately outside the range of the training sets, a combination of several neurons of the ANN are able to 'learn' and memorise the data original variability so as to function as an objective flow regime classifier. Therefore, ANN can still appropriately fulfil the requirement for multiphase flow monitoring processes such as the flow regimes classification and prediction of the individual phase flow rates in the multiphase effectively (Rosa et al 2010).

Two-phase flow test rig
A horizontal air-water test rig for two-phase flow assessments at the Cranfield University's flow laboratory was used to conduct these experiments. A schematic diagram of the test facility is shown in figure 1. The flow loop includes a closed loop PVC pipeline of 50 mm internal diameter with a total pipe length of 21 m. An air compressor provides the air flow and the water is pumped into the loop from a storage tank of 2 m 3 capacity using a 40 m 3 h −1 water pump. The flow rates of air and water are controlled by regulating hand valves and are measured using a turbine gas flow meter (QFG 25B/B/EP1, Quadrina) while the water flow rate is metered with an electro magnetic (EM) flow meter (Altoflux K280/0, Altometer). The measurement section for the two-phase flow is made up of Perspex pipes that allow visualization of the flow regimes. The clamp-on ultrasound Doppler flow sensor was fixed on the bottom of the pipe.

Ultrasonic Doppler senior
The ultrasonic Doppler flowmeter used in this study is a noninvasive fluid flowmeter complete with its sensor, modelled as a DFM-2 manufactured by United Automation Ltd, Southport, UK. This flowmeter is suitable for measuring the flow of any ultrasonic reflective fluid. It measures the frequency shift, processes the signal, computes the flow velocity and gives out digital displays of the flow velocity in foot per seconds. A green LED shows the strength of the ultrasonic signal reflected back from the flow. It is recommended that it is placed on the flow pipe at least ten diameters from the bends, valves and tees, so as to prevent measurement errors from swirls, cavitation and turbulent eddies United Automation Limited (UAL). The basic components of the Doppler flow meter system are shown in figure 2.The device measures the Doppler frequency shift of the ultrasonic signals reflected from the scatters or discontinuities, such as bubbles in the flowing liquid. The sensor of the flowmeter is placed at the bottom of the pipe at a 6 o'clock position for the horizontal flow measurement test to avoid attenuation of the signal from gas voids in the upper pipe section. It is important to have good bonding between the sensor and the external pipe surface; a glycerine gel was used for good coupling which prevents the trapping of air cavities between the pipe surface and the sensor. The continuous wave Doppler flow meter has two transducers: one for generating the sound wave and one for receiving the ultrasound reflected by the scatters in the fluids, such as air bubbles or particles in the flow.

Measurement principle
The Doppler flowmeter systems used in this study have a transducer which has dual piezoelectric ceramic elements.  The transducer is excited by the electronic circuit of the flowmeter in continuous mode; the transmitting part of the transducer sends out an ultrasonic signal and the receiving part, which detects the ultrasonic Doppler sensor, provides the outputs signals. The output signal received is then filtered and amplified by the electronics of the flowmeter. The processed output signal is the Doppler frequency shift signal and it was captured using a data acquisition card (NI-PCI-6040E) and a LabVIEW program controlled sampling frequency of 10 kHz for 20 s for each dataset. The process of developing the flow regimes classification is described in functional blocks flow chart as shown in figure 4.
The relationship between the velocity of the scatters v and the Doppler shift f d is given in equation (2.1) (Sanderson and Yeung 2002).
where v = average flow velocity, c = velocity of sound in the fluid, f d = Doppler shift frequency, θ = angle between ultrasound beam and flow velocity and f t = ultrasound transmitted frequency. Usually two transducers are required for Doppler flow meters. However, these two transducers can be made into separate units or one compact unit, as shown in the figure 3.

Ultrasonic flow signal data acquisition and test matrix
Each of the test data sets of the experiment created by setting the liquid flow rate to the desired value using a hand operated valve and the flow rate is measured using the EM flowmeter. First, the signal corresponding to this flow rate is recorded for calibration and air flow injected into the rig from an air compressor by regulating a valve by hand. The flow of the air supply is varied in steps, thus generating several two-phase sets in one particular liquid flowrate. The gas flow is measured with the turbine gas flowmeter. The temperature and pressure of the gas at the turbine meter location is recorded for each flow. The superficial gas velocities ranged from 0.05 m s −1 to 2.75 m s −1 .
All stated superficial gas velocities are at standard conditions (1 bar, average of 22 °C). The superficial velocity of the liquid flow was varied between 0.004 m s −1 and 2.0 m s −1 . The superficial velocity values were calculated as the ratio between the volumetric flow rate of the phase and the cross-sectional area of the pipe. These flow regimes were characterized by distinct phase and velocity differences in the cross section of the pipe. Each of the two-phase flows of the dataset's flow regimes were visually observed, identified and recorded for comparison with predicted flow regimes. The total test data sets of the experiment tabulated as the two-phase test matrix includes the following flow regimes: elongated bubble flow, slug flow, stratified flow and stratified wavy flow as shown in the figure 5. The flow parameters recorded in the experiments are: the initial liquid flowrate, liquid superficial velocities, superficial gas velocities, ultrasonic reflected signals, temperature at the gas flowmeter, temperature at the test section, pressures at both the gas and test sections. The LabVIEW program controlled the data acquisition at a sampling rate of 10 kHz for 20 s. The preliminary tests show that this sampling frequency is sufficient as it is more than twice the highest frequency of the Doppler signals.
A four-category classification chosen for this purpose and the four regimes considered are: • Stratified flow: when the liquid phase flow at the bottom and the gas phase is at the top, and the interface of the two flow phases is smooth.

Spectral analyses and feature extraction
Spectral analyses techniques are required for the analyses of the signals of two-phase flow to obtain the oscillation period. Two-phase flow signals can be analysed either in the frequency domain to obtain characteristics of the different flow regimes, or using algorithms such as PSD and wavelet transform which is the time-frequency analysis used. In this work, both PSD and wavelet transform techniques have been applied to two-phase flow signal records of the two-phase signals acquired using the ultrasonic Doppler sensor (Shang et al 2004). Frequency domain methods using the PSD have been used in analysing two-phase flow to get oscillation periods based on the Fourier transform of the signal (Xie et al 2004). The wavelet transforms have the capability of analysing and denoising the signals to produce the spectrum in the time-frequency domain.
2.5.1. Power spectral density. Frequency domain methods are often used to reveal the distinctiveness in the signal of flow regimes in two-phase flow systems. The PSD is a method of estimating characteristics of a time-series signal of a stochastic process in the frequency domain that is suitable for detecting the frequency components hidden in the process (Matsumoto and Suzuki 1984). The application of PSD to a time series signal, such as a two-phase flow pressure fluctuation signal, has been studied by several researchers (Xie et al 2004, Sun and Zhang 2008, Santoso et al 2012. The PSD is used to produce the characteristics of the two-phase flow signal in the frequency domain which has shown that signals of the flow regime are distinctively different. Fast Fourier transforms are used to create the PSD spectrum, which assumes that the process signal is stationary. of the signal as shown in the equation below (Xie et al 2004): The application of the PSD function to a real valued continuous data using the autocorrelation sequence can be approximated by a time-average. However, in the application of the function to the measurement signal which is recorded for a finite time interval, this may present some distortions. As a consequence, a modified form of the PSD, called the Welch method, is often adopted in these applications. The Welch method subdivides the signal sample into small length N-point overlapping segments and then obtains the periodogram of each of the segments. The power spectrum is estimated by the average of the periodograms (Xie et al 2004).
The Doppler ultrasonic frequency signals were processed using the MATLAB software package (MATLAB Version: 8.3.0.532) to analyse their spectral contents. The power spectral densities method using the Welch method with a segment length of 256 point and a Hanning window to alleviate distortions compute the spectra (Beale et al 2013) as applied by Xie et al (2004). Examples of the power spectra estimates of the samples of the signals are shown in figures 6 and 7. It can be seen that the spectrum of the slug flow signal has the highest power on the spectrum. The slug flow ultrasound signal contains higher Doppler frequency shifts as well as translational slug film velocity and single phase flow. However, the spectrum of the signal of the single phase flow has produced a higher signal power than both the stratified flow and bubbly flow signal but much less than that of the slug flow signal. The rich ultrasonic shift frequency could be obtained from the signal phase flow because the flow pipe was full. Also, in the slug flow, there were intermittent full pipes and in the  liquid slugs a flow with higher velocity than the corresponding single phase flow. The spectrum of the bubble flow regime has a slightly similar pattern to that of the slug flow.
The principle of extraction of the features from the PSD is based on Parseval's theorem which states that the PSD is the measure of the total energy of the signal if the spectrum is integrated over its entire frequency band. The sampling frequency and length of the signal play an important role in the statistical properties. In the present study, the sampling frequency is chosen to be 10 kHz so as to exceed the Nyquist criterion for the signal of the experiment, and can be seen in figure 7. The PSD spectrum produced from the signal recorded with an effective ultrasonic Doppler sensor has a frequency band from 0-600 Hz. It is obvious that each of the PSD features extracted are distinct and therefore the PSD feature is a good choice to use to identify the flow regimes of gas-liquid flow. As a result, the PSD magnitudes were normalised and their amplitudes correspond to each data record in the frequency range. Each frequency band of the two signals was divided into five bands. The PSD of the ultrasonic signal were partitioned into 120 Hz ranges as shown in table 1, which is representative of each flow signal created using a mean value of the frequency bands of the PSD spectrum computed as P B1 , P B2 , P B3 , P B4 , and P B5 (Matsumoto and Suzuki 1984). Other properties estimated from the spectrum are the weighted mean of the frequency over the entire band and the variance of the mean frequency (Sun and Zhang, 2008). In total, the five power magnitudes, weighted mean frequency, and variance are used as the features to represent the signal of the flow. The mean weighted frequency of the spectral power is given by equation (2.3) and The use of these seven discrete parameters (P P P P , , , , 2 ) to represent the signal was first suggested by Drahos and Cermák (1989) and the method has been implemented by many researchers such as Shaban and Tavoularis (2014) and Xie et al (2004).

Discrete wavelet transform.
The idea of signal decomposition using the discrete wavelet transform (DWT) is not new. However, its usefulness lies in the ability to manipulate the wavelet coefficients to identify characteristics of the signal distinct from the original time signal (Subasi 2005). In this work, the decomposition of the ultrasound Doppler signal and the conductance signal from the two-phase flow were carried out using the DWT. The procedure for the extraction signal features uses DWT to represent the required selection of the wavelet type and level, and multiresolution decomposition and selection effective coefficients of the discrete wavelets of the decomposition to represent the signal (Shang et al 2004).
The DWT of a signal for feature extraction is on the principle of multiresolution signal decomposition in which a signal is filtered using a half band high-pass filter and lowpass filters (Subasi 2005). There are a number of different wavelets and their levels to choose the decomposition of the signals. It is important to select the suitable wavelet type and level to structure the wavelet filter for the decomposition. Wavelet type selection is either by visually inspecting the data for continuity or testing the various types of wavelets with a signal and the most efficient one is selected. If it is of a discontinuous type then Harr or sharp or else a smooth wavelet such as Daubechies wavelets is recommended. Daubechies wavelets level 2 was used to compute the wavelet coefficients of the signal in this study (Kandaswamy et al 2004).
The decomposition of the signals produces approximations and detail levels with different frequency bands by using a successive low-pass and high-pass filtering. These detail levels will not lose their information in the time domain (Bendjama et al 2015). However, useful information can be obtained from the subbands of the dominant frequencies, so statistical measurements of the subbands are representative of these detail levels. The signals of the flow were decomposed continually until all the dominant frequency ranges were viewed. The signals do not have any useful frequency below 40 Hz and that is why the decomposition ended at level 7 which is the level at frequency subbands greater than 40 Hz. Therefore, the Doppler signal was decomposed into detail coefficients of − D D 1 7 where 1-7 refers to the detail wavelet coefficient levels: the first to the seventh and the last approximation is A 7 . The ranges of the frequency subbands are given in table 2. The Daubechies wavelet of the order 2(db2) was used to  compute the wavelet coefficients of the signal. The computation of the DWT of the coefficient was done in the MATLAB software package (Misiti et al 1997).
For each of the datasets, detail wavelet coefficients at the first level, second level and up to the seventh level were computed. Importantly, to reduce the size of the feature extracted from the coefficients, statistical measurements were applied to the values of D D D , , , Features 1-3 represent the frequency distribution of the signal and feature 4 the amount of changes in frequency distribution. These are the statistical features used to represent the two-phase flow and as inputs into the neural network for the flow monitoring (Übeyli and Güler 2005). The wavelet detail coefficients of the signals are distinctly different. Figure 8 shows the detail wavelet coefficients corresponding to the D 1 frequency of the two-phase flow.

Multilayer perceptron neural network model
MLPNN is a nonparametric technique for conducting various processing techniques for solving function approximation, pattern recognition, classification and estimation problems, and its operation is governed by a set of weights and biases (Übeyli and Güler 2005). The general structure of the MLPNN with two successive layers is shown in figure 9. The structure of the MLPNN model can be represented by equation (3.1).
The hidden layer is the unit in between the input layer and output layer. Its adjustments are not accessible from outside the network (Luntta and Halttunen 1999).
f is the activation function which transforms the weighted sums of all the input signals on the neurons. The activation function ( f ) can take many forms such as: threshold functions, or a sigmoidal, hyperbolic tangent or radial basis function. The sigmoidal function is one chosen for this study.
The performance of an MLPNN can be improved by adjusting the weights of the network to reduce E the difference between the desired output and the actual values of the neurons as fast as possible. y dj is the desired value of the output neuron j and is the actual output value whose values can be adjusted and then chosen using the set of targeted outputs. The flow regimes classification with ANN is implemented using pattern recognition algorithms. The pattern recognition comprises three steps: (1) data acquisition and pre-processing, (2) data representation or feature extraction, and (3) decision making or pattern classifying. One important aspect of pattern recognition is the learning from the training data set (Basheer and Hajmeer 2000). In this study, the training process consists of the determination of the MLPNN model parameters which are used to validate their quality, and the ability to classify once the training has been completed (Subasi 2005). Alternatively, training refers to the process of adjusting and selecting the appropriate weights and biases (Bishop and James 1993).

Combined neural network models
A combined neural network (CMLPNN) is a method of improving the performance of the network's predictive accuracy. In designing neural network models, the training data may fail to learn to predict the output accurately, so the network is unable to generalise the concept precisely. The learning system of the network utilises the transformed data to predict the output with greater accuracy. Stack generalisation, introduced by Wolpert (1992), is a method of combining low level network models into a high level neural network to  achieve greater predictive accuracy. By transforming the data into a suitable form, the training process can be enhanced. The generalisation minimizes the error rate of the combined network by 'teaching' a second level network whose inputs are the prediction of the first level network but the second trained with the same target output as the first.
MLPNN were used to form multiple networks and then combined to form a stack generalisation. Figure 10 shows the structure of the combined network. The features extracted from the sensor signals are used as the input to the first level network of the combined neural network. After that the outputs of the first level were fed into the second level network as the inputs. The outputs of the second levels are the result of the predicted flow regimes. Both the first level and second level neural networks are trained with targeted outputs (Übeyli and Güler 2005).
MLPNN were used to form multiple networks and then combined to form a stack generalisation approach to the flow regimes classification. Figure 11 shows the structure of the combined network used to predict targeted flow regimes. Both the first level and the second level neural networks were trained with targeted superficial flow velocities as the outputs.

Flow regime classification network
Several applications of the MLPNN pattern recognition are reported in the literature. It is based on training the neural network to recognise the correct classification for each member of the training datasets. Training the network will be subsequently followed by testing the network to classify the input variable into their correct classes. If the learning process has taught the network the patterns relevant to the test data, then it is expected that the network would classify correctly (Xie et al 2004).
Flow regime identification using ANN pattern recognition can be implemented by using either a supervised neural network (SNN) or an unsupervised neural network, aslo known as a self-organising network. The SNN uses feed-forward networks such as multilayer perceptron and radial-basis function networks for pattern recognition in which the information of predefined classes are used to classify the input variables into the classes. However, the Kohonen-network or self-organizing map used for data feature mapping does not need information of the classes. It uses the network clustering method to group the input variable into several classes that contain similar characteristics (Mi et al 2001). The Kohen self-organizing neural network has been implemented for flow regime classification using measurement data points of distinct flow regimes (Cai et al 1994).
To select the input to the network for the pattern classification, first it is essential to pre-process, balance and normalise the data. The features extracted from the ultrasonic Doppler signal of the flow using both the wavelet method and the PSD methods are pre-processed before presenting them as input variables to the network. As part of the pre-processing, the input dataset, data partitioning and balancing are applied to the feature. There several ratios are used to partition the datasets into training, testing and validation. At this moment, there is no mathematical rule for determining the exact sizes of the training, testing and validation datasets. The often used ratios are 60%, 30% and 10% or 65%, 25% and 15% for training, testing and validation respectively (Basheer and Hajmeer 2000).
Another aspect to preparing the input data is the balancing datasets which distribute the training nearly evenly amongst the various classes to annul the effect of the network from being biased to overrepresented classes. As part of the input   Figure 11. A structure of the flow pattern prediction combined neural network topology (Übeyli and Güler 2005).

Output Class
Confusion Matrix data preparation, first balancing and then the normalisation process is applied to prevent chaos in the network as a result of either the larger numbers overriding smaller ones or premature saturation of hidden nodes. Normalisation usually confines the data into a uniform range of 0-1. A good rule of thumb is to scale the input variables (z i ) and the output range (λ λ , 2 1 ) in the interval of the output values which correspond to the function in equation (3.3) (Basheer and Hajmeer 2000).
Statistical analysis of the features plays an important role in selecting the input variables for successful neural network application. Güler and Übeyli (2006) computed the statistical features (mean, maximum, minimum and standard deviation) to represent the time-frequency features extracted from the Doppler signals using the wavelet transform. The output of the neural network is the indicator of the flow regimes or classes of the classification which is represented with a continuous or binary discrete number. Each of the input variables to the neural network is assigned to the class to which it belongs. Usually, these classes are represented with numerical values. The two most common representations of the classes are: continuous (0.3, 0.5, 0.7, etc.) or discrete (0 and 1 or 0.1 and 0.9). Xie et al (2004) have implemented the continuous number to represent the output of the neural network to indicate flow regimes. They used continuous numbers (0.3, 0.5, 0.7 and 0.9) to represent the flow regimes: bubbly flow, plug flow, churn-intermittent and slug flow respectively. This technique was supported by their earlier work (Xie et al 2003). The continuous designation number to represent the output of neural network flow classification methods has been applied by other researchers as well (Sun and Zhang 2008).
However, due to the importance of the discrete output for extracting rules from the trained neural network, Basheer and Hajmeer (2000) have suggested that the continuous variable should be replaced by discrete or binary numbers for representing the output of the neural network classifier. There are methods and algorithms for discretizing the output variables. The discrete or binary number is often modified from the (0 and 1) to (0.1 and 0.9) so as to prevent saturation (Basheer and Hajmeer 2000). Also, allocating the targets of 0.1 and 0.9 instead of the common  practice of 0 and 1 prevents the outputs of the network from a directly interpretable posterior (Kandaswamy et al 2004).
The most frequently used training model in classification problems is the BP which is adopted for this investigation and in other works (Blaney and Yeung 2008, Arubi 2011, Fan and Yan 2014. The MLPNN has properties such as the ability to learn and transform fewer training set requirements and fast processing. The manner in which the weights can be   (Übeyli and Güler 2005). The training tool has functions for performance, the magnitude of the gradient of performance and the number of validation checks. The magnitude of the gradient and the number of validation checks are used to terminate the training. The gradient will become very small as the training reaches a minimum of performance. If the magnitude of the gradient is less than 1 × 10 −5 , the training will stop (Beale et al 2013) Cross validation in neural network pattern recognition is required to determine the optimum number of hidden units and the model that will perform best with the problem at hand (Bishop andJames 1993, Kandaswamy et al 2004). A best network model for the flow regime classification is obtained after testing several training algorithms. The performance test of the input datasets is determined by the computation of the total classification accuracy and number of training epochs. Total classification accuracy is the number   of correctly classified flow regimes/number of total datasets (Übeyli and Güler 2005).

Flow regimes classifier neural network training and testing
Eighty-six dataset measurements on the horizontal two-phase flow were used for this experiment. Sixty-two datasets were used for training the networks and 24 were used to test the network. Two sets of CMLPNN developed in the MATLAB software package (MATLAB Version: 8.3.0.532 (R2014a) with neural network toolbox) were used to classify the flow regimes of air-water two-phase flow using features from ultrasonic signals of the flow. The inputs to the network are features extracted from the signals of the flow using both PSD and DWTs. The outputs of the combined neural network are (discrete binary) indicators of the four flow regimes: elongated bubble flow, slug flow, stratified flow and stratified wavy flow.
In this training process, the classification scheme of '1-of-C coding method for classification' was adopted to classify the inputs, and each member of the datasets belongs to one output of the four flow regime categories. The four predetermined output values are as designated in the following equations: (3.5)-(3.8). The values are the targets presented to the network as outputs (Subasi 2005). The following BP algorithm and training algorithms used in the network and their performance for the flow regime identification are examined. The multilayer perceptron has three activation functions for regulating its output: pureline, logsig and tansig. In this study, the sigmoidal function was used throughout due to its properties, such as it ranges the output between 0 and 1, nonlinear paving the way for complex mappings of the input to the output and it is continuous and differentiable (Güler and Übeyli 2006). Important aspects of the neural network development are architecture and the training process. Several training algorithms and neural network architectures have been tested with a different number of the hidden layer neurons of the network evaluated during the training process.
Before the fusing of the neural network models, single level neural models were tried out to design the flow regime classifier, but the results were not good enough.
Subsequently, the single levels were integrated into the combined neural network. The neural networks with single layers were found to be superior to the two hidden layers in this experiment. The most efficient configuration for the network with PSD features was 12 neuron hidden layers while that of the DWT features was the 28 neuron in the hidden layers.
Combined neural network design: The combined network algorithm to determine flow regimes involves five sets of neural network models: N1, N2, N3, N4 and N5. The models N1, N2, N3 and N4 are the first level MLPNN formed with the input (features)-outputs relationships. The N5 model is the second level network of the combined network and it uses the outputs of the first level network as the inputs while using the same output as the first level network. Each member of the first level network produces four outputs corresponding to the flow regimes, and these outputs are concatenated to form a vector for the input of the second level of the combined network. Hence, the combined neural network for the flow velocities training for the four different set of features are extracted.

Results and discussion
In this study, horizontal gas-liquid two-phase four flow regimes were classified from ultrasonic Doppler signals processed using ANN. The inputs to the neural network are features obtained from the frequency bands of both PSD and DWTs. After the features are extracted from the signals, they are normalised. Flow regimes classification models are based on feed-forward MLPNN implemented in the MATLAB software package with a neural network toolbox. In order to improve the performance of the ANN classifier the networks were integrated into a two-tier network called a combined neural network (Übeyli and Güler 2005). A total of 85 datasets were divided into training and testing sets-62 data samples were used to train the network and 24 datasets were used to test the network.

Feature extractions
The features from the Doppler ultrasound signals of the flow extracted using two methods of feature extractions were applied in this study. (1) DWT to generate the frequency bands by decomposing the Doppler signal and then applying each of these statistical measures: of mean, maximum, minimum and standard deviation to the results of the wavelet transform. (2) Frequency domain spectral analysis of the ultrasonic Doppler signal has been implemented using PSD based on the Fourier transform technique. The PSD spectrum is further averaged to be an input or representative of the flow in the neural network, and it is necessary to extract statistical moments from the spectra (Xie et al 2004). Seven frequency bands or levels were obtained from the detail wavelet coefficients, so for each of the datasets we obtained 28 features from the 7 detail wavelet coefficient levels and the statistical measures applied to the wavelet levels.

Flow regime identification
The testing of the flow regimes classifier was done using 24 datasets comprising 6 samples of the 4 flow regimes. After the flow regime classification neural network has learned the input-output relationship, then the two classifiers, one for each feature vectors, are tested with the datasets that the network has not seen before. The classification errors in the testing of the new datasets are evaluated and shown in the form of confusion matrices in figures 12 and 13 for the PSD and DWT features respectively. The diagonal cells show the number of datasets that were correctly classified, and the offdiagonal cells show the misclassified datasets. The blue cell in the bottom right shows the total percent of correctly classified cases (in green) and the total percent of misclassified cases (in red). The result shows that the PSD trained network has missed three datasets in the classification, while the DWT trained network has misclassified only a data point.
The flow regimes classification performance was evaluated in the form of confusion matrices. A misclassification is said to occur when the classifier fails to align a flow regime into the supposedly right group or class. From figures 12 and 13, the summary of the classification accuracies of each flow regime and each method of classifier based on the features used in the development of the systems are presented in table 5.

Comparison of visually observed and classified flow
regimes. Table 6 shows the predicted flow regimes with a classifier using the PSD features which is quite good, with only 3 misclassifications out of the 24 data records. The three misclassified flow regimes are cases no. 7 which is identified from slug flow to elongated bubble flow, and no. 17 which is also identified from slug flow to elongated bubble flow. The last misclassified case is no. 23 which is from stratified-wavy flow to stratified flow. These misclassified flow patterns are denoted as F for false prediction whereas the successfully classified flow regimes are denoted as T for true. Also, misclassified flow regimes are very similar to their actual targeted flow regimes. It has been found that only the nearby flow regimes were confused in the neural network as in the results of other work (Sun and Zhang 2008). The images of the flow regimes are shown in the figure 14.
Similarly, in table 7, the DWT feature from the test dataset is applied to the trained neural network. This classifier was able to match all flow regimes except only one in case no. 18, where it was misclassified from stratified flow to slug flow. Importantly, its overall performance is that it can classify the flow pattern up to 96%. As a result, the combined neural network built using the MLPNN and DWT features has a higher quality of classification than the one trained with PSD features. These results are similar to flow pattern classification works found in previous studies (Hernandez et al 2006, Sun andZhang 2008).

Comparison of the performance of PSD and DWT
features. Performances of the six MLPNN structures trained using six different training algorithms for the flow regimes classification are compared. The comparison is illustrated in figure 15. The training algorithm with a single hidden layer gives the best performing network.

Conclusion
In this paper, the development of a clamp-on CWDU sensor and ANN for gas-liquid two-phase flow regimes classification was performed. The ultrasound signals are processed using the methods of PSD and DWTs for the extraction of input features to the ANN models. MLPNN is developed into a combined neural network of two-levels as the classifier and the two methods of feature extractions produced for the network. Comparison of the performances of the two classifier models is assessed. Four two-phase flow regimes such as slug flow, stratified flow, elongated bubble flow and stratified wavy flow have been classified using the method developed.
The results show that the ultrasound signal features of the two phase flow obtained using the DWT perform better accurate classification than compared to the features extracted with PSD methods. The CMLPNN developed for the classification using the PSD features and for the DWT features have accuracies of 87% and 95.6% respectively. In conclusion, the present study has demonstrated that DWTs feature extraction and the MLPNN classifier have met the industrial requirement of flow regime classification (Sun et al 2013).
In contrast to the invasive instruments used in other works, this approach is very important for industrial application given that the sensor used is non-invasive, non-radioactive and uses ultrasound technology. Based on the analysis of the experimental results, the proposed method is able to provide objective classification of four flow regimes in the horizontal pipe. Other ultrasonic methods reported in the literature employed pulse echo ultrasound plus a neural network or subjective methods of flow regimes indemnification (Jha et al 2013, Figueiredo et al 2016. The key strengths of the neural network based methods are fast classification and flexible procedure for finding good nonlinear solutions. The continuous wave Doppler sensor is suitable for monitoring flow processes that are ultrasonical reflection fluids such as crude oil, petrocarbons, oil-gas and oil-water mixtures. Importantly, the sensor can fit well onto existing pipework and it is suitable for both plastic and metallic pipes. However, on a horizontal pipe, it is important to mount the sensor at the bottom of the pipe to avoid gas voids in the upper section of the pipe. Poor coupling of the sensor with the pipe, gas voids or bends would temper with the strength of the signal received. The results of the present study demonstrated that the proposed approach of wavelet transforms and the MLPNN classifier has met the industrial requirement of flow regime classification (Sun et al 2013).
Further studies are needed in the application of this clampon objective flow regime classification system to investigate oil-water two-phase flow, especially for the deployment of this technology to address the requirement of the clamp-on ultrasound flow monitoring meter for oil well testing. Also, more studies are recommended on the feasibility of the information of the ultrasound Doppler sensor and void fraction measurement, such as a gamma densitometer, which would make the system a complete multiphase flow meter.