{'fraction_non_alphanumeric': 0.024877876819257693, 'fraction_numerical': 0.008712292894193483, 'mean_word_length': 4.934847579199044, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Recognition of Plants with Complicated Background by Leaf Features

Leaf classification is a significant and meaningful work. However, the interference and overlapping of objects may affect the recognition effect of leaves with complicated background. In this paper, a hybrid framework of classifying leaves with complicated background is proposed. Firstly, a novel watershed segmentation based on iterative opening and closing reconstruction is introduced to segment leaves from complicated background, which contains texture and shape information of leaf. Then, the block Local Binary Pattern(LBP) operators, whose dimensionality is reduced by locally linear embedding(LLE), are extracted as texture features. In addition, the shape features of leaves are described with the Fourier descriptors. Finally, the texture features and shape features are combined as the input of Support Vector Machine(SVM) classifier to realize the accurate classification of plant species. Experimental results show that the proposed method is effective.


Introduction
There are a great variety of plants on Earth. The classification of these plants will be difficult only by sense of sight. Thus, the plants classification and recognition based on machine vision have been a hot topic [1]. Plant classification is the process that a plant is correctly assigned to a series of related plants. In recent years, information technologies, including image processing, pattern recognition and so on, have been introduced into plant classification to make up the deficiency of sight classification ability [1][2][3][4]. The leaves, as significant components of plants, are often used for computer plant recognition and classification.
At present, most studies of the plant classification and recognition focus on leaf specimens with single background [2,3]. But common pictures of leaves inevitably contain complicated background. Consequently, the plant classification method of single background is not suitable for practical applications. In this case, we can obtain those leaf images by the study on the segmentation of plant leaves with complicated background. Then the recognition method with single background can be used to process the segmented leaves.
On the study of leaves segmentation with complicated background, Camargo [5] utilized Gustafson-Kessel clustering to segment leaf images and introduced genetic algorithm to extract leaf. However, the robustness of this algorithm was low for leaves with overlapping and complicated background. Wang et al. [6] put forward auto-marked watershed algorithm to segment leaf images, and then extracted the Hu moments and Zernike moments as leaf shape features to recognize the plants. However, this method remains shape information by depending on the previous information of leaf  2 shape and neglects the texture information of leaves. Tang et al [7] proposed the marker-controlled watershed segmentation algorithm based on Hue-Saturation-Intensity (HSI) space to extract target leaves from soybean leaf images with complicated background. But the algorithm may cause the over-segmentation if the target is other plant leaves with apparent leaf vein.
When facing feature extraction, it is mainly about the color, shape and texture features of leaf [8][9][10]. In the early stage of plants classification and recognition, the researchers paid more attention to the shape feature of leaf. Ingronille et al [8] extracted twenty-seven shape features of leaves to classify the Oak. However this method was not much effective in the recognition of the plant leaves, because leaf shapes of different plants may be similar. In recent years, wavelet transform and grey-level co-occurrence matrix(GLCM) were introduced to extract the texture features of plant leaves. Cope et al [11] utilized Gabor filter to capture the texture features to recognize leaves, and the recognition rate can up to 85.16%. Rossatto [12] introduced determine volume as the texture features to recognize leaves images. Ojala [13] first applied the LBP algorithm to extract the texture features of face. There was good robustness about the gray scale of the LBP operator, so that, it could decrease the influence of illumination variation on the texture features of leaves. Yadav [14] introduced discrete wavelet transform (DWT), LBP, first-order statistics(FOS) and histogram Fourier(HF) to form four hybrid texture features to classify the microscopic images of hardwood species. LIU [15] combined texture features (LBP, MLCM and Gabor filter) and shape features (Hu moments and Fourier descriptors) as the fusion feature in plants recognition to improve the recognition rate [16], a great diversity of features would cause too much redundant information and high-dimensional data.
In order to reduce dimensionality of data, a large number of algorithms include LLE, Linear Discriminant Analysis (LDA), Principal component analysis(PCA) and so on were proposed. Especially LLE is often used to discover the intrinsic low dimensional manifold structure of high-dimensional data and to classify and recognize data influenced by noise effectively [17]. On the flip side, when the images are translated or rotated, LLE can also achieve a good dimension reduction [18].
It is obvious that the key point of leaf based plant recognition in natural environment is how to segment the single plant leaves from leaf images with complicated background and promote the efficiency of leaves recognition. To solve the problem, this paper proposed an efficient watershed segmentation method based on iterative opening and closing reconstruction for leaf classification with complicated background. Then, the block LBP operators are used to describe the texture features of the segmented leaf images. Considering that the high-dimensional texture features with block LBP operators may influence the effectiveness of leaves classification, we introduce LLE algorithm to reduce the dimensionality of texture features. Meanwhile the Fourier descriptors [19] are extracted from segmented contour images as the shape features of leaves. Finally, the low-dimensional texture features and the shape features are combined to be classified by an SVM classifier. The results of experiment show that the proposed method is effective for the classification of the leaf images with complicated background.

Segmentation Process
Leaf segmentation with complicated background as shown in Figure 1 can be given as follows: ○ 1 Obtain image without non-green background and its HSI space; ○ 2 I vector is iteratively reconstructed based on opening and closing operations;

Non-green Background Removal
Studies have shown that leaf color is concentrated on the green pace of RGB color space [20]. Therefore, we transform the target image from RGB color space into the J vector space as formula (1), which mainly composed by green space. x The threshold j  of J vector space is selected by Otsu thresholding method [21]. The gray values of J vector space are divided into two regions, one is below j  and the other exceeds. Therefore, the pixels in target image corresponding the former region are set to zero to remove the non-green background. The process is shown in Fiugure 2(B).
The HSI model reflects the way people perceive colors, I space represents intensity of objects. As shown in Fiugure 2(C), the image without non-green background is transformed from RGB space to HSI color space. The following watershed segmentation focus on I vector.

Iterative Opening and Closing Reconstruction for Leaf Segmentation
Watershed segmentation [22][23] is a region growing technique belonging to the class of morphological operations. A digital watershed is defined as a small region that cannot be assigned uniquely to any influence zones of local minima in the gradient image. To decrease the possibility of over-segmentation, a watershed segmentation algorithm based on iterative opening and closing reconstruction is proposed. The process is designed as the following steps.
Step 1: In order to reduce the influence of leaf texture and vein on segmentation, I vector will be iteratively reconstructed based on opening and closing operations as follows: a. Calculate the Inverse Different Moment (IDM) of I vector to describe the texture complexity of leaf, as shown in formula (2).
where ( , ) G i j is the GLCM of I vector, K is the gray level. b. If IDM is greater than the threshold  , I vector is accepted as the final reconstructed image R as shown in Figure 2(D), and the iterative algorithm is stopped; otherwise, go to next step.
c. Execute erosion and reconstruction operations to obtain reconstruction image based on opening; based on this, dilation and reconstruction operations are applied to obtain the opening and closing based reconstruction image which is regarded as a new I vector. Then return to a.
It can be seen from Figure 3 that the vein and texture of leaf are weakened after iterative reconstruction, and the complete leaf can be extracted by watershed segmentation.

Figure 3. Results of iterative reconstruction
Step 2: Calculate the partial derivative x G and y G by convolving the reconstructed image R with Sobel horizontal and vertical operators respectively [24]. Recalculate the pixel gray value by x G and y G , as shown in formula (3) where GAI is the Gradient magnitude image of R, Figure 2(E) shows the Gradient magnitude image. Figure 4(A-B) shows that the gradient of the leaf contains a large number of local minima which lead to over-segmentation. Marker-watershed segmentation was proposed is to filter out the undesired minima of the gradient magnitude image according to the foreground and background markers [25]. Therefore, the over-segmentation of the watershed segmentation can be averted as shown in Figure  4(C-D). Step 4: Binary image of R is obtained by threshold segmentation. The Euclidean distance of each pixel in binary image is calculated, which is defined as the distance between that pixel and the nearest nonzero pixel of the binary image. Then the Euclidean matrix is obtained and can be transformed to get the Watershed transform ridge image which is used as background marker (BGM) as shown in Figure  2(G).
Step 5: The gradient magnitude image is modified by foreground and background markers with the mandatory minimum technique of which principle is shown in Figure 5, so that the regional minima only exist in certain desired locations. Figure 2(H) shows the Modified gradient magnitude image.
Step 6: The modified gradient magnitude image is transformed into label matrix L by watershed algorithm as shown in Figure 2

Feature Extraction of Leaf
The features of plant leaves include the color, shape, texture and so on. In order to avoid the influence of seasonal variation on the leaf color, in this paper we only extract the shape and texture features of the leaf. In general, Local Binary Pattern and Fourier Descriptors are extracted as texture and shape features respectively.

Local Binary Pattern
LBP is a texture descriptor with illumination invariance. Its encoding process can be described as follows: Define a 3×3-neighborhood of each pixel. Then take the pixel value of the central point pc as the threshold. Compare the pixel values of the eight neighbors with pc, and if the pixel value is bigger than pc, the pixel is set to 1, otherwise 0. The eight pixels in clockwise direction form a binary number. The binary number is converted to a decimal number, which is the LBP operator for the center point of the window. The LBP operator is measured by the cost function as formula (4).
The encoding method of LBP is shown in Figure 6.

Block LBP Algorithm
The dimensionality of the LBP operator is 2P where P is the number of periphery of the window. The extracted texture feature are too deficient to describe the texture information. Therefore, we use the block LBP operator to effectively describe the texture feature of leaf.
The extracting process of block LBP feature of leaf image can be divided into three steps: ○ 1 he leaf image is segmented into 5×5 regions, as shown in Figure 7. ○ 2 Figure 8 shows the LBP histogram of each region. ○ 3 concatenate all of histograms into an enhanced feature vector in descending order as the leaf feature descriptor.  A 5×5×2P (P=8) dimensional block LBP feature can be obtained from the above process as the texture feature of leaf. But the block LBP feature cannot describe the shape feature of leaf. Therefore, this paper introduces the shape feature as a supplement to the leaves recognition.

Shape Feature of Leaf
Generally, Fourier descriptors, with rotation invariance, is used to describe the contour feature of leaf [13]. After the leaf image is grayed and binarized, the edge of leaf is extracted by Sobel operator [24].  Figure 9 shows the extraction process of leaf edge.  We take the modulus of   a u as the shape features.

Locally Linear Embedding Algorithm
Due to the high dimensionality and enormous redundancy information of block LBP, it is inconvenient to be used for leaf classification. Therefore we introduce the LLE algorithm to reduce the dimensionality of block LBP feature.

Basic Principle of LLE
Assume a high-dimensional database X RD  is composed of n real-valued vectors i x with dimensionality D . These data are sampled from the underlying manifold. First we calculate all the Euclidean distances between the sample points in the database X . K sample points close to sample point i x are defined as the neighbors P of i x . Each sample point and its neighbors are located at or near the localized linear region of the manifold. The local geometry of these regions are represented by the linear coefficients of each sample point which is reconstructed by their neighbors. The reconstruction error function is described as formula (7a).
where W is the weigh matrix, ij W is the elements of W , and represents the contribution value of the j th sample point to thei th reconstruction. The calculation of low-dimensional vectors i y need to minimize the error function (8).
Next we deal with the d+1 eigenvalues of sparse symmetric M in ascending order, and remain the last d eigenvalues to minimize the reconstruction error. The low-dimensional output Y is the eigenvectors corresponding to the last d eigenvalues.

Effect of Dimensionality Reduction
LLE algorithm is able to reduce feature dimensionality under the condition of retaining the main texture feature information. At the same time, LLE could introduce the category information of the sample leaves into algorithm. It reduces the distance of samples in the same class and increases the distance of samples between different classes. So the samples are easy to be classified. Figure 10 shows the clustering effect of features which are obtained by block LBP, block LBP with PCA and block LBP with LLE respectively. As shown in Figure 10(a), the block LBP features in cluster are scattered, and there were no clear boundaries between the three types of leaf samples. Block LBP features with PCA are plotted in Figure  10 9 We extracted six samples respectively from two types of leaves. The former 50-dimensional features of block LBP and block LBP with LLE algorithm are compared in Cartesian coordinate system as shown in Figure 11. In the figure, the solid line represents the Acer palmatum, and the dotted line with triangle represents the Ulmus parvifolia Jacq. As can be seen from Figure 11(a), block LBP features are so overlapping and can hardly be classified that the recognition rate is low. In Figure 11(b), after reducing the dimensionality by LLE algorithm, the features of two kinds of leaves are distinguished.

The Structure of Hybrid Algorithm
The process of plant leaf recognition with complicated background is divided into three parts: segmentation, features extraction and classification. The flow chart of leaf recognition is shown in Figure 12.
Procedure of leaf recognition: ○ 1 The watershed segmentation based on iterative opening and closing reconstruction is introduced to extract single leaf from complicated background. ○ 2 he segmented leaves are preprocessed. ○ 3 Extract the block LBP operators as the texture features of leaves, and reduce the dimensionality of the features by LLE. ○ 4 Calculate the Fourier descriptors as the shape features of leaves. ○ 5 The texture and shape features of leaves are combined into integrated features and classified by SVM classifier.

Experiment and Result Analysis
Fifty species of plant leaves were selected from Wuhan Botanical Garden as the experimental samples. And each species includes 20 training samples with single background and 10 testing sample with complicated background. SVM classifier learns the training samples to get training model which is used to recognize the testing samples. In order to verify the rotation invariance of the algorithm, leaves are rotated to three different angles as shown in Figure 13.   10 The results of the leaf segmentation are shown in Figure 14, the single leaves can be successfully extracted from complicated background by the proposed segmentation method. After segmentation, the single leaves are pretreated by scale normalization and graying. The leaf image is divided into 5×5 sub-images. We calculate the 28-dimensional LBP operator of each sub-image. In order to obtain the consistent block LBP features for leaves in different angles, the calculated LBP features are concatenated in descending order to form a 5×5×28 dimensional feature vector. The dimensionality of block LBP features are reduced by LLE. The tests revealed that when the local neighborhood parameter is set to 40 and the dimensionality is reduced to 50, the recognition rate can be improved without affecting the recognition speed. Fourier features of leaf edge are extracted to combine with 50-dimensional block LBP features. The SVM classifier, whose parameters are c=8,γ=0.0156, learns the combined features of training samples to build a model, and the testing samples are recognized on the basis of this model. Table 1 and Figure 15 shows the comparative recognition results of different algorithms.  Figure 15. Histograms of three algorithms recognition rate with the increase of plant species.
As shown in Table 1, the recognition rate of LBP features is 71.03%, increasing to 84.22% after reducing the dimensionality. When combined with Fourier features, recognition rate is up to 91.71%. And the low-dimensional LBP features greatly reduce recognition time, since the dimensionality of the Fourier descriptors is low, it has little effect on recognition speed. Figure 15 shows that the recognition rate of the three algorithms is reduced with the increase of plant species. However the recognition rate of the combination of LLE, LBP and Fourier descriptors is still above 90%. Therefore, the proposed method can achieve good results both in recognition rate and recognition time.

Conclusion
In this paper, an efficient framework is proposed to realize the leaf recognition in complicated background. We proposed a watershed segmentation algorithm based on iterative opening and closing reconstruction to segment leaves from complicated background. To improve the recognition rate of leaves, the block LBP operators and Fourier descriptors are extracted as leaf texture and shape features respectively after segmentation. In addition, the dimensionality of block LBP operators is reduced by LLE algorithm to enhance the efficiency of the recognition. Finally, the shape and low-dimensional texture features are combined to be classified by SVM. Experimental results show that segmentation algorithm and recognition method proposed in this paper are effective. The future research works will focus on how to eliminate the influences of surrounding factors.