{'fraction_non_alphanumeric': 0.031444041527991656, 'fraction_numerical': 0.01698872385872535, 'mean_word_length': 4.912481644640235, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluation of random forest method for agricultural crop classification

Abstract This study aims to examine the performance of Random Forest (RF) and Maximum Likelihood Classification (MLC) method to crop classification through pixel-based and parcel-based approaches. Analyses are performed on multispectral SPOT 5 image. First, the SPOT 5 image is classified using the classification methods in pixel-based manner. Next, the produced thematic maps are overlaid with the original agricultural parcels and the frequencies of the pixels within the parcels are computed. Then, the majority of the pixels are assigned as class label to the parcels. Results indicate that the overall accuracies of the parcel-based approach computed for the Random Forest method is 85.89%, which is about 8% better than the corresponding result of MLC.


Introduction
With the availability of satellite sensor technology, thematic maps become a very attractive tool to the researchers, to understand the land use/cover characteristics of the earth's surface. In agricultural perspective, the most interesting applications include classifying agricultural landscapes [Duro et al., 2012], determining land use areas [Huang and Fipps, 2006], soil moisture estimation [Sajjad, 2010], and revealing crop residues in agricultural fields which is an important consideration to reduce soil erosion [Daughtry et al., 2006], estimating the effect of crop classification error [Stehman and Milliken, 2007], land cover mapping in an agricultural setting using classification [Oetter et al., 2000]. In this light, various image classification algorithms are developed to satisfy the needs of different applications and classification problems. One of the most widely used algorithms is pixel-based Maximum Likelihood Classification (MLC) algorithm. It is also known as a statistical approach since it relies on a statistical model. In statistical approaches, an appropriate data model is used and parameters of the model can be approximated from the data [Elith et al., 2008;Horning, 2010]. To assign each pixel in an image to a class represented in the signature file, the MLC method assumes that the image data for each class in each band is normally distributed. In the MLC procedure, the probability that a given pixel belongs to a specific class is calculated and each pixel is assigned to the class that has the highest probability [ENVI, 2005]. Through the MLC, pixels in an image domain are automatically categorized into pre-determined land cover classes according to their spectral characteristics. The method uses only spectral information and spatial contexts are ignored. On the other hand, pixels need to be considered as groups based on their spatial characteristics to provide more reliable results. Hence object-based classification methods are introduced. The object based classification methods use different object features such as spectral values, shape, and texture, and it can be performed by using both image object attributes and the relationship among different image objects [Gitas et al., 2004]. A simple way of applying the objectbased methods is parcel-based classification approach that is carried out by integrating satellite image and vector boundary data [e.g. Berberoglu et al., 2000;Aplin and Atkinson, 2001;Smith and Fuller, 2001;Turker and Ozdarici, 2011]. The method can be performed on the images in three stages: (i) before classification, (ii) during classification, and (iii) after classification [Hutchinson, 1982]. In the parcel-based approach, first, the images are classified in pixel-based manner and the classified output is then integrated with the vector boundary. After computing the statistics within the parcels, a class label is assigned based on the modal class. In this way, the within field internal spectral variability is minimized. To automate the parcel-based classification method, multiple object-based image algorithms are developed in recent years. These algorithms delineate the pixels as groups based on their texture and contextual characteristics [e.g. Chen et al., 2006;Lee and Warner, 2006;Xiao et al., 2010;Dronova et al., 2011]. As an alternative to the traditional pixelbased and object-based approaches, in recent years, multiple learning based algorithms have been developed to obtain more accurate and more reliable information from satellite image classification. The most widely used learning based algorithms are RF, bagging, boosting, decision tree, artificial neural network, supported vector machine and k-nearestneighbour. These algorithms are known as machine learning methods. Contrary to the statistical approaches, machine learning methods are non-parametric since they do not rely on any assumption about data distribution. These methods are data driven and they learn the relationship between predictor and response data [Breiman, 2001;Horning, 2010]. Using sufficient size data set and parameters, the machine learning methods can automatically extract rules and restrictions to find the best model for the new data using decision rules created from input data. As a machine learning based classifier, the RF classification algorithm is superior to many tree-based algorithms since it is not sensitive to noise and is not subject to overfitting [Watts and Lawrence, 2008]. Jay et al. [2009] stated that both complex and homogeneous plant groups are classified successfully with an overall classification accuracy of 88.37% by utilizing RF method. Cutler et al. [2007] classify invasive plant species by RF using ecological data. There are also many studies to test the performance of RF classifier by comparing the classification results with the ones obtained from other classification algorithms. Waske and Braun [2009] classify temporal SAR images using learning based methods (RF and boosting) and MLC. The study shows that learning based methods give higher classification accuracy (around 10%) than MLC. Prasad et al. [2006] generate plant cover maps for the four species using Regression Tree Analysis (RTA), RF, Bagging Trees (BT) and Multivariate Adaptive Regression Splines (MARS). He compares these four methods by assessing the outputs through multiple statistical evaluation indicators: correlation, Kappa, and its variants, variable importance, and the output maps. Results show that BT and RF are superior to other methods, yet the RF gives slightly better performance. This study examines the RF method to classify major crop types of Karacabey Plain and it compares the results with the MLC method in pixel-based and parcel-based approaches. First, in methodological, we give a brief literature survey about RF method followed by the description of study area and data. Next, we examine the RF method followed by the description of a quantitative evaluation method. Then, we provide the results computed for the RF and MLC methods. In the last section of the paper some conclusions of the study are presented.

Study Area and Data
Study area is selected from Karacabey Plain (Bursa), which is located in Marmara region in northwest of Turkey (Fig. 1). The geographic boundaries of the area are E 28º10′18″ -E 28º18′06″ and N 40º13′43″ -N 40º08′37″. The test site covers an agricultural land approximately 95 km 2 . Karacabey Plain is a representative region of agricultural structure and characterized by rich, loamy soils having good weather conditions. Due to these characteristics, it is one of the most productive and valuable agricultural regions of Turkey. In this study only the main crop types namely; corn, tomato, pepper, wheat, sugar beet and rice are classified. In the study area, the same crop type in different fields has different spectral characteristics due to different plant growth stages. To guarantee that the training areas representing each crop type have unimodal distribution, which means that the reflectance values for training pixels comes from one distinct type of land feature, sub-classes of the agricultural crops are separately characterized as two wheat types, two rice sub-classes, three corn sub-classes, two sugar beet types, two tomato, and two pepper sub-classes. In this way, we prevent possible multi-model distribution and provide more representative training samples to the classification algorithms. After the classifications, due to the spectral overlaps, the classes of tomato and pepper are merged, forming Tomato/Pepper class in the analyses. SPOT5 multispectral (10 m) image used in the classification was acquired on July 24, 2004 in a clear sky condition and of good quality. The image has four spectral bands; green (0.50-0.59 µm), red (0.61-0.68 µm), NIR (0.79-0.89 µm), and SWIR (1.58-1.75 µm). In addition to the SPOT 5 image, a vector data containing boundaries of agricultural parcels are used in the analyses as well. The vector data is produced with the 1:5000 cadastral maps of the study area [Turker and Arikan, 2005]. It includes crop information such as; crop type, crop moisture, bare soil conditions and weeds, etc. that was recorded during the image acquisition. To geometrically correct the SPOT 5 image, the vector data is utilized as a reference source, and the second order polynomial transform and the nearest neighbor (NN) resampling techniques are used. The projection system used was Universal Transverse Mercator (UTM). After defining the geometric correction parameters, in total 20 Ground Control Points (GCP) is carefully selected from the distinct locations (e.g. intersections of the roads) for the reference data and the uncorrected image. GCPs are collected uniformly throughout the entire scene to ensure an even distribution. The SPOT 5 image is geometrically corrected using the second order polynomial and the nearest neighbor (NN) resampling techniques. The Root Mean Square Error (RMSE) is computed as ±0.41 pixels and it is observed that the result of geometric correction is quite satisfactory to perform the image classification analysis [Turker and Ozdarici, 2011].

Random Forest
Ensemble classification methods are learning algorithms that construct a set of classifiers instead of one classifier, and then classify new data points by taking a vote of their predictions. Recent ensemble classifiers are bagging, boosting and RF. In bagging algorithm, many bootstrap samples are drawn from a training data set with replacement to learn a classifier and a tree is constructed for each bootstrapped sample such that successive trees are constructed independently from earlier trees, and a simple majority vote is taken for prediction [Liaw and Wiener, 2002]. On the other hand, boosting uses iterative re-training, and the weights of incorrectly classified samples are increased as the iterations progress to make them more important in the next iterations. Boosting generally reduces both the variance and the bias of the classification, and in most cases it is considerably more accurate than bagging; however, it has some disadvantages. It is slow, it can over-train and it is sensitive to noise [Gislason et al., 2006]. RF classifier can be described as the collection of tree-structured classifiers. It is an advanced version of bagging [Breiman, 2001] such that randomness is added to it. Instead of splitting each node using the best split among all variables, RF splits each node using the best among a subset of predictors randomly chosen at that node. A new training data set is created from the original data set with replacement. Then, a tree is grown using random feature selection. Grown trees are not pruned [Archer, 2008;Beriman, 2001]. This strategy makes RF unexcelled in accuracy [Breiman and Cutler, 2005] when compared to other existing algorithms including discriminant analysis, support vector machines and neural networks [Liaw and Wiener, 2002]. RF is also very fast, it is robust against over fitting, and it is possible to form as many trees as the user wants needs [Breiman and Cutler, 2005]. Two parameters must be defined by user to initialize RF algorithm. These parameters are N and m, which are the number of trees to grow and the number of variables used to split each node, respectively. First, N bootstrap samples are drawn from the 2/3 of the training data set. Remaining 1/3 of the training data, also called out-of-bag (OOB) data, are used to test the error of the predictions. Then, an un-pruned tree from each bootstrap sample is grown such that at each node m predictors are randomly selected as a subset of predictor variables, and the best split from among those variables is chosen. It is crucial to select the number of variables that provides sufficiently low correlation with adequate predictive power [Horning, 2010]. Breiman [2002] suggests that setting number of variables (m) equal to the square root of M (number of overall variable) gives generally near optimum results. RF uses Classification and Regression Tree (CART) algorithm to create the trees [Beriman 2001]. At each node, split is performed according to a criterion (e.g. GINI index) in CART algorithm. In this study, GINI index is utilized to perform the split. The GINI index measures class homogeneity and can be written as the equation below [1]: , where T is a given training set, C i is the class that a randomly selected pixel belongs to, and , T f C T î h is the probability that the selected case belongs to class C i [Pal, 2005]. As GINI index increases class heterogeneity also increases. On the other hand, a drop in GINI index increases class homogeneity. If a child node of GINI index is less than a parent node, then the split is successful. Tree splitting is terminated when GINI index is zero, which means only one class is present at each terminal node [Watts et al., 2011]. Once all N trees are grown in the forest, the new data is predicted based on the outcome of the predictions of N trees [Liaw and Wiener, 2002]. RF algorithm explained above works for image classification as follows. Suppose N is chosen as 1000. Algorithm generates 1000 trees that mean 1000 different classification result for a particular pixel. Suppose that pixel is classified as forest in 800 trees, the pixel is classified as land in 100 trees and pixel is assigned to water class in 100 trees. In this case, the predicted output for this pixel will be forest.

Pixel-based and Parcel-based Approach
After classifying the MS SPOT 5 image with the RF and MLC methods using the same training samples, the evaluation of the thematic maps is performed by two different approaches: one is the traditional evaluation method, pixel-based, and the other is parcelbased approach. In traditional pixel-based image classifications, each pixel in an image are automatically categorized into land cover classes based on the spectral characteristics [Lillesand et al., 2004].On the other hand, in the case of agricultural applications, pixelbased classification methods may cause problems due, for example, to the variations in soil moisture conditions, nutrient limitations or pests and diseases. The other problem may be due to the mixed pixels that are located at the boundary of two or more land cover types. Hence, when the classification is performed on per-pixel basis, these factors may cause to assign a combination of the reflectance from two or more land cover types, which causes misclassification [Smith and Fuller, 2001]. The basic idea behind a parcel-based classification is that the image is divided into homogenous objects using the knowledge of agricultural field boundaries. With regard to crop classification, this means that the location and the extent of each field are known. During classification, each pixel is assigned to a final class of the entire object according to their statistical properties, instead of determining the class label for each pixel separately. Thus, besides the pixel-based approach, the classification results are evaluated through the parcel specific manner as well, in this study. In parcel-based approach, first, a per-pixel MLC and RF are performed on the MS image, and the frequency of the classified pixels is computed for each parcel. Then, the class labels of the parcels are assigned based on the majority class computed.

Results and Discussion
Classification accuracy of RF method depends on user-defined parameters as N and m. The process of parameter selection directly affects the classification performance. Hence, multiple parameter combinations (N and m) are tested and assessed to the RF method to obtain more reliable crop maps of the study area (Tab. 1). The results obtained for the training set, OOB error, test accuracy, kappa and computational time of the selected combinations are given in Table 1. The parameter combinations of N = 200 and m = 2 provide the most effective performance in this study. The resulting thematic maps are evaluated using one of the most very widely used methods, error matrix. The error matrix is used compute the relationship between the known reference data (ground truth) and the corresponding results of an automated classification on category-by-category basis [Lillesand, 2004]. Approximately one third of the reference data (a total of 1021 sample fields), homogenously distributed, are utilized to assess the accuracy. The accuracies of the thematic maps are tested using stratified random sampling method since the crop types and also the reference data do not have a uniform distribution in the study area. Thus the random samples are scattered to the reference data based on the class percentages to represent each class effectively in the evaluation process. During the sampling process, the training areas are excluded so that no bias is included in the assessment. The random points are scattered to the reference fields based on the class percentages. A set of 570 samples is determined based on the equation below [2].
where N refers to the sample size π i is the proportion of a population in the i th class out of k classes that has the proportion closest to 50%, b i is the desired precession for this class (e.g. 5%), B explains the upper ( k a ) x 100th percentile of the chi square ( X 2 R ) distribution with 1 degree of freedom, k is the number of classes. The same training samples are utilized in the validation process of the produced thematic maps. Table 2 indicates the pixel-based results computed for the MS SPOT 5 image. Based on the results, the overall accuracies computed for the RF and MLC method are found as 76.15% and 72.55%, respectively. The RF method improves the producer's accuracy of the class wheat and class corn around 5% and 17%, respectively. When the user's accuracies are examined, similar results are obtained for both methods except for the class tomato/ pepper. A significant improvement around 9% is observed for the class tomato/pepper when the RF method is used.  Figure 2. The other crop maps classified utilizing different parameter combinations of the RF method have also similar accuracies with the result obtained for the optimum parameters. Some of the results computed for the parcel-based approach are provided in Table 4.

Conclusion
This study examines the performance of the RF and MLC methods by pixel-based and parcelbased approaches. Pixel-based classification results indicate that the RF method improves the overall accuracy of the MLC method around 4% and it is computed as 76.15%. The parcel-based classification results also reveal that the highest overall accuracy of 85.89% is obtained for the RF method, which is higher 8% than the corresponding parcel-based MLC result. These promising results can be explained by the well-built algorithm of the RF method along with the parcelbased post-classification strategy. It is also worthy noticed that similar results are achieved when different parameter combinations are tested for the RF method, which indicates the consistency of the RF classification algorithm. In conclusion, it can be stated that the RF method along with the parcel-based approach can be a reliable way to produce crop maps in high accuracies for agricultural lands.