{'fraction_non_alphanumeric': 0.034437304333498106, 'fraction_numerical': 0.007332344702586917, 'mean_word_length': 4.512715712988193, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Method for Image Co-segmentation with Seeded Foreground

Image segmentation is a key technology from image processing to analysis. Without proper segmentation, it is impossible to recognize correctly. In this paper, we propose a method for image co-segmentation based on the biased normalized cuts using a semi-supervised way to deal with foreground regions. In order to take advantage of biased normalized cuts to solve problem, we use 2D adaptive Wiener filter to smooth the seeded parts of images, then divide images into a set of super-pixels, after that take super-pixels as vertices to form a weighted undirected graph. Thus, the co-segmentation can be seen as an issue of graph partition that solved by biased normalized cuts. The experiments on image data sets show the superior performance of our method.


Introduction
Image segmentation is an important preprocessing for machine vision and image recognition. In general, people are only interested in foreground region of images. So, the image segmentation is very significant. It can divide the image into regions with different properties and extract the target corresponding to user's interest. In images, the parts that people interested are often called foregrounds.
Compared with single image segmentation, co-segmentation can segment a series of images for the similar targets simultaneously. The similar targets can be described as follows: in images, different objects belonging to the same type or an object observed from different directions. The cosegmentation can take advantage of the similarity between multiple foregrounds [1]. So, in contrast to the single ones, co-segmentation can get more information between the images and obtain more accurate results. Co-segmentation is widely used in image retrieval, object tracking, human motion capture, large-scale object recognition, and so on [2].
In our work, it is only need to mark foreground in one image, and then the method deals with a group of images automatically. The number of images is arbitrary, including only one image. In order to focus on a local region, a scrawl is used to mark the region of interest. 2D adaptive Wiener filter is served to smooth the seeded region and keep the region edge clear. After that, divide the images into super-pixel blocks and consider the blocks as vertices. So the images are changed into a graph with weighted and undirected edges. Now, the co-segmentation can be processed as an issue of graph partition and can be calculated by biased normalized cuts. Word "biased" here expresses that the method focuses on a part of images. The scrawl is used to mark the foreground, and it is the only interaction in our method. Biased normalized cuts can do a good job in this situation. The results show This paper is organized as follows: Section 2 describes related works on unsupervised and supervised co-segmentation. Section 3 introduces our seeded foreground co-segmentation method. In Section 4, we show the experiment results of our method. Finally, a conclusion of this article is presented in Section 5.

Related work
In recent years, the methods of co-segmentation have attracted wide attention. Rother et al. [3] proposed the first co-segmentation method, which is used to segment a pair of images with similar foreground. It utilizes the traditional Markov Random Fields (MRF) model and the penalty term of foreground region. Mu and Zhou [4] extended Rother's method by increasing a quadratic global constraint. Mukherjee et al. [5] proposed an algorithm based on the half-integrality. Hochbaum and Singh [6] improved the color histogram feature constraint, and used maximum flow algorithm based on graph theory. Vicente et al. [7] presented a dual decomposition algorithm, after that a cosegmentation algorithm based on the target object [8] was proposed. The method [8] divided the image into some local regions and extracted the common area from the regions. It used co-saliency map [9] as prior knowledge adding to the traditional MRF energy function. Then it optimized the method by graph-cuts. Methods [3][4][5][6][7][8][9] belong to unsupervised co-segmentation methods. This kind of methods have two drawbacks. The first one is that if the background of images are similar, in other words, the images are overall similarity, the methods will consider the background as foreground and can not work well. The second is that these methods are commonly used to deal with a pair of images, so they can not handle more images simultaneously.
Schnitman et al. [10] proposed a method that first segmented one image from a data set, and then generated classifiers to segment other images. Joulin et al. [11] used discriminative clustering method to achieve the combination of unsupervised discriminant and supervised classifier. Batra et al. [12] proposed an interactive co-segmentation method which need scribbles to label the images. Methods [10][11][12] belong to supervised co-segmentation methods. In contrast to the unsupervised ones, these ones are more complex, but can deal with more pictures and obtain better results.

Method of seeded foreground co-segmentation
Suppose there are n images, and each contains m pixels. The pixels are regarded as vertices and there are edges between vertices, then the images can be changed into a connected graph [13]. As a result, the co-segmentation can be considered as an issue of graph partition. However, with the number of images increases, the required storage space grows exponentially. This shortcoming limits the number of images in co-segmentation method. To break this constraint, we cluster the pixels into several groups. Each group is seen as a super-pixel [14], and then super-pixels are used as the vertices.

Image smoothing and generating super-pixels
In an image, one super-pixel is a small area which is composed of a series of adjacent pixels with similar position, color, brightness, texture and so on. In order to make a super-pixel belongs to either foreground or background, we take advantage of the seeded region to smooth the foreground. In this work, there is a manual interaction which draws a scrawl in one image to mark the interesting region. After that, use simple linear iterative clustering method [14] to put the regions together, and then use a two dimensional adaptive Wiener filter to smooth the region and keep the edge clear. Using superpixels instead of image pixels can reduce the number of vertices greatly. Besides, the method that deals with super-pixels is insensitive to noise.
Assuming that the size of each super-pixel is approximately the same and the size is roughly equal to m s , so an image has about m/m s super-pixels and the distance between two neighboring super-pixels is about:

Graph model
In a graph, there are vertices and edges. Super-pixels are vertices, and the relationship between two super-pixels can be quantized as a numerical value that treated as the weight of edge.
For two super-pixels, they may come from one image or two different images. For the first case, in one image, if two super-pixels are close to each other, they will be closely related. In other words, the corresponding edge has a large weight. In contrary, the edge will have a small weight. Between two super-pixels, when the distance D is larger than 3d (see formula 1), the relationship is considered small and the corresponding edge can be ignored. For the other case, two super-pixels come from different images, we set an edge between every two super-pixels because of requiring more information. If there are more edges, the corresponding matrices of the graph will be more accurate, too.
For two super-pixels s a and s b , the histogram of HSV color space is used to measure the distance and MR8fast texture [15] is the feature vector. Define d(s a , s b ) as: where T(n) represents the n-th feature. When there exists an edge between s a and s b , the weight w ab is: where δ is a constant. When there is no edge, the value of w ab is set to 0.

Biased normalized cuts
where k is a constant that keeps the angle between vector x and vector u in a certain range. In order to calculate the solution for formula 8, we combine the optimal result of normalized cuts [16] with vector u. Then the optimal solution of our biased normalized cuts is:  , λ i (i=2, ..., K) is the i-th smallest eigenvalue of matrix L, K is the amount of eigenvalues, h i represents the eigenvector corresponding to λ i , g i represents the weight for each eigenvector, and γ∈(-∞, λ 2 ).

Experiments and results
In order to check the performance, we do experiments on the iCoseg data set [12]. The data set contains 38 groups of images. For a group, each image contains one or more objects.
We first compare our approach with a basic method that using eigenvectors of the Laplacian matrix. The basic method is for the whole image, and does not sensitive to any local parts. An experiment is done on the "goose" image set. There are 10 images with a resolution of 500×333 pixels. The results are shown in figure 1 with NO. 1, 3, 6, 9 images. From figure 1(b) to (d), the results are respected to the second, third and fourth eigenvectors. Among these results, it is difficult to find a set of useful one. In our work, we roughly mark the area of goose in the first original image. The red scrawl is the input seed, which is used to let the approach focus on the parts of goose. Figure 1  The second experiment is to compare our approach with Meng's co-segmentation method [17]. Meng's method uses saliency information and utilizes the shortest path to solve the problem. Experiment results of method [17] and our approach are shown in figure 2. The first three lines are about "skating" data set. The first line is original images, the second line is Meng's results and the third line is ours (next three lines are also arranged in this way). For the outcomes in [17], the second and fifth results each contain a big piece of background, which leads to bad consequences. While, our results do not have this drawback. The last three lines are about "helicopter" data set. Results in [17] can not segment the propeller of helicopter, which is a shortcoming. In the fourth and fifth results, for the fuselage of helicopter, method in [17] are better than ours. Our shortcoming is that regarding several small pieces of background as the foreground.
In order to quantify the experiment results, we use correct rate and error rate to compare the two co-segmentation results with the ground truth. Two quantitative indicators are described as follows: where N(*) represents the amount of pixels, and "total" is all of the pixels in the image. In Q x y , x belongs to either background B or foreground F which is judged by the co-segmentation results, and y is the ground truth. Formula 12 represents the correct rate that dividing the correct number of points by the total number, and formula 13 represents the error rate that dividing the wrong number of points by the total number. Quantitative indicators for "goose", "skating" and "helicopter" data sets are shown in table 1. From the quantitative data, only in "goose" image set, the result of [17] is a little better than ours. However, for another two image sets and the average consequences, ours are better.

Conclusion
In this paper, we presented an interactive seeded foreground method for image co-segmentation. The only simple interaction is to mark the interesting parts with a scrawl, in other words, mark out the seeded parts. In order to smooth the seeded region and keep the edge of the region clearly, we use the two dimensional adaptive Wiener filter to smooth the seeded parts in all the images. The smooth process can be conducive to get a good result of super-pixels. And super-pixels change the cosegmentation into a graph partition issue. In the graph, the seeded parts help the method focus on the interesting region. At last, the biased normalized cuts calculate the result of graph partition and segment the foreground region. Experimental consequences on several image sets show the superiority of the proposed method.