{'fraction_non_alphanumeric': 0.05261783522653088, 'fraction_numerical': 0.01749575662619141, 'mean_word_length': 4.32464372610358, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Shape-Color Differential Moment Invariants under Affine Transformations

We propose the general construction formula of shape-color primitives by using partial differentials of each color channel in this paper. By using all kinds of shape-color primitives, shape-color differential moment invariants can be constructed very easily, which are invariant to the shape affine and color affine transforms. 50 instances of SCDMIs are obtained finally. In experiments, several commonly used color descriptors and SCDMIs are used in image classification and retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs get better results.


Introduction
Image classification and retrieval for color images are two hotspots in pattern recognition. How to extract effective features, which are robust to color variations caused by the changes in the outdoor environment and geometric deformations caused by viewpoint changes, is the key issue. The classical approach is to construct invariant features for color images. Moment invariants are widely used invariant features.
Moment invariants were first proposed by Hu [1] in 1962. He defined geometric moments and constructed 7 geometric moment invariants which were invariant under the similarity transform(rotation, scale and translation). Researchers applied Hu moments to many fields of pattern recognition and achieved good results [2,3]. Nearly 30 years later, Flusser et al. [4] constructed the affine moment invariants (AMIs) which are invariant under the affine transform. The geometric deformations of an object, which are caused by the viewpoint changes, can be represented by the projective transforms. However, general projective transforms are complex nonlinear transformations. So, it's difficult to construct projective moment invariants. When the distance between the camera and the object is much larger than the size of the object itself, the geometric deformations can be approximated by the affine transform. AMIs have been used in many practical Student is the first author. arXiv:1706.04382v1 [cs.CV] 14 Jun 2017 applications, such as character recognition [5] and expression recognition [6]. In order to obtain more AMIs, researchers designed all kinds of methods. Suk et al. [7] proposed graph method which can be used to construct AMIs of arbitrary orders and degrees. Xu et al. [8] proposed the concept of geometric primitives, including distance, area and volume. AMIs can be constructed by using various geometric primitives. This method made the construction of moment invariants have intuitive geometric meaning.
The above-mentioned moment invariants are all designed for gray images. With the popularity of color images, the moment invariants for color images began to appear gradually. Researchers wanted to construct moment invariants which are not only invariant under the geometric deformations but also invariant under the changes of color space. Geusebroek et al. [9] proved that the affine transform model was the best linear model to simulate changes in color resulting from changes in the outdoor environment. Mindru et al. [10] proposed moment invariants which were invariant under the shape affine transform and the color diagonal-offset transform. The invariants were constructed by using the related concepts of Lie group. Some complex partial differential equations had to be solved. Thus, the number of them was limited and difficult to be generalized. Also, Suk et al. [11] put forward affine moment invariants for color images by combining all color channels. But this approach was not intuitive and did not work well for the color affine transform. To solve these problems, Gong et al. [12,13,14] constructed the color primitive by using the concept of geometric primitive proposed in [8]. Combining the color primitive with some shape primitives, moment invariants that are invariant under the shape affine and color affine transforms can be constructed easily, which were named shape-color affine moment invariants(SCAMIs). In [14], they obtained 25 SCAMIs which satisfied the independency of the functions. However, we find that a large number of SCAMIs with simple structures and good properties are missed in [14].
In this paper, we propose the general construction formula of shape-color primitives by using partial differentials of each color channel. Then, we use two kinds of shape-color primitives to construct shape-color differential moment invariants(SCDMIs), which are invariant under the shape affine and color affine transforms. We find that the construction formula of SCAMIs proposed in [14] is a special case of our method. Finally, commonly used image descriptors and SCDMIs are used for image classification and retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs proposed in this paper get better results.

Related Work
In order to construct image features which are robust to color variations and geometric deformations, researchers have made various attempts. Among them, SCAMIs proposed in [14] are worthy of special attention. SCAMIs are invariant under the shape affine and color affine transforms. Two kinds of affine transforms are defined by (2) where SA and CA are nonsingular matrices.
For the color image I(R(x, y), G(x, y), B(x, y)), let (x p , y p ), (x q , y q ), (x r , y r ) be three arbitrary points in the domain of I. The shape primitive and the color primitive are defined by whereĀ represents the mean value of A, A ∈ {x, y, R, G, B}. Then, using Eq. (3) and (4), the shape core can be defined by where n and m represent that the sCore is the product of m shape primitives which are constructed by N points (x 1 , y 1 ), (x 2 , y 2 ), ..., (x n , y n ). k < l, r < n, k, l, r ∈ {1, 2, ..
Where N and M represent that the cCore is the product of M color primitives which are constructed by N points (x 1 , y 1 ), (x 2 , y 2 ), ..., (x N , y N ). G < K < L, y ), B (x , y )) by two transformations defined by Eq. (1) and (2), (x p , y p ), (x q , y q ), (x r , y r ) in I are the corresponding points of (x p , y p ), (x q , y q ), (x r , y r ) in I. Gong et al. [14] have proved C (p, q, r) = |CA| · C(p, q, r) Further results can be concluded sCore (n, m; d 1 , d 2 , ..., d n ) = |SA| m · sCore(n, m; d 1 , d 2 , ..., d n ) (9) cCore ( Then there is a relation It must be said that max{n, N }, max i {d i } and max i {D i } are named the degree, the shape order and the color order of SCAMIs, respectively. In fact, Eq. (11) can be expressed as polynomial of shape-color moment. This moment was first proposed in [16] and defined by Gong et al. [14] proposed that they constructed all SCAMIs of which degrees 4, shape orders 4 and color orders 2. They obtained 24 SCAMIs which are functional independencies using the method proposed by Brown [17]. However, we will point out in the Section 3 that they omitted many simple and wellbehaved SCAMIs.

The Construction Framework of SCDMIs
In this section, we introduce the general definitions of shape-color differential moment and shape-color primitive, firstly. Then, using the shape-color primitive, the shape-color core can be constructed. Finally, according to Eq.(11) and the shape-color core, we obtain the general construction formula of SCDMIs. Also, 50 instances of SCDMIs are given for experiments in the Section 4.

The Definition of The General Shape-Color Moment
Suppose the color image I(R(x, y), G(x, y), B(x, y)) have the k-order partial derivatives (k = 0, 1, 2, ...). The general shape-color differential moment is defined by where (R (k) (x, y), G (k) (x, y), B (k) (x, y)) represent the k-order partial derivatives of (R(x, y), G(x, y), B(x, y)).R,Ḡ,B represent the mean values of R, G, B. δ(k) is the impact function which is defined by We can find that Eq. (14) and Eq.(13) are identical, when k = 0. Therefore, the shape-color moment is a special case of the general shape-color differential moment.

The Construction of The General Shape-Color Primitive
Suppose the color image I(R(x, y), G(x, y), B(x, y)) have the k-order partial derivatives (k = 0, 1, 2, ...). (x p , y p ), (x q , y q ), (x r , y r ) are three arbitrary points in the domain of I. The general shape-color primitive is defined by where We can find that C(p, q, r) defined by Eq.(4) is a special case of SCP k (p, q, r), when k = 0.

The Construction of The General Shape-Color Core
Definition 3.
Using Definition 2, the general shape-color core is defined by

The Construction of SCDMIs
Theorem 1.
Let the color image I (R(x, y), G(x, y), B(x, y)) be transformed into the image I (R (x , y ), G (x , y ), B (x , y )) by Eq.(1) and Eq. (2), (x p , y p ), (x q , y q ), (x r , y r ) in I are corresponding points of (x p , y p ), (x q , y q ), (x r , y r ) in I, respectively. Suppose that R(x, y), G(x, y), B(x, y), R (x , y ), G (x , y ), B (x , y ) have the korder partial derivatives (k = 0, 1, 2, ...). Then there is a relation where Further, the following relation can be obtained Eq.(23) can be expressed as polynomial of SCM k pqαβγ . Eq.(11) is a special case of Eq.(23) when k = 0. The proof of Eq.(24) is exactly the same as that of Eq.(12) proposed in [14].

The Instances of SCDMIs
We can use Eq.(24) to construct instances of SCDM Is by setting different k values. However, color images are discrete data, the partial derivatives of each order can't be accurately calculated. With the elevation of order, the error will be more and more, which will greatly affect the stability of SCDMIs. So, we set k = 0, 1 in this paper.
At the same time, when k = 1, SCP 1 (p, q, r) is defined by where The construction methods of 50 instances are shown in Table 1. In order to more clearly explain that SCDMIs can be expanded into the polynomial of SCM k pqαβγ , we give the shape-color moment polynomial of SCM Is 3 0 which is represented by SCDM Is 17 0 /SCDM Is 17

Experimental Results
In this section, some experiments are designed to evaluate the performance of SCDMI50. Firstly, we verify the stability and discriminability of SCDMI50 by using synthetic images. Then, some retrieval experiments based on real image databases are performed. Also, we chose some commonly used image descriptors for comparison. It is worth noting that we have to choose the method to calculate the first order partial differentials of R(x, y), G(x, y) and B(x, y). The 5 points difference formulas are used for approximating the first partial derivatives of discrete color images, which are defined by where C ∈ {R, G, B}. We choose this method because it guarantees the computational accuracy of the first partial differential to a certain extent and also maintains a relatively fast calculation speed.

The Stability and Discriminability of SCDMI50
We select 50 different kinds of butterfly images, which are shown in the Fig.(1.a). Then, 5 shape affine transforms and 4 color affine transforms are applied to each image. So, one image can get 20 transformed versions which are shown in the Fig.(1.b). Thus, we obtain the database containing 1000 images. 10% images are used as the training data and the rest make up the testing data.  For comparison with SCDM I50, we chose some commonly used features of gray images or color images.
-Hu moments, which were composed of 7 invariants under the shape similarity transform, proposed in [1]. Hu moments were designed for gray images. -AMIs which were composed of 17 invariants under the shape affine transform proposed in [18]. AMIs were designed for gray images. -RGhistogram, which consisted of 60-dimensional features and was scaleinvariant in color space, proposed in [19]. -Transformed color distribution, which consisted of 60-dimensional features, was proposed in [19]. Transformed color distribution was scale-invariant and sift-invariant in color space. -Color moments consisted of the first, second and third geometric moments of each channel of the color image, which were proposed in [15]. Color moments are sift-invariant in color space. -GPSOs were invariant under the shape affine transforms and the color diagonal-offset transform, which were proposed in [10]. GPSOs consisted of 21 moment invariants.
Subsequently, image classification is performed on the butterfly database by using different kind of features. The Chi-Square distance is selected to measure the similarity of two images. Finally, we plot the classification accuracies obtained by using different features, which are shown in the Table 2. On the one hand, we can find that the result obtained by using SCDM I50 is better than those obtained by using other features. And, color information is very important for the classification. On the other hand, the Table 2 show that SCDM I50 have good stability and distinguishability for the shape affine and color affine transforms, and demonstrate that the construction formula of SCDMIs designed in the Section 3 is correct.

Image Retrieval on Real Image Database
In order to further illustrate the performance of SCDM I50 better than the commonly used features listed in the Section 4.1, the database COIL-100 proposed in [20] is chosen for our experiment. COIL-100 contains 7202 images of 100 categories, each of which has 72 images taken from different angles. We choose 30 classes, each class contains 5 images. For each image, 6 color affine transformations are applied. Thus, 900 images of 30 categories are obtained, each of categories contains 30 images which are shown in the Fig.(2).  Obviously, the result obtained by using SCDMI50 is far superior to those obtained by using other features. This is because that traditional image features are constructed by using only color information or shape information. However, we use two kinds of information while constructing SCDMI50. In addition, traditional image descriptors are only stable for simple changes in color space, such as the diagonal-offset transforms. When the color space changes drastically, they are less robust.
Finally, in order to observe the performance of SCDM Is 0 and SCDM Is 1 clearly, we compare the retrieval results of SCDM I50, SCDM I 0 25 and SCDM I 1 25 which are shown in the Fig.(4) Because of the error caused by the inaccuracy of partial derivative calculation, the result obtained by using SCDM I 1 25 slightly worse than the result obtained SCDM I 0 25, but far better than the results obtained by other commonly used features. Meanwhile, SCDM I 1 25 increase the number of invariants which have simple structures and good properties. We combine SCDM I 1 25 and SCDM I 0 25 to get SCDM I50 which achieve the best retrieval result in our experiment.

Conclusion
In this paper, we proposed a kind of shape-color differential moment invariants (SCDMIs) of color images, which are invariant under the shape affine and color affine transforms, by using partial derivatives of each color channel. It is obvious that all SCAMIs proposed in [14] are the special cases of SCDMIs when k = 0. Then, we correct a mistake in [14] and obtain 50 instances of SCDMIs, which have simple structures and good properties. Finally, several commonly used image descriptors and SCDMIs are used in image retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs get better results.