{'fraction_non_alphanumeric': 0.03807743171143235, 'fraction_numerical': 0.013589310958092024, 'mean_word_length': 4.696103896103896, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Dual-Polarization Information Guided Network for SAR Ship Classification

How to fully utilize polarization to enhance synthetic aperture radar (SAR) ship classification remains an unresolved issue. Thus, we propose a dual-polarization information guided network (DPIG-Net) to solve it.


I. INTRODUCTION
HIP classification plays an important role in ocean surveillance. It can offer rich traffic information, helpful for trade developments. Synthetic aperture radar (SAR) is an active microwave sensor. Its operation is often unhindered by both light and weather, so it is very suitable for marine ship classification. Nowadays, SAR ship classification is receiving much attention.
Similar to SAR automatic target recognition (ATR) methods [1]- [5] designed for vehicle targets, traditional ship classification methods [6]- [11] focus on feature extraction based on experience, but the manual process is labor-and time-consuming. Modern deep learning methods [12]- [21] are receiving more attention. For example, Hou et al. [12] designed a simple convolutional neural network to classify ships in Gaofen-3 images. Huang et al. [13] proposed a group squeeze excitation sparsely connected convolutional network to extract robust ship features. Wang et al. [14] studied transfer learning to solve few-shot ship classification problem. Wang et al. [15] proposed a semi-supervised learning method via self-consistent augmentation to boost classification accuracy. He et al. [16] designed a densely connected triplet CNNs and integrated Fisher discrimination regularized metric learning for ship classification in medium-resolution SAR images. Zhang et al. [17] fused HOG features into CNNs to reduce model risk. However, these works did not consider ship polarization information, resulting in limited performance, especially for low-resolution SAR images.
Several works [18]- [21] tried to utilize polarization for better classification performance. For example, Zeng et al. [18] proposed a loss function for better dual-polarization feature training, but their network ignored feature interaction, which might lead to local optimization. Zhang et al. [19] designed a squeezeand-excitation Laplacian pyramid network for multi-resolution feature extraction, but their network did not highlight more salient features, causing limited accuracy gains. Xiong et al. [20] established a mini hourglass region extraction network for dualchannel feature fusion, but they did not consider channel correlation, resulting in insufficient utilization of polarization information. Zhang et al. [21] established a polarization fusion and geometric feature embedding network to increase feature richness, but their network treated each polarization branch equally, resulting in difficult training and incomplete feature extraction.
From above, it remains a challenging and unresolved issue to make full use of polarization information to further boost SAR ship classification performance. Previous works [6]- [21] have not provided a simple and effective implementation way so far. Therefore, we propose a dual-polarization information guided network (DPIG-Net) to address it. DPIG-Net utilizes available dual-polarization information from Sentinel-1 satellite to guide SAR ship classification from two aspectsfeature extraction and feature fusion. In the feature extraction process, we first design a novel polarization channel cross-attention framework (PCCAF) to model feature correlation, which can extract more representative features. In the feature fusion process, we design a novel dilated residual dense learning framework (DRDLF) to refine features, which can enable better feature fusion benefits.
Results on the open three-and six-category OpenSARShip datasets [22] reveal the state-of-the-art classification accuracy of DPIG-Net, compared with the other eleven competitive models.
The main contributions of this paper are as follows.
1) DPIG-Net is proposed for the sufficient polarization utilization to boost classification accuracy. It is a brand-new architecture to achieve dual-polarization SAR ship classification. Compared with the other state-of-the-art methods, DPIG-Net can make more full use of ship polarization information, and has the potential to implicitly mine useful dual-polarization feature patterns for better classification accuracy. 2) PCCAF is proposed for the representative polarization feature extraction. It is a brand-new framework for dual-polarization feature extraction. Compared with the other state-ofthe-art methods, PCCAF can model the correlation between different polarization channels by the proposed cross-attention subnetwork so as to serve for better feature extraction. 3) DRDLF is proposed for the refined polarization feature fusion. It is a brand-new framework to achieve dual-polarization feature fusion. Compared with the other state-of-the-art methods, DRDLF can maintain a large receptive field in network depth; its idea of feature reuse is conducive to the deep supervision of feature learning, reducing overfitting risk. 4) For the community of SAR ship detection and classification, we provide an idea of using polarization information to guide the intelligent interpretation of SAR images, and we contribute a network framework (PCCAF-DRDLF) which makes it possible to make full use of dual-polarization information.
The rest is organized as follows. II. DPIG-NET Fig. 1 shows the network architecture of DPIG-Net. It is similar to [24], but it is closely related to ship polarization. The data used in this work is the open OpenSARShip dataset, whose images samples are collected from Sentinel-1 SAR satellite. Sentinel-1 works in dual-polarization mode, i.e., vertical-vertical (VV) and vertical-horizontal (VH). The offered data is denoted by SVV and SVH which are plural. Since SVV has higher scattering energy of ships [23], it is selected as the middle main branch, i.e. I2 = |SVV|. SVH reflects less scattering energy of ships than SVV [23], so it is selected as the upper branch, i.e. I1 = |SVH|. See [23] for more descriptions. Moreover, the down branch in PCCAF is used to measure polarization channel difference for more comprehensive description of ship characteristics, and its input is where * denotes a complex conjugate operation. Thus, SVV and SVH used in our work must be complex data, rather than the previous commonly-used amplitude-based real data. To the best of our knowledge, OpenSARShip might be the only data that can meet this condition. Noted that FUSAR-Ship [12] only offered amplitude real data, so I3 cannot be obtained by Eq. (1). Moreover, images in FUSAR-Ship are not paired in the form of VV-VH or HH-HV, which makes our network unable to apply to it. Especially, our current work only considers dual-polarization case due to the limitation of available data. If full-polarization data is available in the future, one can expand DPIG-Net into four parallel branches to receive four different polarization inputs (or more branches for the cross-channel modeling).
PCCAF receives three types of data (I1, I2, and I3) for feature extraction. Its output is ZS which contains high-level semantic features of three types of data. DRDLF receives ZS for feature fusion using some dilated residual dense blocks and a global residual learning from the main branch I2. Finally, 2D feature maps are flattened into 1D feature vector to transmit into a fully-connected (fc1) layer. The terminal fc2 is responsible for the category prediction with a soft-max function.
DPIG-Net shows a tendency of feature aggregation from input three branches to the terminal feature integration. Previous works usually only adopted I2 to predict ship categories, i.e., the middle main branch of PCCAF. Differently, we make full use of polarization information (I1 and I3) to guide the classification prediction of I2. We call the above paradigm the dual-polarization information guided SAR ship classification.

A. Polarization Channel Cross-Attention Framework (PCCAF)
First, PCCAF establishes a simple encoder f to preliminarily extract features from three types of data. The encoder structure is shown in TABLE I. The encoder f uses standard convs to extract features, batch normalization (BN) [25] to ensure training, and ReLU to activate neurons. The max-pooling operation is to reduce the size of feature maps. With network deepening, the channel width increases by a multiple of 2. Our feature encoder f only has four stages, rather than usual five stages [17]. This is to avoid the loss of spatial features due to the small size of SAR ships. Their outputs are denoted by Z1, Z2, and Z3 for the subsequent processing. Note that more advanced encoder might bring better performance, but it is not the scope of this letter. We design a cross-attention subnetwork to model the correlation between different polarization branches. The design concept of the cross-attention subnetwork is that the middle main branch generates referenced feature maps to guide the other two auxiliary branches. Most existing attention networks [27], [29] merely refine their own feature maps in the uncrossed mode, which cannot solve multi-branch dual-polarization-guided case. That is, their module input has only one entry, but our proposed cross-attention subnetwork is specially designed for dual-polarization ship mission, i.e., our module input has two entries. The cross-attention subnetwork can be summarized as where Zr denotes the referenced feature maps (in this paper, Zr = Z2, i.e., the main VV branch), Zi denotes the feature maps to be corrected (in this paper, Zi means the VH branch Z1 or the polarization difference branch Z3), αi denotes the learned mapping, and Ai denotes the cross-attention map. Fig. 2(a) shows its network implementation. We take Z1 and Z2 as an example to introduce; the same to Z3 and Z2. We first concatenate the two input feature maps directly, and then, three convs with a skip connection are employed to learn inputs' interrelation. Finally, the learning knowledge will be activated by a sigmoid to obtain the final cross-attention maps A1.
Furthermore, for better skip connection fusion between shallow low-level features and deep high-level features, we design a self-attention module (SA-Module) to refine previous features. SA-Module's motivation is also related to SAR image characteristics, e.g. speckle noises and sea clutters. It can relieve their interferences to enhance ships' saliency, as shown in Fig. 2(a). SA-Module can highlight more important global information in space, suppress low-value information, promoting network information flow. Ablation studies in Sec. V-A indicate that it can offer a ~2% accuracy improvement on the six-category task. It generates a self-attention map to modify input and then the result is added to the raw conv branch. The above is described as where Ci denotes the i-th conv feature map, fSA denotes the SA- Module operation, f3×3 denotes the 3×3 conv. Fig. 2(b) shows the implementation process of SA-Module. Spatial features of the i-position are denoted by ϕ by a 1×1 conv learning. Spatial features of the j-position are denoted by θ by another one 1×1 conv learning. The relationship between i-position and j-position is denoted by f which is obtained by an adaptive learning between ϕ and θ, where the normalization process is equivalent to a soft-max function. The representation of the input at j-position is denoted by g learned by another one 1×1 conv. The response at i-position is obtained by a matrix element-wise multiplication between input Ci-1 and self-attention map fSA (Ci-1). The final resulting cross-attention map is acted on the other two branches by matrix element-multiplication to obtain refined polarization-guided features, i.e., where Zi' denotes polarization-guided features that will be used to guide the main polarization branch. Finally, the output of the main polarization branch is the concatenation of three types of features, i.e., Concat , , where ZS denotes the output of PCCAF. We find that the feature concatenation performs better than feature adding, because the former can avoid the resistance effects between different polarization features with our subsequent feature fusion operations.

B. Dilated Residual Dense Learning Framework (DRDLF)
DRDLF uses some dilated residual dense blocks (DRDBs) to fuse the extracted polarization features coming from the previous PCCAF stage. The input of DRDLF is ZS which is associated with dual-polarization information using the concatenation operation of Eq. (5) where Z1' denotes the feature maps of I1 VH information, Z2 denotes that of I2 VV information, and Z3' denotes that of VV-VH correlation information. ZS is refined by a 3×3 conv for feature concentration and channel dimensionality reduction. The result is denoted by F0. Then, several DRDBs are used for feature aggregation. DRDB is motivated by RDB [26] designed for image super-resolution tasks. However, there are many speckle noises around SAR ship images, so we insert a dilated rate 2 to the standard conv for larger receptive fields. Fig. 3 shows DRDB's implementation. Its input is the previous output Fi, and its output is denoted by Fi+1. DRDB contains three 2-dilated 3×3 conv layers whose results are denoted by D1, D2, and D3. They are concatenated directly as DS. For the residual learning in the entire DRDB, a 1×1 conv is used for channel reduction. Finally, the sum between Fi and DS is its output. In DRDLF, we arrange n DRDBs for feature fusion where n is set to the optimal value 3 empirically. The results of n DRDBs from F1 to Fn are concatenated and then processed by a 1×1 conv for overall channel reduction. The result is denoted by Q0.
We observe that after a series of DRDB processing, the details of the main VV branch might be gradually diluted, causing unstable training and deteriorating performance. Thus, we propose a global residual learning to solve this problem. As in Fig.  1, the global residual learning connects PCCAF and DRDLF that can maintain the dominant position of the main branch and make the other two branches smoothly play an auxiliary guiding role. This is an important design idea of our dual-polarization guided network. The global residual learning is described by where Q1 denotes the final output of DRDLF. From Fig. 1, we set another two 3×3 convs to process Q1 for more semantic features Q2, helpful for balancing spatial and semantic information.
To sum up, combined with the above designed PCCAF and DRDLF, our proposed DPIG-Net can make full use of the polarization information ignored in previous works. The other two types of polarization data are well refined to assist the feature extraction and feature fusion of the main branch. Finally, an effective dual-polarization information guided SAR ship classification paradigm is realized. DPIG-Net successfully handles the problems of how to conduct polarization guidance and how to carry out more effective polarization guidance, of great value.

III. EXPERIMENTS A. Dataset
The open OpenSARShip dataset [22] is used to evaluate the effectiveness of DPIG-Net. It offers VV-VH dual-polarization SAR ship data from Sentinel-1. The raw data is the single look complex (SLC) type. Same as [19], its two subsets are used for experiments, i.e., a three-category subset and a six-category one. As mentioned before, OpenSARShip is the only one dataset that can satisfy our experimental requirements, i.e., paired dual-polarization complex data with corresponding ground truth labels .  TABLE II and TABLE III show more data descriptions. Fig. 4 and Fig. 5 show some samples of different ship categories.

B. Training Details
We train DPIG-Net by 100 epochs from scratch using Adam with a learning rate of 0.0001. The network parameters are initialized by [30]. Samples are resized to 224×224 by bilinear interpolation. The batch size is set to 16. The multi-category cross entropy [15] serves as the loss function of networks. We reproduce other models basically consistent with their raw reports.        Tanker  Bulk carrier  143  23  43  0  22  2  Cargo  69  325  19  27  83  48  Container ship  67  16  359  0  29  2  Fishing  0  2  0  22  0  1  General cargo  6  20  2  0  9  5  Tanker  13  91  1  4  19  14 The accuracy (Acc) equals (TP+TN)/(TP+TN+FP+FN). TP denotes the true positives, TN denotes the true negatives, FP denotes the false positives, and FN denotes the false negatives.

IV. RESULTS
A. Classification Performance 1) Accuracy. TABLE IV is the quantitative evaluation of different models. The top-10 best results among 20 trainings are used to calculate the average and standard deviation except for DenseNet-LRCS [33]. DPIG-Net outperforms the other eleven comparative models obviously. The second-best model offers a 79.84% accuracy on the three-category task which is still lower than ours by 1.44%, and a 56.83% accuracy on the six-category task which is still lower than ours by 1.85%. This reveals the state-of-the-art classification performance of DPIG-Net. Note that such accuracy increment is already a huge progress in the SAR ship classification community. Compared with the other methods, DPIG-Net can make more full use of ship polarization information, and has the potential to implicitly mine useful dual polarization feature patterns for better classification accuracy.
2) Computational Efficiency. Fig. 6 shows the classification time comparison with different methods. DPIG-Net consumes more time (5.12ms) to classify ships than most other methods, but it is still faster than DenseNet-LRCS [33]. Furthermore, the speed gap between DPIG-Net and other methods is relatively  This indicates that DPIG-Net might be a little heavy, which exactly leads to its more running time in our experiments as in Fig.  6. Thus, the speed optimization will be studied in the future. B. Confusion Matrix TABLE V-VI are the confusion matrix of DPIG-Net. DPIG-Net can identify most ships successfully, i.e., the diagonal value is greater than others at the same line in most cases.

V. DISCUSSIONS A. Discussion on PCCAF
To confirm the effectiveness of PCCAF, we conduct some ablation studies on it, including the polarization-guided practice and the proposed cross-attention module. The results are shown in TABLE VII. From TABLE VII, the polarization-guided practice can offer obvious accuracy gains. Taking the six-category task as an example, I1 (the VH polarization channel) boosts the accuracy by 1.47%, and I3 (the polarization channel difference) boosts the accuracy by 2.67%. The combination of two inputs is better than the single alone; the combination of three inputs is better than the combination of two inputs. The above shows the effectiveness of polarization information. Moreover, the offered accuracy gain is greater than some previous works [19], [21]. This shows that PCCAF can make more full use of polarization information. Finally, the proposed cross-attention module can improve the classification accuracy further (a ~2% improvement on the six-category task), which is in line with the subjective analysis in Sec. II-A. This is because it can establish the correlation between channels to extract features with more mutual recognition. As a result, the information flow between channels is promoted for better feature extraction.
We discuss the effect of different inputs in the main branch on results as shown in TABLE VIII. The VV I2 offers better results than others since it contains more ship scattering energy.
We conduct another one study to verify the advantage of feature concatenation over feature adding. Results are in GBCNN [32] VGGNet-Grey [31] HOG-ShipCLSNet [17] Mini-Hourglass Net [20] Zeng et al. [18] Wang et al. [14] Hou et al. [12] GSESCNN [13] Time (ms)   of SA-Module in the cross-attention subnetwork in TABLE X. SA-Module improves accuracy further since it can enable more prominent features for multi-stage residual fusion. Furthermore, SA-Module can ease negative effects of speckle noises and sea clutters of SAR characteristics, to enhance ships' saliency, as in Fig. 2(a). This is in line with experimental results in TABLE X.

B. Discussion on DRDLF
To verify the effectiveness of DRDLF, we conduct some ablation studies on it. The results are shown in TABLE XI. DRDB improves the accuracy by 1.54% on the three-category task and by 2.08% on the six-category task. It can learn context information more effectively for achieve more concentrated feature fusion effects. Furthermore, the global residual learning boosts the accuracy further, because it can restore original feature details from the main branch I2 effectively, which avoids possible feature loss from multi conv and pooling operations.
We determine the number of DRDB empirically via experiments in TABLE XII. From TABLE XII, the accuracy increases first and then decreases as the increase of the number of DRDBs. One possible reason is that excessive DRDBs may lead to overfitting for its large number of network parameters. For this, we set the number of DRDB to the optimal value 3.
VI. CONCLUSIONS DPIG-Net is designed for dual-polarization guided SAR ship classification. PCCAF is designed for better dual-polarization feature extraction. DRDLF is designed for fine dual-polarization feature fusion. DPIG-Net utilized available dual-polarization information from Sentinel-1 to guide better ship classification. We perform extensive experiments on the public Open-SARShip dataset to confirm the effectiveness of DPIG-Net. Results reveal the state-of-the-art classification accuracy of DPIG-Net, compared with the other eleven competitive models.
Limitations and Future Works. The running speed of DPIG-Net is not attractive, so we will optimize its speed. Moreover, DPIG-Net merely considers polarization guidance at the feature map level, so other levels, e.g., decision-level, will be studied.