{'fraction_non_alphanumeric': 0.03503960999390616, 'fraction_numerical': 0.015691651432053626, 'mean_word_length': 4.622922734281309, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Efficient Framework for Visible-Infrared Cross Modality Person Re-Identification

Visible-infrared cross modality person re-identification (VI-ReId) is an important task for video surveillance in poorly illuminated or dark environments. Despite many recent studies on person re-identification in visible domain (ReId), there are few studies dealing with VI-ReId. Besides challenges that are common for both ReId and VI-ReId such as pose/illumination variations, background clutter and occlusion, VI-ReId has additional challenges as color information is not available in infrared images. As a result, the performance of VI-ReId systems is typically lower than ReId systems. In this work, we propose a 4-stream framework to improve VI-ReId performance. We train a separate deep convolutional neural network in each stream using different representations of input images. We expect that different and complementary features can be learned from each stream. In our framework, grayscale and infrared input images are used to train the ResNet in the first stream. In the second stream, RGB and 3-channel infrared images (created by repeating infrared channel) are used. In the remaining two streams, we use local pattern maps as input images. These maps are generated utilizing local Zernike moments transformation. Local pattern maps are obtained from grayscale and infrared images in the 3rd stream and from RGB and 3-channel infrared images in the last stream. We improve the performance of the proposed framework by employing a re-ranking algorithm for post processing. Our results indicate that the proposed framework outperforms current state-of-the-art on SYSU-MM01 dataset with large margin by improving Rank-1/mAP by 34.2%/37.9% and 37.4%/34.8% under all-search and indoor-search modes, respectively.


Introduction
Person re-identification (ReId) can be simply defined as the retrieval of the images of a person from a gallery set consisting of the images captured by different cameras with different field-of-views, using a query image of the same person. It is an important problem for various real-world scenarios, especially for security. Therefore, this problem has recently attracted great attention of researchers working in computer vision and machine learning fields.
The illumination conditions, visible body parts, occlusion and background complexity can vary ex-for video surveillance or miscellaneous applications. This issue is studied in literature as visible-infrared cross-modality person re-identification (VI-ReId) [1,2,3,4,5]. VI-ReId is the problem of retrieving the images of a person from a gallery set consisting of RGB (or infrared) images given an infrared (or RGB) query image. One of the most important cues for person images is obtained from the color for ReId. Therefore, the lack of color information in infrared images makes VI-ReId a very challenging problem.
In this paper, we show that ResNet [6] architectures trained with the use of RGB and infrared images together can outperform the current state-ofthe-art. These architectures are widely used for image classification and other computer vision problems. They can learn the common feature representations for RGB and infrared images of the same individual as well as the distinguishing properties between the individuals better than the existing methods proposed for VI-ReId. In this study, we introduce a 4-stream framework built with ResNet architectures. There is no weight sharing between the ResNets in the framework, and in order to obtain different and complementary features as much as possible, we train each of them using a different representation of input images. In the first stream, RGB images converted to grayscale and infrared images are used together. In this way, features are extracted using only the shape and pattern information. In the second stream, to take advantage of the color information, RGB images and the 3channel infrared images (obtained by repeating the infrared image) are used. As mentioned above, one of the most important cues for ReId is obtained from the color information. However, due to the lack of color in infrared images, local shape and pattern information is critical for VI-ReId. Therefore, in addition to the features obtained from the raw (RGB, grayscale and infrared) images in the first two streams of the framework, in the last two streams, features are obtained from the local pattern maps. In the 3rd stream, we first apply the local Zernike moments (LZM) transformation [7] on grayscale and infrared images to expose the local patterns in the images. With the LZM transformation, different number of local pattern maps are generated by computing the Zernike moments around each pixel. We train the ResNet in the 3rd stream by using the LZM pattern maps of grayscale and infrared images. In the last stream of the framework, we generate the pattern maps by ex-posing the local patterns separately in R, G and B channels.
The contributions of this paper are summarized as follows: • We propose a novel framework consists of 4 streams. In order to obtain complementary features from each stream, we train a ResNet architecture in each stream by using a different representation of input images.
• This is the first study employing the LZM transformation for ReId and training a deep convolutional neural network with the LZM pattern maps.

Related Work
In recent years, deep convolutional neural networks (DCNN) have led to a serious progress in many different computer vision areas. Person reidentification (ReId) is one of the challenging computer vision problems, and in order to cope with the problems such as illumination conditions, variations in pose, closure and background clutter, researchers have developed many DCNN frameworks tackling ReId in different ways [8,9,10,11].
In majority of the studies on ReId, only the images in visible spectrum (RGB-based) are taken into account when addressing the challenges, and the developed methods are intended for RGB-based images only. There are only a few studies for visible-infrared or visible-thermal cross modality person re-identification. However, they are important issues for video surveillance or miscellaneous applications that have to be performed in poorly illuminated or dark environments. In [1], the authors have analyzed different network structures, including one-stream and two-stream architectures and asymmetric full-connected layer, for visible-infrared re-identification and found the performance of the one-stream architecture to be better. In addition, they have proposed deep zero padding to contribute to the performance of the one-stream network. Since the authors are training the architectures using grayscale input images, they do not take advantage of the color of RGB images. Kang et. al.
[5] propose a one-stream model for cross-modality re-identification. They create single input images by placing visible and infrared images in different channels or by concatenating them. Using these input images consisting of positive and negative pairs, they train a DCNN employing 2-class classification loss. In the study, some pre-processing methods are also analyzed. Ye et. al. [2] use a 2stream DCNN to obtain modality-specific features from the images. One stream of the network is fed with RGB while the other is fed with infrared (or thermal) images, and on top of the streams, there is a shared fully connected layer in order to learn a common embedding space. The authors train the 2-stream model using multi-class classification loss along with a ranking loss proposed by them. A very similar 2-stream model is employed in [3], but this model is trained utilizing contrastive loss instead of ranking loss used in [2]. To improve discrimination ability of the features extracted by 2-stream model, they propose a hierarchical cross-modality metric learning. In this method, modality-specific and modality-shared metrics are jointly learned in order to minimize cross modality inconsistency. In [12], a generative adversarial network (GAN) is proposed for visible-infrared re-identification. The authors use a DCNN, which is trained employing both multi-class classification and triplet losses, as generator to extract features from RGB and infrared images. The discriminator is constructed as a modality classifier to distinguish between RGB and infrared representations. An another work utilizing a variant of GAN has introduced by Kniaz et. al. [13] for visible-thermal re-identification. In their method, a set of synthetic thermal images are generated for an RGB probe image by employing a GAN framework. Then, the similarities between the synthetic probe and the thermal gallery images are calculated. Wang et. al. [4] propose a network dealing the modality discrepancy and appearance discrepancy separately. They first create a unified multi-spectral representations by projecting the visible and infrared images to a unified space. Then, they train a DCNN using the multi-spectral representations by employing triplet and multi-class classification losses. There are also some other works [8,14] trying to improve RGB-based ReId by fusing the information from different modalities.
In the literature, most of the existing studies on cross-modality matching or retrieval have been performed for heterogeneous face recognition (HFR) and text-to-image (or image-to-text) matching in the past decade. Studies on HFR, which are more relevant to VI-ReId, can be reviewed in three perspectives [15]: image synthesis, latent subspace, and domain-invariant features. In the first group, before the recognition, face images are transformed to the same domain [16,17,18]. The approach followed by the studies in the second group is the projection of the data of different domains into a common latent space [19,20,21]. In the recent works exploring domain-invariant feature representations, deep learning based methods are proposed. In [22], the features are obtained from a DCNN pre-trained on visible spectrum images, and metric learning methods are used to overcome inconsistencies between the modalities. He et. al. [23] build a model in which the low-level layers are shared and the high-level layers are divided into infrared, visible and shared infrared-visible branches. Peng et. al. [24] propose to generate features from facial patches and utilize a novel cross-modality enumeration loss while training the network. In [25], Mutual Component Analysis (MCA) [26] is integrated as a fully-connected layer into DCNN, and an MCA loss is proposed.
In this study, we propose a framework that utilizes the pattern maps generated by local Zernike moments (LZM) transformation [7]. Zernike moments (ZM) [27] are commonly used in many different computer vision problems as robust holistic image descriptors, i.e., character [28], fingerprint [29] and iris [30] recognition, and object detection [31]. LZM transformation is proposed by Sariyanidi et. al. [7] to utilize ZM at local scale for shape/texture analysis. In this transformation, images are encoded by calculating the ZM around each pixel. Thus, the local patterns in the images are exposed and a rich representation is obtained. In literature, LZM transformation is used in various studies such as face recognition [7,32,33,34], facial expression [35,36] and facial affect recognition [37], traffic sign classification [38], loop closure detection [39,40], interest point detector [41]. In these works, the features are obtained directly from the LZM pattern maps. Unlike them, in this study, we use the LZM pattern maps as input images to train DCNNs. This is the first study training DCNNs with the LZM pattern maps, but there are some attempts employing ZM in convolutional networks. Mahesh et. al. [42] initialize the trainable kernel coefficients by utilizing ZM with different moment orders. In [43] and [44], learning-free architectures are built constructing the convolutional layers by using ZM. Sun et. al. [45] introduce a novel concept of Zernike convolution to extend convolutional neural networks for 2D-manifold domains.

Proposed Method
In this section, we begin by giving the details of ResNet [6] architectures. Then, we describe the LZM transformation [7] and introduce our 4-stream framework proposed for VI-ReId. Finally, we explain the ECN re-ranking algorithm [10] used as a post-processing.

ResNet Architecture
In many studies performed with DCNNs [46,47,48,49], it is observed that with the increase of the depth of the models, the performance achieved on the challenging datasets such as ImageNet [50] have also increased. However, after some depth, the models lose their generalization capabilities, which leads to a rapid decrease in performance. This is defined as notorious problem of vanishing gradients [51,52]. ResNet architecture has been proposed by He et. al. [6] to overcome this problem. The main idea of ResNet architecture is to learn the residual function F (x) = H(x) − x instead of the mapping H(x) to be learned by several stacked layers. Then, the mapping H(x) becomes F (x) + x. This operation is illustrated in Figure 1 which shows the building block of ResNet architecture. In this building block, F (x) represents the output of a few stacked layers for input x, and F (x) + x is obtained by element-wise addition. With the shortcut connections shown in Figure 1, the signals can be directly propagated between the building blocks. In this way, the vanishing gradients problem can be avoided and deeper models can be created [6]. In the experiments performed in this study, we show the performance of our proposed framework by using 3 different ResNet architectures which have different depths. These architectures have Table 1: ResNet architectures used in this study. The building blocks are given in brackets, and each row in the brackets indicates the kernel sizes and the number of output channels of convolution layers used in the building blocks. 50, 101 and 152 layers (we call these architectures as ResNet-50, ResNet-101 and ResNet-152, respectively, in the rest of the paper), and the details of them are provided in Table 1. Each ResNet model shown in Table 1 starts with a 7 × 7 convolution layer followed by a 3 × 3 max pooling. The next layers of the models consist of different number of building blocks stacked and there is a global average pooling layer at the top of them. In Table  1, we show the building blocks in brackets where each row in the brackets indicates the kernel sizes and the number of output channels of convolution layers used in the building blocks.

LZM transformation
Zernike moments (ZM) [27] are commonly used to generate holistic image descriptors for various computer vision tasks [28,29,30]. Independent holistic characteristics of the images are exposed with the calculation of ZM of different orders [53]. In order to reveal the local shape and texture information from the images by utilizing ZM, local Zernike moments (LZM) transformation is proposed in [7] .
Zernike moments of an image are calculated using orthogonal Zernike polynomials [27]. These polynomials are defined in polar coordinates within a unit circle and represented as follows: where R nm (ρ) are real-valued radial polynomials, and ρ and θ are the radial coordinates calculated as Here, x i and y j are the pixel coordinates scaled to range of [−1, 1]. R nm (ρ) polynomials are defined as (4) In (1) and (4), n and m are the moment order and the repetition, respectively. They take values such that n − m is even, 0 ≤ n and 0 ≤ m ≤ n. Real and imaginary components of some V nm (ρ, θ) Zernike polynomials are shown in Figure 2. Finally, ZM of an image f (i, j) are calculated as: In the LZM transformation, images are encoded in k × k neighborhood of each pixel, as follows: represents the Zernike polynomials used as k × k filtering kernels and defined as We give real and imaginary components of some V k nm (i, j) filters in Figure 2, and in Figure 3, we show a few pattern maps generated using the LZM filters. The filters constructed using m = 0 are not used in the LZM transformation since the imaginary components of Zernike polynomials are not generated. According to the moment order n, the number of complex filters is calculated using if n is even, if n is odd.
Since we create a pattern map with each of the real and imaginary components of complex filters, the total number of local pattern maps becomes 2 × K(n).

Person Re-Identification Framework
The proposed framework for VI-ReId is shown in Figure 4. This framework includes 4 streams, and in each stream, a separate ResNet architecture is trained using both infrared and RGB images. Since There is a large visual difference between infrared and RGB images since infrared images have a single channel with invisible light information. This visual difference is reduced by converting RGB images to grayscale in the upper stream of the proposed framework, and the training is performed using infrared and grayscale images. In this way, ResN et gray ir in the upper stream of the framework learns to extract robust features for the person images by utilizing only the shape and texture information.
For ReId, one of the most important cues for person images is obtained from the color. Therefore, in the second stream of the proposed framework,  There is no weight sharing between ResNets in the framework, and each ResNet is trained with a different representation of input images. In the first stream, grayscale and infrared input images are used, while RGB and 3-channel infrared images (created by repeating infrared channel) are used in the second stream. In the other streams, LZM pattern maps are used as input images. These maps are obtained from grayscale and infrared images in 3rd stream and separately from R, G and B channels in 4th stream. We employ multi-class classification with softmax cross-entropy loss for each stream separately during the training. In the evaluation phase, the final representations are constructed by concatenating the feature vectors obtained from each stream. Best viewed in color.
ResN et rgb ir is trained using RGB and 3-channel infrared (created by repeating the infrared channel) images. Unlike the one that is trained in the upper stream, the model trained in this stream uses color information of the RGB images and extracts discriminative features for the images in different modalities with visually large differences.
In VI-ReId, due to the lack of color in infrared images, the local shape and pattern information is of great importance while matching the infrared and RGB images. Therefore, in addition to the two streams mentioned above, there are two other streams in the proposed framework in order to obtain stronger features related to local shape and pattern information. In the third stream, first, the local patterns are exposed with applying the LZM transformation on the grayscale and infrared images. Then, ResN et LZM (gray ir) is trained using the generated LZM pattern maps. As indicated in Section 3.2, a number of complex pattern maps are obtained as a result of the LZM transformation. The input tensor I LZM (gray ir) for ResN et LZM (gray ir) is prepared by concatenating the real and imaginary components of the complex maps. I LZM (gray ir) is denoted as where I gray/ir represents the grayscale or infrared (12) In order to train the ResNet architectures in the proposed framework, multi-class classification with softmax cross-entropy loss is employed. Thus, in addition to learning to extract common features for the images in different modalities, the ResNet models also learn to extract features that express dif-ferences between individuals. As shown in Figure  4, the loss for each stream is calculated separately during the training. In the evaluation phase, the final representations for the person images are constructed by concatenating the feature vectors obtained from each stream and normalized using 2norm.

Re-Ranking
We utilize Expanded Cross Neighborhood (ECN) re-ranking algorithm [10] as a post-processing to further improve the re-identification performance. In this algorithm, the distance between a probe image p and a gallery image g i from a gallery set G with B images G = {g i |i = 1, 2, · · · , B} is defined as (13) Here, pN j and g i N j are the jth neighbors in the expanded neighbor sets N (p, M ) and N (g i , M ) of the probe and ith gallery images, respectively. d(·) represents the distance between the images, and M is the total number of neighbours in a set. In order to construct the expanded neighborhood sets, first, an initial rank list L(p, G) = {g o 1 , · · · , g o B } in increasing order is created for each image by calculating the pairwise euclidean distances between all images in the probe and gallery sets. Then, the expanded neighbor set N (p, M ) for a probe image p is given as where the set N (p, t) consists of the t nearest neighbors of probe p, and the set N (t, q) contains the q nearest neighbors of each of the images in N (p, t) such that: The expanded neighbor set N (g i , M ) for gallery image g i is obtained in the same way. In Equation 13, as suggested by the authors of [10], we compute the distance between the pairs using a list comparison similarity measure proposed in [54] and defined as In this equation, [·] + = max(·, 0), K is the number of nearest neighbors to be considered, and pos i (b) and pos j (b) show the position of image b in the rank lists L i and L j , respectively. The distance d between the pairs is calculated as d = 1 − R after scaling the range of the values of R between 0 and 1. For the parameters t, q and K of ECN re-ranking algorithm, we use the same setting given in [10] such that t = 3, q = 8 and K = 25.

SYSU-MM01 Dataset
The SYSU-MM01 [1] is a dataset collected for VI-ReId problem. The images in the dataset were obtained from 491 different persons by recording them using 4 RGB and 2 infrared cameras. Within the dataset, the persons are divided into 3 fixed splits to create training, validation and test sets. In the training set, there are 20284 RGB and 9929 infrared images of 296 persons. The validation set contains 1974 RGB and 1980 infrared images of 99 persons. The testing set consists of the images of 96 persons where 3803 infrared images are used as query and 301 randomly selected RGB images are used as gallery. For the evaluation, there are two search modes, all-search and indoor-search. In the all-search mode, the gallery set consists of the images taken from RGB cameras (cam1, cam2, cam4, cam5), and the probe set consists of the images taken from infrared cameras (cam3 and cam6). In the indoor-search mode, the gallery set contains the RGB images only taken from the cameras (cam1 and cam2) located in the indoor. In the standard evaluation protocol of the dataset, the performance is reported using cumulative matching characteristic (CMC) curves and mean average precision (mAP). CMC and mAP are calculated with 10 random split of the gallery and probe sets, and the average performance is reported. We follow the single-shot setting for all the experiments. In Figure 5, some sample images from SYSU-MM01 are shown.

Training the Networks
While training the ResNet networks in the proposed framework, we set the resolution of all RGB and infrared input images to 492 × 164. In the experiments where the LZM transformation is used, we normalize the images to have zero mean and unit variance. We choose Nesterov Accelerated Gradient [55] to train the networks and set the maximum number of iterations to 100K, the mini-batch size to 8, momentum to 0.9 and weight decay to 0.0005. For the initial value of the learning rate, we use 0.01 and decay it 10 times using exponential shift with the rate of 0.9, such that where lr init and lr new are the initial and the updated learning rates, respectively, and iter is the current number of iterations and max iter is the maximum number of iterations.

Re-Identification Performance
In this section, we call the ResNet models in the streams of the proposed framework as RX gray ir , RX rgb ir , RX LZM (gray ir) and RX LZM (rgb ir) , respectively, to avoid confusion while reporting the experimental results. The X in the labels indicates which ResNet model is used and takes one of the values 50, 101 and 152. In this section, we show the experimental results under all-search mode.
As mentioned in Section 3.2, the number of filters used in the LZM transformation depends on the moment degree n (Eq. 8). Therefore, we have first carried out experiments with model R50 LZM (gray ir) in order to determine the optimal values for both the moment degree n and the filter size k. The results obtained with different parameter sets are given in Table 2. According to this table, the results are close to each other, but slightly better results are obtained when 3/5 and 3/7 are used for n/k pair. For this reason, we prefer to use n = 3 and k = 5 to generate the LZM pattern maps in the rest of the study. Table 3 shows the results obtained by training a ResNet-50 in each stream of the proposed framework. According to the results, better performance is achieved with R50 gray ir and R50 LZM (gray ir) , where grayscale & infrared input images are used, compared to those of R50 rgb ir and R50 LZM (rgb ir) using RGB & infrared input images. Reducing the visual differences between infrared and RGB images by converting RGB images to grayscale makes a positive contribution to the performance. When the features obtained from the first two streams and the last two streams are combined, a significant performance increase is observed as can be seen in 3rd and 6th rows of Table 3. With the concatenation of the features extracted by R50 gray ir and R50 rgb ir , 4.5% improvement is achieved for Rank1 as compared to R50 gray ir . Likewise, the concatenation of the features extracted by R50 LZM (gray ir) and R50 LZM (rgb ir) improves Rank1 by 3.3% as compared to R50 LZM (gray ir) . This shows that we generate complementary features with the models that are trained with grayscale & infrared and RGB & infrared input images. In the last row of Table 3, the results achieved by combining the features obtained in all streams of the proposed framework are given. When they are compared to those of R50 gray ir + R50 rgb ir and R50 LZM (gray ir) + R50 LZM (rgb ir) , it is seen that the improvement obtained for Rank 1 is 5.3% and 3.7%, respectively. Therefore, taking this performance improvement into account, we can conclude that the models trained with the LZM pattern maps generate features that are different from and complementary to the ones produced by R50 gray ir and R50 rgb ir .
While performing the experiments whose results are given in Table 2 and 3, we have initialized the weights of ResNet-50 with scaled Gaussian distribution proposed in [46]. However, many ReId studies [9,10] in the literature use pre-trained models instead of the models with randomly initialized weights. The vast majority of these pre-trained  models are trained using ImageNet [50] which is a large-scale image classification dataset. In this way, the information of the model capable for image classification is utilized for ReId. By following the same approach, we have conducted experiments using ResNet-50 pre-trained on ImageNet for each stream of our framework. The results are given in Table 4. In the 3rd and 4th streams of the framework, the number of channels of the input images are greater than 3 because the LZM pattern maps are used. Therefore, in the experiments, we remove the first convolutional layer (conv1) of the pretrained ResNet-50 and insert a randomly initialized convolutional layer whose number of input channels matches the number of LZM pattern maps. For the first stream of the framework, we create 3-channel input images by repeating the grayscale/infrared images for R, G and B channels. If the results given in Table 3 and 4 are compared, it is seen that we achieve a significant performance improvement for each stream by using pre-trained ResNet-50 models. With the concatenation of the features from 4 streams, we improve Rank1 and mAP by 6.6% and 6.3% as compared to the results in the last row of Table 3 and reach to 45% and 45.9%, respectively. According to recent research on computer vision tasks such as image classification, it is observed that deeper networks are more accurate [56]. In order to show the performance of our framework with deeper networks, we have conducted experiments using ResNet-101 and ResNet-152 which have much more layers than ResNet-50. In the experiments,  we have used the models pre-trained on ImageNet, and we show the results in Tables 5 and 6. As can be seen in 3rd and 6th rows of the Tables 5 and 6, a significant performance improvement is obtained with the concatenation of the features generated in the first two streams and in the last two streams, as in Tables 3 and 4. The highest results are achieved when the features from all the streams are combined. Using ResNet-101 and ResNet-152, we improve the best Rank1 result given in  Table 7 shows the results obtained using ResNet-50 for each stream of the proposed framework. We have computed the results given in the second row of the table by using R101 gray ir and R101 rgb ir for the third and fourth streams of the frame-  Tables 8 and 9 show our results in comparison with the state-of-the-art.

Comparison With the State-Of-the-Art
In [1], zeropadding is utilized to enable a one-stream network to learn domain-specific structures automatically. TONE+HCML [3] and TONE+XQDA [3] use a two-stage framework that includes feature learning and metric learning. BCTR [2] and BDTR [2] have a two-stream framework that employs separate networks for RGB and infrared images to extract domain specific features. Kang et. al. [5] train a one-stream network using single input images generated by placing visible and infrared images in different channels or by concatenating them. In [12], a generative adversarial network is used in order to extract common features for the images in different domains. D 2 RL [4] train a onestream network with multi-spectral images generated by projecting RGB and infrared images to a unified space. Different than these works, in this study, we train multiple ResNet architectures by using the different representation of the input images and generate complementary features from each one of them for RGB and infrared images. As can be seen from Tables 8 and 9, we outperform the current state-of-the-art with a large margin. When we use ResNet-152 architectures in the all streams, our framework improves VI-ReId performance on SYSU-MM01 under all-search mode by 18.5%/19.1% and under indoor-search mode by 20.5%/20.1% in Rank1/mAP. By concatenating all the feature vectors generated with ResNet-50, ResNet-101 and ResNet-152, we obtain at least 1.5% additional improvements for Rank1 and mAP under both search modes. When we perform the re-ranking [10], the margin of the improvement further increases. We achieve 63.1%/67.1% and 69.1%/77.0% under all-search and indoor-search modes, respectively.

Conclusion
In this study, we have introduced a 4-stream framework for VI-ReId using ResNet architectures. In each stream of the framework, we train a ResNet by using a different representation of input images in order to obtain complementary features as much as possible from each stream. While grayscale and infrared input images are used to train the ResNet in the first stream, RGB and 3-channel infrared images are used in the second stream. The first stream learns the features by using only the shape and texture information. The second stream uses the color information of the RGB images and learns to extract common features for the images with visually large differences. Unlike the first two streams, the input images in the other two streams are local pattern maps generated by employing the LZM transformation. Due to the lack of color, which provides the most important cues, in infrared images, the local shape and texture information is critical for VI-ReId. In the 3rd and 4th streams, we expose this information from the images by generating the LZM pattern maps and train the ResNets using these maps as input images. With the exhaustive experiments performed employing 3 different ResNet architectures with different depths, we have demonstrated that each stream extracts different and complementary features and has a significant contribution to the performance. Our framework outperforms current state-of-the-art on SYSU-MM01 dataset under all-search and indoor-search modes with a large margin. We further increase the improvement margin by utilizing re-ranking.