A Convolutional Baseline for Person Re-Identification Using Vision and
  Language Descriptions
  Classical person re-identification approaches assume that a person of
interest has appeared across different cameras and can be queried by one of the
existing images. However, in real-world surveillance scenarios, frequently no
visual information will be available about the queried person. In such
scenarios, a natural language description of the person by a witness will
provide the only source of information for retrieval. In this work, person
re-identification using both vision and language information is addressed under
all possible gallery and query scenarios. A two stream deep convolutional
neural network framework supervised by cross entropy loss is presented. The
weights connecting the second last layer to the last layer with class
probabilities, i.e., logits of softmax layer are shared in both networks.
Canonical Correlation Analysis is performed to enhance the correlation between
the two modalities in a joint latent embedding space. To investigate the
benefits of the proposed approach, a new testing protocol under a multi modal
ReID setting is proposed for the test split of the CUHK-PEDES and CUHK-SYSU
benchmarks. The experimental results verify the merits of the proposed system.
The learnt visual representations are more robust and perform 22\% better
during retrieval as compared to a single modality system. The retrieval with a
multi modal query greatly enhances the re-identification capability of the
system quantitatively as well as qualitatively.
