{'fraction_non_alphanumeric': 0.02037351443123939, 'fraction_numerical': 0.0, 'mean_word_length': 4.836633663366337, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multi-modal scene categorization using multi-tasks learning

The combination of standard visible RGB channels and near infrared (NIR) information has been observed to perform better than RGB-only data in the scene categorization task. However, RGB-NIR data are limited due to the collection difficulties. With limited data, it is hard for most deep networks to learn an effective solution, while humans are able to solve such kind of problem within a little of exercises. It may be because we are able to learn prior knowledge from other tasks or a good mentor and these knowledge is helpful. Inspired by this hypothesis, a new training scheme is proposed in order to introduce the prior knowledge into the intermediate layers of network. First of all, transfer learning is adopted to learn single modal features from related tasks. Then a knowledge distillation method is explored to fuse the RGB and NIR features. As these pre-training stages have been finished, global fine-tuning is employed to jointly train the whole network. The experimental results on the RGB-NIR dataset demonstrate the efficiency of our proposed approach in comparison with the other existing methods.