{'fraction_non_alphanumeric': 0.01697045883092395, 'fraction_numerical': 0.00251414204902577, 'mean_word_length': 5.243137254901961, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Cross Modal Deep Learning Based Approach for Caption Prediction and Concept Detection by CS Morgan State

This article describes the participation of the Computer Science Department of Morgan State University, Baltimore, Maryland, USA in the ImageCLEFcaption under ImageCLEF 2018. The problem of automatic image caption prediction involves outputting a human-readable and concise textual description of the contents of a figure appeared in a biomedical journal. It is a challenging problem in biomedical literature as it requires techniques from both the fields of Computer Vision to understand the visual contents of the image and Natural Language Processing (NLP) to turn the understanding of the image into words in the right order to generate the textual description for caption. Recently, deep learning methods have achieved state-of-the-art results on this challenging problem in general domain of natural photographic images. For the tasks of caption prediction and concept detection, we used the merge architectures for the encoder-decoder recurrent neural network (RNN) due to it’s success over inject for caption generation. The merge model combines both the encoded form of the image input with the encoded form of the text description generated so far. The combination of these two encoded inputs is then used by a very simple decoder model to generate the next word in the sequence. In this article, we present main objectives of experiments, overview of these approaches, resources employed, and describe our submitted runs and results with conclusions and future directions.