{'fraction_non_alphanumeric': 0.028481012658227847, 'fraction_numerical': 0.003955696202531646, 'mean_word_length': 4.911214953271028, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Master 2 Internship Proposal: Using interpretability methods to explain Vision-Language models for medical applications

Recent advances in Deep Learning have led to the growth of interpretable Machine Learning, which seeks to help understand the decisions of a model. Indeed, in various fields such as Medicine, Finance and Security, it is important for models to be trustworthy and reliable. As part of this internship, we want to develop a method to explain the decisions of medical Vision-Language models. State-of-the-Art Vision-Language models such as UNITER [3] are built based on the transformer architecture to extract representations from texts and images. Those representations are then used in multimodal applications such as visual question answering [1] and image captioning [7]. However, due to the complex architecture of those models, explaining them remains a challenge. Applying interpretability methods to those models can be a way to make them more reliable. In the context of medical data, we want to be able to explain why a radiology report does or does not fit a X-ray. To this end, we will rely on medical datasets such as MIMICCXR [4]. This internship is focused on the development of interpretability methods for Vision-Language models.