{'fraction_non_alphanumeric': 0.025428042572882923, 'fraction_numerical': 0.00851457658491439, 'mean_word_length': 5.046586457750419, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adversarial Robust Training in MRI Reconstruction

Deep Learning has shown potential in accelerating Magnetic Resonance Image acquisition and reconstruction. Nevertheless, there is a dearth of tailored methods to guarantee that the reconstruction of small features is achieved with high fidelity. In this work, we employ adversarial attacks to generate small synthetic perturbations that when added to the input MRI, they are not reconstructed by a trained DL reconstruction network. Then, we use robust training to increase the network's sensitivity to small features and encourage their reconstruction. Next, we investigate the generalization of said approach to real world features. For this, a musculoskeletal radiologist annotated a set of cartilage and meniscal lesions from the knee Fast-MRI dataset, and a classification network was devised to assess the features reconstruction. Experimental results show that by introducing robust training to a reconstruction network, the rate (4.8\%) of false negative features in image reconstruction can be reduced. The results are encouraging and highlight the necessity for attention on this problem by the image reconstruction community, as a milestone for the introduction of DL reconstruction in clinical practice. To support further research, we make our annotation publicly available at https://github.com/fcaliva/fastMRI_BB_abnormalities_annotation.


Introduction
Deep Learning (DL) is certainly a candidate in replacing traditional medical imaging reconstruction techniques in the near future. In a recent retrospective study, Recht et al. (2020) suggests that clinical and (up to 4×) DL-accelerated images can be interchangeably utilized. Nevertheless, results of that study demonstrated that clinically relevant features can lead to discordant clinical opinions. This adds to the concerning results presented at the Fast-MRI challenge, which was held during NeurIPS 2019 1 : top performing deep learning models failed in reconstructing some relatively small abnormalities such as meniscal tear and subchondral osteophytes. We argue that in the current state, it must be acknowledged Figure 1: Overview of the proposed approach. At each training iteration, an attacker identifies which small features are difficult to be reconstructed, given the current network parameters and the input MRI. The network parameters are then updated so that the reconstruction of the small features is encouraged.
that irrespective of a remarkable improvement in image quality of accelerated MRI, the false negative reconstruction phenomenon is still present. Such phenomenon closely relates to the instability of deep learning models for image reconstruction, which was discussed in Antun et al. (2020). Instabilities can manifest in the form of severe reconstruction artefacts caused by imperceptible perturbations in sampling and/or image domains. Otherwise, instabilities can lead to what in  defined as false-negative features: small perceptible features, which in spite of their presence in the sampled data, have disappeared upon image reconstruction. Instances of this can be tumors or lesions that disappeared in the reconstructed images. Arguably, this could be a direct consequence of the limited training data, often available in medical imaging, which could be reflected in being not representative of some pathologies. This can cause a dangerous outcome as shown in Cohen et al. (2018), where reconstructed images were hallucinated, in the sense that important features representative of tumors were not reconstructed, simply because they were not available in the training distribution. In an attempt to better explain the false negative phenomenon, in , we investigated two hypotheses: i) the under-sampling is the cause for the loss of small abnormality features; and ii) the small abnormality features are not lost during under-sampling; albeit they result to be attenuated and potentially rare. A graphical overview of the approach is shown in Fig. 1 Based on these assumptions, if an under-sampling procedure had removed the information related to a small feature available in the fully-sampled signal, a reconstruction algorithm would be able to recover that information only through other confounded structural changes, when present. This lead to a realization of the necessity for new loss functions or metrics to be adopted in image reconstruction. In practice, deep learning reconstruction models trained to only maximize image quality can fail when it comes to reconstructing small and infrequent structures, as experimentally shown. With respect to the second hypothesis, were the feature still available, although rare, learning based reconstruction method should be able to reconstruct said features; yet this is contingent upon placing the right set of prior during training and for this, adversarial training is a valid solution.
To investigate these hypotheses, False Negative Adversarial Features (FNAF) were introduced. These are minimally perceivable features synthetically injected in the input data -through adversarial attacks -so that they would mislead the reconstruction network making it unable to reconstruct those features. Subsequently, in pursuance of reconstructing the small features, a robust training approach was adopted, where the loss function would over-weight the reconstruction importance of such features. Besides further highlighting the complexity of the false-negative feature reconstruction and the instability of reconstruction methods problems,  addressed the importance of extending the FNAF search space to be representative of real-world abnormalities.
This paper extends  leading to the following contributions: • A musculoskeletal (MSK) radiologist manually annotated features that are relevant in the diagnosis and monitoring of MSK related diseases, including bone marrow edema and cartilage lesions. Bounding boxes were places in relevant regions of the validation set of the Fast-MRI knee dataset. The manual annotation will be available in https://github.com/fcaliva/fastMRI_BB_abnormalities_annotation.
• We quantitatively investigated the effects of robust training with FNAF on reconstructing real-world abnormalities present in the Fast-MRI knee dataset, using the available bounding boxes.
• We quantitatively investigated the effects of robust training with FNAF on a downstream task, such as feature abnormality classification. The experimental study shows that when a FNAF-based robust training is employed, the false negative rate in abnormality classification is reduced. Nonetheless, we argue that a more sophisticated reconstruction architecture would benefit more from the robust training.
• We further highlight the need for using features that better represent real-world features, as part of the robust training procedure, speculating that this would reduce network instability and improve reliability and fidelity in image reconstruction.

MRI Reconstruction with Deep Learning
Magnetic resonance imaging (MRI) is a first-choice imaging modality when it comes to studying soft tissues and performing functional studies. While it has been widely adopted in clinical environments, MRI has a fundamental limitation, which depends on the way data are collected: this is a sequential and progressive procedure where data points are acquired in the k-space. The higher the resolution, the more data points are needed to be sampled in order to satisfy the desired image quality. A naïve strategy to reduce scanning time is to acquire a reduced number of phase-encoding steps. However, this results in aliased raw data, which -for instance in parallel imaging -are resolved by exploiting the knowledge about the utilized coil geometry and spatial sensitivities (Glockner et al., 2005). The new paradigm in image reconstruction is that DL-based approaches are capable of converting under-sampled data to images that include the entire information content (Recht et al., 2020). According to Liang et al. (2019) and Hammernik and Knoll (2020), this is key in fast MRI. There exists multiple approaches to accomplish MRI reconstruction by means of DL. Liang et al. (2019) suggests three main categories of methods: data-driven, modeldriven or integrated. Data-driven approaches do not require particular domain knowledge and mainly rely on the availability of a large amount of data. As a result, they tend to be data hungry in their effort to learn the mapping between k-space and the reconstructed MRI. Conversely, model-based approaches mitigate the need for big-data by restricting the solution space through the injection of physics prior knowledge. Those methods, which reproduce the iterative approach of compressed sensing, belong to the model-based set. Ultimately, integrated approaches can combine positive aspects of both previous solutions.

Adversarial attacks
To apply adversarial attacks to MRI reconstruction with deep learning, it is important to understand the most studied forms of adversarial attacks, seen as the practice of adding small imperceptible perturbations to input images with the aim to mislead machine learning models. There exists a vast literature which attempts to explain adversarial examples (Goodfellow et al., 2015;Bubeck et al., 2018;Gilmer et al., 2018;Mahloujifar et al., 2019;Shafahi et al., 2019a). One notable theory by  states that adversarial examples are a consequence of data scarcity, and is linked to the fact that the true data distribution is not captured in non-sufficiently large dataset. Another profound explanation is provided by Ilyas et al. (2019), which shows that adversarial successes are mainly supported by a model's ability to generalize on standard test set by using non-robust features. In other words, adversarial examples are more likely a product of datasets rather than that of machine learning models. To make a model resistant to adversarial attacks without additional data, one could employ adversarial training and provide the model with a prior that remarks the fact that non-robust features are not useful as demonstrated in Goodfellow et al. (2015) and . These findings are orthogonal to the second investigated hypothesis: if we interpret the distribution of FNAF as the distribution of robust features, we may attribute FNAF reconstruction failure to the dataset's inability to capture FNAF's distribution.
While the majority of adversarial attacks focus on discriminative models, Kos et al. (2017) proposed a framework to attack variational autoencoders (VAE) and the VAE-GAN. Specifically, input images were imperceptibly perturbed so that the generative models would synthesize target images that belong to a different class. Although reconstruction models can be seen as generative, we differ from this body of work, mainly because we focus on generating perceptible features that perform non-targeted attacks.
Going beyond small perturbations, a set of more realistic attacks produced by 2D and 3D transformations was proposed in Xiao et al. (2018) and Athalye et al. (2018). Similarly to our work, these studies perform perceptible attacks. Arguably, the most realistic attacks are physical attacks, which in Kurakin et al. (2017) were achieved by altering the physical space before an image was digitally captured. Kügler et al. (2018) proposed a physical attack on Dermoscopy images by drawing on the skin around selected areas. Although these attacks could more easily translate to real world scenarios, it would be nearly impossible to perform physical attacks to MRIs. Recently, Chen et al. (2020) proposed a data augmentation method, which based on adversarial attacks, it introduced bias field to cardiac MRIs which in turn boosted the performance in an image segmentation task.

Adversarial attacks and robustness in medical imaging
Robustness and adversarial attack topics in machine learning have mainly been associated with security, where an adversary is assumed to attack a machine learning system. In Liu et al. (2019), the authors investigated how robust CNN for image segmentation are to visually-subtle adversarial perturbations, and showed that in spite of a defense mechanism in place, the segmentation performance on the unperturbed images was generally of higher quality than that obtained on perturbed images. In Kügler et al. (2018), an attacker is a malicious user that tries to fool a medical system by applying physical attacks. In practice, it altered Dermoscopy images by drawing on the skin. Cases like this are hard to find in real life, and if we rely on the genuine assumption that no malicious entities try to mislead diagnosis or treatment (Finlayson et al., 2019), it is easy to understand why ideas from the robustness community have found limited application in medical imaging. Nonetheless, concerns of fraud have been voiced, and it has been argued that robustness shall go beyond security. While independent and identically distributed machine learning focuses on improving the prediction outcome in average cases while neglecting edge cases, robust machine learning focuses on finding the worst case scenarios and edge cases to potentially patch them. This perspective is very useful in medical imaging analysis and applications, as the danger for each mis-prediction can be high and abnormalities can be edge cases.

Methods
The goal of this study is to enhance the quality of accelerated MRI that are reconstructed with deep learning approaches. A complete overview of the proposed approach is shown in Fig. 1. In the interest of accomplishing such task, we adversarially attacked a reconstruction network to identify small features which were difficult for that network to reconstruct. We refer to them using the acronym "FNAF", which stands for "False Negative Adversarial Features". Simultaneously, we applied ideas of robust training to make the reconstruction network less sensitive to such identified small features, and reduce the reconstruction error on said features. Ultimately, we tested whether the approach generalized to real world features by analyzing a subset of reconstructed MRI from the knee fastMRI dataset (Zbontar et al., 2018). To do so, MR images were fed into a classification network, which was purposefully trained to establish whether -upon reconstruction -the real small features were still present in the MRI.

False negative adversarial attack on reconstruction networks
Adversarial attacks aim to maximize the loss L of a machine learning model, parameterized by θ. This can be achieved by changing a perturbation parameter δ within the set S ⊆ R d of the allowed perturbation distribution . Here, S was restricted to be a set of visible small features in all the locations of an inputted MR image. Formally, this is defined as: max In principle, L could be any arbitrary loss function; we used the mean-of-squares error function, which is a common choice for image reconstruction problems. For traditional image reconstruction, the reconstruction loss is minimized so that all the features including the introduced perturbation (i.e. small features) are reconstructed. Conversely, the attacker aims to maximize Eq. 1 and identify a perturbation, which the network is not capable of reconstructing. Next, let δ be an under-sampled perturbation which is added to an under-sampled image and δ the respective fully-sampled perturbation, the objective function becomes: U can be any under-sampling function, which is comprised of an indicator function M that acts as a mask in the k-space domain, and an operator that allows for a conversion from image to k-space and vice-versa such as the Fast Fourier Transform F and its inverse F −1 . The under-sampling as well as the k-space mask M functions are the same as in the implementations provided by Zbontar et al. (2018).
As we synthetically construct the small added features, we can measure the loss value within the area where the introduced features are located to assess whether they have been reconstructed. To do so, a mask was placed over the reconstructed image and the perturbed target image, so that only the area of the small feature was highlighted. The area was relaxed to also include a small region at a distance d from the feature border. The motivation for the mask accounting for boundaries is: if only the loss of the FNAF's foreground was measured, it may not capture cases where the FNAF had blended-in with the background. Therefore, the loss was computed in a 5 pixels distance range from the boundary of the FNAF. The loss is defined as where x and y are the original and reconstructed MRIs respectively. T is an indicator function which masks over the FNAF in the fully-sampled MRI and the reconstructed images. Weights α and β are hyper-parameters, which were set to 1 and 100 during adversarial training (details in Section 4.1). This allows one to better preserve both image quality and robustness of FNAF. During the evaluation of the attack, α and β were set to 0 and 1 respectively, to guarantee that the loss value was representative of the FNAF reconstruction. The loss can be maximized by either random search (RS) or finite-difference approximation gradient ascent (FD). Under random search settings, random shapes of features δ are placed at random locations in the image, and the δ which maximizes the loss in Eq. 2 is identified. As demonstrated in Bergstra and Bengio (2012), random search is an effective optimization technique. Nevertheless, the location of the δ feature is crucial when looking for finding FNAF. Unfortunately, the (x, y) coordinates of δ are non-differentiable. This limitation can be addressed by employing the finite central difference, which makes it possible to approximate the partial derivatives for each parameter p and optimize the low-dimensional non-differentiable parameters to update the location parameter and maximize Eq. 2 via gradient ascent.

Under-sampling information preservation verification
A benefit of having a synthetic feature generator is that one can quantify the amount of preserved information after k-space under-sampling. To guarantee that the information of δ was preserved irrespective of under-sampling in the k-space, the following condition needs to be satisfied: where D is a distance function, and is a noise error tolerance threshold; x + δ and x obey: as U is linear and closed under addition. MSE is used for D.

Robust training with False Negative Adversarial Features
Our attack formulation allows the reconstruction models to simultaneously undergo standard and adversarial training. In contrast, small perturbations-based adversarial training approaches require the models to be trained only on robust features . This allows one to do FNAF-robust training on a pre-trained model and speed up convergence. To accelerate training, we adopted ideas from Shafahi et al. (2019b). Briefly, to conduct FNAF-robust training, the model utilized a training set which included original and adversarial examples, including the examples that were generated during the search for the worst adversarial case. The inner maximization can be performed by either random search or finite-difference approximation gradient ascent -described above. In our experiments, we opted for random search as it proved to be a more effective form of attack. Intuitively, random search reduces the implementation to be a data augmentation approach.

Generalization to real-world abnormalities
There exists multiple possible strategies when it comes to analyzing how well the proposed approach can generalize on reconstructing real-world abnormalities. We aimed to answer the following question: "How well does a robust training approach -which encourages the reconstruction of FNAF -improve network's performance on reconstructing real (rare or common) small features?"

Qualitative study
In , a qualitative approach was pursued, where a MSK radiologist inspected and identified abnormalities of clinical relevance in 51 volumes from the validation set. A complete list of the abnormalities identified in the dataset is reported in Appendix 5. These include meniscal tears and degeneration as well as cartilage lesions. Upon visual inspection, the MSK radiologist advocated for a marginal improvement in the reconstruction of cartilage lesions, and meniscal tears which can be attributable to the FNAF robust training. Numerically, while a baseline UNET was able to reconstruct 1 out of 8 cartilage lesions and 8 out of 9 meniscal lesions, the introduction of robust training somewhat improved the reconstruction of such features: 40% and 100% of these features where respectively successfully reconstructed. We understand that this investigative approach could be biased by human cautiousness, therefore in this paper we followed a quantitative experimental approach.

Quantitative study
Abnormalities present in the part of the training set and the whole validations set of the Fast-MRI dataset were manually annotated by means of bounding boxes. The benefits of producing such annotation are three-fold. Bounding boxes allow for restricting the region in which the reconstruction error can be calculated. This somehow overcomes the limitation of global evaluation metrics which merely provide an 'average' evaluation of the reconstruction quality, which in the first place, might not be representative of the reconstruction accuracy in the small regions that contain abnormalities. Furthermore bounding boxes allow for implementing an unbiased tool (i.e. a classifier capable of assessing the presence or absence of the abnormalities in the reconstructed image, making it possible to fairly compare multiple reconstruction approaches. Ultimately, we make the annotation publicly available, to encourage and facilitate further research on the topic of robustness in medical image reconstruction as well as abnormality classification in MSK imaging.

Fast-MRI dataset annotation
A total of 418 MR exams from the knee Fast-MRI dataset were annotated, 321 and 97 exams from the training and validation sets respectively. The exams included coronal knee sequences, proton density-weighted with and fat suppression. For more details about the acquisition parameters, interested readers are invited to refer to Zbontar et al. (2018). Each MRI volumes were examined for cartilage, meniscus, bone marrow lesions (BML), cysts, and based on the Whole Organ Magnetic Resonance Scoring (WORMS) scale Peterfy et al. (2004) by the MSK radiologist involved in the study. Bounding boxes were placed for cartilage and BML in 4 sub-regions of the knee at the tibio-femoral joint; medial and lateral femur, medial and lateral tibia. Cartilage lesion were defined as partial or full thickness defect observed in one or more slices extending to include any breadth. Bone marrow lesions annotated included any increased marrow signal abnormality adjacent to articular cartilage in one or more slices, at least encompassing 5% of the articular marrow region. Similar bounding boxes were placed in the two sub-regions of meniscus: medial and lateral. Meniscal lesions were defined to include intrasubstance signal abnormality, simple tear, complex tear or maceration. Given the sparse occurrence of the cysts, a single bounding box label was used to encompass all encountered cystic lesions in any of the sub-regions. Fig. 2 is exemplary of the performed annotations. Specifically, the MRI on the left-hand side, displays the presence of cartilage lesions in the medial tibial (red) and femoral (purple) compartments. In the middle a cartilage lesion in the lateral femur is marked by a golden color-coded bounding box. On the right-hand side a map representative of the distribution location of abnormalities in the training set is shown.

Experimental setup
We conducted our experiments on the Fast-MRI knee dataset with single-coil setting, including 4× and 8× acceleration factors (Zbontar et al., 2018). We evaluated our methods with two other deep learning based reconstruction methods, U-Net Ronneberger et al. (2015) -a popular baseline for MRI reconstruction, and invertible Recurrent Inference Machines (I-RIM) (Putzky and Welling, 2019) -which is the winner entree to the single-coil Fast-MRI challenge. For U-Net, we followed the training procedures described in Zbontar et al. (2018). For I-RIM we followed the training procedures described in  and used the official released pre-trained model.

FNAF implementation details
FNAF were constrained to include 10 connected pixels. Attack masks were placed within the center of a 120×120 crop of the image, to ensure that the added features were small enough as well as placed in reasonable locations. For random search, 11 randomly shaped FNAF were generated at random locations for each samples in the validation set and the highest adversarial loss was recorded. With respect to the finite-difference gradient ascent, the search for the x and y coordinates where to place the FNAF's centers happened in 2 iterations. The iteration number was chosen to satisfy a reasonable computation time and keep the number of forward passes for one sample somewhat constant for both methods. The FD step size h was set to 10 and the learning rate to 1E5. Attacks were rejected when the information-preservation (IP) loss was less than 1E − 4. This was particularly important for FNAF-robust training, as it help preventing from hallucinating the reconstructed images by adding non-existing features.
With regard to FNAF-robust training, the data augmentation approach described in Section 4.1 was followed. The adversarial loss of Eq. 5 was used. In Eq. 5, beta was set to 100 to encourage the reconstruction of the introduced features. To prevent from overfitting on the FNAF attack successes, the selection of the model (to be attacked) was done by choosing the model which showed the best reconstruction loss on the validation set, while the adversarial loss was ignored.

Attack evaluation metrics
The average attack loss for the validation set and the attack hit rate were computed. The average attack loss is defined in Eq. 5. An attack is considered to be a hit when the loss is higher than a threshold value γ, which was empirically set to 0.001, as it observed the FNAF disappeared when the loss value was greater than 0.001. It is worthed mentioning that the hit rate wes set to be conservatively low. To achieve this effect, γ was set at a high value, and as a consequence there may be cases in which, despite L ≤ γ, the FNAF were lost. We speculate that the actual hit rate is likely to be higher than the value reported in the current work.

Abnormality classification
Performing anomaly classification at the image level, can hardly discriminate whether ALL clinical abnormalities have correctly been reconstructed. This is especially true, when an image includes more than one abnormality as per the case in the left of Fig. 2. A good way to provide a less coarse analysis is to perform it in a patch-based fashion. This has the drawback of introducing severe class imbalance, therefore we performed adaptive patch sampling based on the distribution of abnormality in the training set as shown in the right of Fig. 2. Two configurations of patch-size were explored, namely 32 × 32 and 64 × 64. With respect to the 32 × 32 patches, these were created by utilizing a sliding window with stride 2; whereas for the 64 × 64 patches a stride of 8 pixels was adopted. The available annotation allowed to define whether a patch comprised abnormalities or no.
To further enforce class balance the number of normal patches was capped to the number of abnormal patches, by randomly selecting normal patches within the pool of normal patches. As a result the two datasets comprised of: 103652 train/2552 validation data points in the 32 × 32 patch dataset, and 7644 train/2110 valid data samples in the 64 × 64. Importantly, since we do not have access to patients information in the Fast-MRI dataset, to prevent any form of data leakage across data splits, we pertained the original datasets training and validation splits, refraining from creating a test split, as potential data-leakage could lead to an overoptimistic classification performance.
Squeezenet (Iandola et al., 2016) pretrained on the Imagenet dataset was the network of choice. The selection was driven by two factors: while providing excellent classification performance, Squeezenet has a small footprint by design, and is fully convolutional. This means that it can accept as its input, images of any size. This, without requiring major architectural modifications. Furthermore, limited memory footprint and flexibility to inputs of various dimensions, are particularly appealing for us, in the prospective of a future combination of image reconstruction and abnormality detection in an end-to-end fashion. Two fine-tuning strategies were tested, in one all the layers pre-trained on Imagenet with exception for the last classification convolutional layer were frozen. The last -namely the classification layer -was replaced by two 1 × 1 convolutional layers, the first produced 512 feature channels, the second provided two features channels, which were activated with a ReLU activation function and fed into a Global Average Pooling to provide the logits subsequently inputted to a SoftMax activation function. The last is a function that converts an N -dimensional vector of arbitrary real values to a N -dimensional vector of real values in the range [0, 1] summing to 1. In this case, N = 2 and the obtained values can be interpreted as probability values that a particular image includes an abnormality or not. In the second fine-tuning strategy, all the weights in the modified version of the Squeezenet were fine-tuned. Ultimately, for completeness we also trained a Squeezenet from scratch.
Classification networks were trained and validated on patches extracted from the original Fast-MRI dataset. At test time, patches extracted from the Fast-MRI dataset as well as those extracted from images reconstructed by using a baseline UNet and a Unet trained with FNAF robust training were fed to the already trained networks.
For fine tuning, a binary cross-entropy loss was minimized, using stochastic gradient descent as the optimizer with learning rate as 1E − 3 (0.1 decay rate), weight decay 5E-4 and 128 samples per batch. Early stopping was implemented when no classification improvement in the validation set were observed for 60 epochs, with the maximum number of epochs set to 100.

Results and Discussion
Examples of the FNAF are shown in Fig. 3. The results of the attack are reported in Table 1 and confirm that hypothesis 2 is true in many cases. The attack with FD is weaker than that with RS, which is counter-intuitive. Many factors might contribute to this result, including tuning the optimizer hyper-parameters and the number of iterations. Nonetheless, the high success rate of the random search method for both models showed that it is fairly easy to find a FNAF in the search space that was heuristically defined. Although I-RIM was more resilient to the attacks as opposed to the baseline U-Net, the attack rate was still fairly high. This is concerning but somewhat expected since deep learning methods are not explicitly optimized for such objective, so these FNAF are at the tail-end of the distribution or even out-of-distribution with respect to the training distribution. Fortunately, we can modify the objective as specified in Section 3.3 to produce a FNAF-robust model which appears  to be resilient to the attacks. Furthermore, this shows a minimal effect in the standard reconstruction quality, which is reported in Table 2.

Under-sampling information preservation verification
To investigate the second hypothesis, the acceptance rate of the adversarial examples was measured. The information-preservation loss reported in Table 3 shows that a very high acceptance rate was seen across all the tested settings. This proves that in most cases the small feature's information was not completely lost through under-sampling. Fig. 4 shows a small negative correlation between IP loss and FNAF loss. This is consistent with our expectations as more information would weaken the attacks. Nevertheless, such negative correlation is weak, indicating that there is not a strong association. Therefore the preservation of information alone cannot predict the FNAF-robustness of the model. So the information loss due to under-sampling is a valid but insufficient explanation for the existence of FNAF.

Transferability of adversarial features across reconstruction networks
We analyzed the location distribution of the worst case FNAF and observed the absence of a well defined pattern in the FNAF's location . However, the location distribution was similar across non-FNAF-robust models. This allows us to investigate a potential transferability of adversarial features across reconstruction networks. FNAF examples from U-Net were taken and applied to I-RIM. As a result a 89.48% attack rate was observed. The high transferability is similar to what is observed in Goodfellow et al.   (2015) and Alcorn et al. (2019). This can be interpreted as the proof that the training data did not capture the FNAF distribution.

Generalization to real-world abnormalities
To measure how well the annotated abnormalities were reconstructed, the bounding box regions in the reconstructed images were compared against those in the respective fullysampled MRI from the Fast-MRI dataset. Pixel-wise MSE was the adopted metric. MSE was preferred over other whole-image-quality metrics such as SSIM due to the variety of bounding boxes size (number of pixels) that characterized the abnormality regions. Table 4 shows that for 8× accelerations, the FNAF-robust U-Net outperformed the U-Net baseline in 8 out of the 11 classes. This holds true for the total average value. Nevertheless, FNAFrobust U-Net shows no improvements in the 4× setting. A trend in Fig. 5 shows that the reconstruction quality increases (loss decreases) as the size of abnormal region increases. The same consideration applies to both methods, which confirms the intuition that smaller abnormalities are harder to reconstruct. Fig. 6 shows that for the 8× acceleration, the improvement of FNAF-robust U-Net over the baseline increases as the size of bounding box decreases, which preliminarily validates the objective of FNAF-robust training to better reconstruct smaller abnormalities, while it shows that both methods are similar in the 4× setting.

Presence of clinically relevant features classification
With respect to the classification of abnormalities within the patches that were extracted from the reconstructed MRIs, a Squeezenet where only the final classification layer was finetuned showed superior performance compared to the other two training settings, namely fine-tuning of all layers and and training from scratch. The results for the better performing classification model are reported in Table 5 and Table 6 where it is shown that the FNAF robust training approach contributed to enhancing the reconstruction quality of clinically Figure 5: Correlations between reconstruction loss and the size of abnormalities relevant features. This is represented by a reduction in the number of false negative features in the classification of real abnormalities. We inspected the prediction probabilities (normal/abnormal patch) on false negative cases in images reconstructed with UNET and the proposed approach, and interestingly observed that for the network, which consumed 32×32 patches, in 61% of the cases the classifier's prediction was less confident when the reconstruction was performed using FNAF robust training as opposed to the baseline UNET. Our interpretation for such decrease in classification confidence, is that FNAF robust training proved an added value to the reconstruction. Furthermore, we suggest that this decrease in confidence is more prevalent on smaller patches as opposed to the 64×64 patches, because the small abnormalities are mainly local features, which when analyzed on larger patches might be smoothed out.
It is worth mentioning, that a drop in classification performance was expected when the input image was DL reconstructed. However, we assumed that such drop in performance would mainly be associated to a covariate shift which comes from subtle changes in the pixels , which according to the abnormality classifier was reconstructed after the proposed FNAF robust training approach (i.e. true positive feature) (C). The difference image between the two DL reconstructions is shown in (D). Before reconstruction, the input MRI was 4× undersampled.  Obviously, this is hard to prove, mainly because errors propagate from the under-sampling and zero-filling in k-space, to the reconstruction error, up to the actual classification error per-se, which might in turn not exclusively depend on errors in the reconstruction.
Overall, we advocate that the marginal improvements may be explained by the semantic difference between FNAF and real-world abnormalities, although it certainly requires further investigation. Ideally, we want to construct the space of FNAF to be representative of not only the size but also the semantics of real-world abnormalities. Possible direction for improving FNAF and making them more realistic include: i) relax the pixel constraint more so that the FNAF space can include real-world abnormalities; ii) model the abnormality features by introducing domain knowledge. A recently published work, Chen et al. (2020) suggests that by introducing domain knowledge in the adversarial attacks one could improve reconstruction performance but also boost performance in downstream tasks.
An exemplary effect of FNAF robust training in the reconstruction of real features is visually shown in Fig. 7. A coronal section of knee MR (fully-sampled MRI) from the validation set is depicted in (A). This is an MRI from the Fast-MRI dataset which was utilized as image reconstruction ground truth, as well as an input in the validation of the abnormality classification network. In (A), the bounding box highlights the presence of a bone marrow lesion at the lateral edge of the femur, based on the radiologist's manual annotation. From an accurate analysis, it can be observed that the bounding box includes a portion of the adjacent joint fluid with high signal to test for confounding. In the reported experiment, prior to reconstruction, MRIs were undersampled at a 4× AF. In (B), the reconstruction result by using a baseline UNET is displayed. This shows that UNET recreated parts of the BML features in the corresponding region of the femur. Despite that, the lesion was not detected by our lesion classification model, making this a False Negative case. In (C), the resulting reconstructed image with the proposed robust training approach, appears to have better reconstructed BML features in the corresponding region, and in turn, it is identified by the lesion detection model, correctly making it a True Positive case. Ultimately in (D), a difference map between UNET and FNAF reconstructions is shown. Radiologist's opinion propounds that FNAF robust training allowed for a finer reconstruction of particular features, which better mimic the ground truth and that are crucial for a precise clinical interpretation. The region of interest is indicated by the gray arrow in (D). To summarize, our experiments show that by training on our imperfect FNAF, one can force convolution filters to be more sensitive to small features. We speculate this is the reason for the ob-served marginal real-world improvements, and it is indicative of a promising direction to move forward to improve clinical robustness.

Conclusions
The connection between FNAF to real-world abnormalities is analogous to the connection between lp-bounded adversarial perturbations and real-world natural images. In the natural images sampled by non-adversary, lp-bounded perturbations most likely do not exist. But their existence in the pixel space goes beyond security, as they reveal a fundamental difference between deep learning and human vision (Ilyas et al., 2019). Lp-bounded perturbations violate the human prior: humans see perturbed and original images the same. FNAF violate the reconstruction prior: an algorithm should recover (although it may be impossible) all features. We relaxed this prior to only small features, which often are the most clinically relevant. Therefore, the failure of deep learning reconstruction models to reconstruct FNAF is important even if FNAF might not be representative of the real-world abnormalities. Lp-bounded perturbations inspired works that generate more realistic attacks, and we hope to bring the same interest in the domain of MRI reconstruction.
This work expanded  to show the possibility of translating adversarial robust training with FNAF to real world abnormalities. For this, a MSK radiologist annotated the Fast-MRI dataset by placing bounding boxes on top of abnormalities observed in the menisci, cartilage and bone in knee MRIs. Subsequently, a fully convolutional neural network (i.e. Squeezenet) was employed to assess whether the abnormalities had been reconstructed by the deep learning reconstruction methods. This was addressed as a patchbased image classification problem and demonstrated that our robust training approach contributes to a reduction in the number of false negative features.
In the big picture, we investigated two hypotheses for the false negative problem in deep-learning-based MRI reconstruction. By developing the FNAF adversarial robustness framework, we show that this problem is difficult, but not impossible. Within this framework, there is potential to bring the extensive theoretical and empirical ideas from the adversarial robustness community, especially in the area of provable defenses Wong and Kolter (2018);Mirman et al. (2018); Raghunathan et al. (2018); Balunovic and Vechev (2020) to tackle the problem. It is important to remark that in this work, we only applied FNAF robust training to a baseline UNET; Subhas et al. (2020) showed that the size of a reconstruction network highly affects the output quality, specifically whether small diagnostic features are reconstructed. We expect that a more powerful model would benefit more from the robust training we proposed.
We suggest that our work gives further credit for the necessity of new loss functions or metrics to be adopted in image reconstruction, which would be more aware of finer details, as per the clinically relevant small features. Furthermore, we hope this will be a starting point for more research in the field, so that a more realistic search space for the FNAF could be identified. This would certainly enhance the generalization capabilities of FNAF to realworld abnormalities. We made the bounding box annotations, as well as the qualitative findings from the MSK radiologist involved in the study, publicly available (Appendix 5) so that they can serve as a validation set for future body of work.