{'fraction_non_alphanumeric': 0.022137745974955277, 'fraction_numerical': 0.010137149672033392, 'mean_word_length': 4.913177611282503, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modeling human observer detection in undersampled magnetic resonance imaging (MRI) reconstruction with total variation and wavelet sparsity regularization

Purpose: Task-based assessment of image quality in undersampled magnetic resonance imaging (MRI) provides a way of evaluating the impact of regularization on task performance. In this work, we evaluated the effect of total variation (TV) and wavelet regularization on human detection of signals with a varying background and validated a model observer in predicting human performance. Approach: For the human observer studies we used two-alternative forced choice (2-AFC) studies with a small signal known exactly (SKE) task but with varying backgrounds for fluid-attenuated inversion recovery (FLAIR) images reconstructed from undersampled multi-coil data. We used a 3.48 undersampling factor with TV and a wavelet sparsity constraints. The sparse difference-of-Gaussians (S-DOG) observer with internal noise was used to model human observer detection. That S-DOG model was used to predict the percent correct of human observers for a range of regularization parameters. Results: We found that the human observer detection performance remained fairly constant for a broad range of values in the regularization parameter before decreasing at large values. Using the TV, wavelet sparsity, or a combination of the constraints did not meaningfully improve task performance. The model observer tracked the performance of the human observers as the regularization was increased but over-estimated the PC for large amounts of regularization for TV and wavelet sparsity, as well as the combination of both parameters. Conclusions: For the task we studied, the S-DOG observer was able to reasonably predict human performance with both total variation and wavelet sparsity regularizers over a broad range of regularization parameters. We also did not find a large improvement in task performance due to regularization.


Introduction
Task-based assessment of image quality 1, 2 for reconstructed MRI images is critical to the development and validation of accelerated reconstruction techniques. This approach has been extensively used in other imaging modalities [3][4][5][6][7][8][9][10][11][12] There has been research using task-based techniques in MRI for parallel imaging 13 and compressed sensing 14 but most methods utilized for assessment of image quality in MRI are typically root mean square error (RMSE) and structural similarity index (SSIM).
These are measures of pixel value differences between the original and reconstructed image. 15,16 While these measures do give an indication of similarity between images, neither RMSE nor SSIM take into account the specific task for which the image will be used. As a result, images with the same RMSE and SSIM could produce different performance in these tasks. Observer models are an alternative way to assess image quality by taking into account human visual principles as well as the task for which the image will be used. [17][18][19] Previous results related to evaluating MRI reconstruction of undersampled images using an approximation to the ideal linear observer [20][21][22] showed that the area under the ROC curve (AUC) for the channelized Hotelling observer with Laguerre Gauss channels showed only a small improvement with regularization. However, both SSIM and RMSE showed a large improvement. Our preliminary results using human observer studies 23,24 have shown a similar result in that there was no large improvement with regularization for this task. The purpose of this work is to evaluate constrained reconstruction of undersampled MRI data using TV, wavelet, and a combination of these constraints based on human observer performance in detecting a small signal in a 2-AFC task.
A model observer (S-DOG) is used to model human observer performance in this detection task.
Along with optimizing reconstruction, a goal of this work is also to reduce the number of future human observer studies needed in undersampled MRI reconstruction by using model observers to evaluate image quality.

Undersampled acquisition in MRI
For this study we consider 1-D undersampling of FLAIR images ( Figure 1) with an acquisition that samples every fourth phase encoding line plus fully sampling the middle 16 kspace lines resulting in an effective undersampling factor of 3.48. Data used in the preparation of this article were obtained from the NYU fastMRI Initiative database. 25 As such, NYU fastMRI investigators provided data but did not participate in analysis or writing of this report.

Constrained Reconstruction from multi-coil Data
Constrained reconstruction minimizes a data agreement functional with additional constraints. For this work we consider a total variation and wavelet constraint 16 and multi-coil parallel imaging using SENSE 26 with the coil sensitivities estimated using the sum of squares method which leads to real estimates of real-valued objects.
The following functional was minimized when reconstructing these images: where H(f ) is the undersampled Fourier operator acting on the discretized object f , the Fourier data is g, Ψ is the Daubechies 4 wavelet transform, ||∇f || 1 is the L 1 norm of the gradient which is the total variation operator and α Ψ and α T V are the regularization parameters for the wavelet and TV constraints respectively. 16 Sample images with different amounts of TV and wavelet regularization are shown in Figure 2 and Figure 3 respectively. The reconstruction of the images was done using the Berkeley Advanced Reconstruction Toolbox (BART) toolbox. 27 As the regularization increases, there are reduced aliasing artifacts but also reduced resolution. The arrow in image A shows the location of one of the undersampling artifacts. As the regularization increases, there are reduced aliasing artifacts but also reduced resolution but with different texture than in TV regularization. The arrow in image A shows the location of one of the undersampling artifacts.

Two-alternative forced choice experiments
In each individual trial of the 2AFC experiment we presented three 128x128 pixel images: one image of an anatomical background with the signal, the signal, and one image of an anatomical background without the signal. The signal image was always in the center, and the location (left or right) of the anatomical image with the signal was randomly chosen for each trial. An example trial is shown in Figure 4. The signal location is always in the middle of the anatomical image, which makes this task a signal known exactly (SKE) with varying backgrounds and the human observer only determines whether or not it is present in the image. The arrow present in this image was not used in the observer study and is there to help the reader identify the signal. In order to see differences in performances due to regularization, the images used were subtle. These images were generated with no regularization but the artifacts are more visible in the image to the right.

Experimental procedure
For each experimental condition, 200 2AFC trials were carried out by four observers. As training, all observers repeated an initial set of 200 trials until the performance plateaued. Based on performance in the training trials, the signal amplitude was chosen so that the mean percent correct for the observers would be close to 90 percent when no regularization was used.
The observer studies were done using a Barco MDRC 2321 monitor in a dark room. The

Model Observers
The model that we used in our study was the S-DOG channelized Hotelling Observer. 17 The channel data g c is obtained by taking the inner product of the image with the channels centered at the signal. The elements g c,j of the vector g c are given by: where C j is the j th channel in the spatial domain and f is the object in the spatial domain. We used the S-DOG channels of the form in the frequency domain where k is the distance from the zero frequency, Q is the multiplicative factor of the bandwidth, σ j = σ 0 * α j , where j denotes the j th channel and σ j denotes the standard deviation of each channel. The parameters that were used for the S-DOG were Q = 2, α = 2, and σ 0 = 0.015 which were used by Abbey 17 (Figure 5).
To compute the test statistic (λ) for each image, we create a template (w) for the observer and take the inner product with the channel outputs (g): with the template being the inverse of the sample channel covariance matrix (K c ) times the mean signal difference (s): The channel covariance matrix K c is computed by taking the average of the channel covariance matrix with and without the signal. The mean signal difference is computed by subtracting the sample average of the channel outputs without the signal from the average channel outputs with the signal.
In order to match the S-DOG observer to our human data, it was necessary to add internal noise. We used an internal noise model that added noise proportional to the covariance of the variability in the image data. 17 This calibrated model with the images with no regularization was used to predict all other experimental conditions with regularization.
We utilized six regularization parameter values for each of the three regularization conditions that were generated with 4x undersampling. For TV and wavelet regularization, the parameters used were: 0, 0.001, 0.002, 2 * 10 −1 , 5 * 10 −1 , and 0.1. These values were also used when both types of regularization were combined.

Results and Discussion
The S-DOG overestimated human performance without the addition of internal noise. However, the pattern of performance as TV increased was the same as both the human observers. Once the internal noise was added, tracking the human performance can be seen with the S-DOG in Figure 6.
The S-DOG closely tracks human performance for small amounts of regularization (which would be used in practice) and over-estimates the human performance for large amounts of regularization. Similarly, the S-DOG is also able to track human observer performance for varying wavelet regularization but again overestimates performance at the largest values of regularization as seen in Figure 7. Wavelet regularization was not found to meaningfully improve human performance in this task. Lastly, the combination of wavelet and TV regularization does not lead to a large improvement in human performance for this detection task. Figure 8 shows the prediction for performance based on the S-DOG before doing the observer study where the two types of regularization were combined. In Figure 9, the predictions were validated using an observer study. The same S-DOG model was used in this plot as in Figures 6, 7, 8. In all cases, the internal noise was chosen so that the model observer matched the average human observer performance with no regularization. The S-DOG observer tracked the human observer data for the other regularization parameters. The S-DOG slightly overestimates for the largest regularization parameter.

Fig 8
The S-DOG observer predicts that even with a combination of wavelet and TV regularization, the average human observer performance does not meaningfully improve for this task and that it degrades at high levels of regularization.

Fig 9
The 2-AFC study with equal regularization parameters for the wavelet and TV constraints shows that the model observer can reasonably predict the performance for the combination of constraints and that the human observer performance does not have a large improvement in performance with the combination of these constraints. The same S-DOG model was used was matched to the average human observer data with no regularization by adjusting the internal noise.
It is difficult to quantify task performance based on subjective evaluation of image quality by looking at the images. Figure 10 show sample images with a signal for different levels of regularization. Previous work showed that the MSE and SSIM change in a meaningful way with regularization 22 but the task performance and visual assessment remain similar. As the regularization increases, there are reduced aliasing artifacts but also reduced resolution. Image quality and task performance (as measured by the average percent correct shown in the images) is difficult to assess from the sample images.

Conclusion
To our knowledge, this is the first application of a model observer for tracking human observer detection performance in undersampled MRI reconstruction. The S-DOG model observer performance followed the same pattern as the average performance of the human observers, but overestimated without addition of internal noise. However, the S-DOG was able to track human performance for TV and wavelet constraints, as well as the combination of both TV and wavelet. One of the results of this study is that model observers are able to track human observer performance as the regularization changes. We also found that commonly used regularization parameters of TV, wavelet, and combination TV and wavelet constraints do not lead to a large improvement human detection performance for this task.

Data, Materials and Code Availability
The code used for placing the signals in the raw k-space data, and running the observer studies and model observers can be found at: His research quantifies the information content of medical images from a task-based perspective.