{'fraction_non_alphanumeric': 0.030885427203452696, 'fraction_numerical': 0.014026569559646639, 'mean_word_length': 5.011349817592217, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatic Cardiac Disease Assessment on cine-MRI via Time-Series Segmentation and Domain Specific Features

Cardiac magnetic resonance imaging improves on diagnosis of cardiovascular diseases by providing images at high spatiotemporal resolution. Manual evaluation of these time-series, however, is expensive and prone to biased and non-reproducible outcomes. In this paper, we present a method that addresses named limitations by integrating seg- mentation and disease classification into a fully automatic processing pipeline. We use a UNet inspired architecture for segmentation of car- diac structures such as the left and right ventricular cavity (LVC, RVC) and the left ventricular myocardium (LVM) on each time instance of the cardiac cycle. For the classification task, information is extracted from the segmented time-series in form of comprehensive features handcrafted to reflect diagnostic clinical procedures. Based on these features we train an ensemble of heavily regularized multilayer perceptrons (MLP) and a random forest classifier to predict the pathologic target class. We evalu- ated our method on the ACDC training dataset (4 pathology groups, 1 healthy group, 20 patients per group). We achieved dice scores of 0.930 (LVC), 0.879 (RVC) and 0.873 (LVM) and a classification accuracy of 94% in a 5-fold cross-validation. Our results underpin the potential of machine learning methods for accurate, fast and reproducible segmenta- tion and computer-assisted diagnosis (CAD).


Introduction
Cardiac remodeling plays an inherent role in the progressive course of heart failure. Clinical manifestations are changes in size, mass, geometry, regional wall motion and function of the heart [1], which can be assessed timely and monitored non-invasively by cardiac magnetic resonance imaging (CMRI). In today's clinical routine, the huge benefits of comprehensive quantitative measurements are still not exploited due to the associated labour time, subjective biases and lack of reproducibility. Accurate automatic approaches for simultaneous multistructure segmentation and CAD are thus desirable assets for a large spectrum of cardiac diseases.
Convolutional neural networks (CNN) have recently shown outstanding performance in medical image segmentation [2], where they typically take the form of UNet-like architectures [3]. So far, a limited number of fully-automatic cardiac CNN segmentation methods have been proposed [4][5][6], however, they did not consider to segment all volumes of the cardiac cycle. Highly accurate multistructural segmentations on the entire cardiac cycle are crucial for automatic pathology assessment, especially when considering geometrical and dynamical changes of anatomical structures. CAD approaches originate from the field of lesion detection and classification [7], which primarily focuses on texture information to discriminate healthy from pathological tissue. To the best of our knowledge, a CAD system for cardiac remodelling pathologies and myocardial infarct patients has not been proposed before.
In this paper, we present an approach for automatic classification of cardiac diseases associated with pathological remodelling. Based on multi-structure segmentation for each time step of the CMRI, we extract domain-specific features, which are very much motivated by a cardiologist's workflow, to then train an ensemble of classifiers for disease prediction (see Figure 1). We evaluated our methods for segmentation and classification on the MICCAI ACDC data set [8] as part of the challenge's training phase and report preliminary results.

Cardiac cine-MRI Dataset
The ACDC training dataset [8] comprises short-axis cine-MRI of 100 patients acquired at the University Hospital of Dijon using two MR scanners of different magnetic strengths (1.5 T and 3.0 T). Each time-series is composed of 28 to 40 3D volumes, which partially or completely cover the cardiac cycle. As typical for CMRI, the data is characterized by a high in-plane resolution ranging from 0.49 to 3.69 mm 2 and a low resolution in the direction of the long axis of the heart (5 − 8 mm slice thickness). Note that some data exhibit severe slice misalignments, which originate from different breath hold positions between slice stack acquisitions. Manual expert segmentations for all structures of interest (LVC, LVM, RVC) on end diastolic (ED) and end systolic (ES) phase instants are provided as ground-truth. The dataset is divided into five evenly distributed subgroups (4 pathological groups and 1 group of healthy subjects): patients with previous myocardial infarction (MINF), dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy (HCM), abnormal right ventricle (ARV) and normal (healthy) subjects (NOR). Additional information for all patients is provided in form of height and weight.

Segmentation
Data preprocessing For the segmentation part of the ACDC challenge we resampled all volumes to 1.25 × 1.25 × 10 mm per voxel to account for varying spatial resolutions. The grey level information of every image was normalized to zero-mean and unit-variance.
Network architecture We designed a network architecture inspired by the 3D UNet [9], which consists of a context aggregating pathway followed by a localization pathway. Both are interconnected at various scales to allow for recombination of abstract context features with the corresponding local information. We carefully adapted the architecture to cope with specific challenges of CMRI (see Figure 2): Due to low z-resolution of the input, pooling and upscaling operations are carried out only in the x-y-plane. Context in the z-dimension is solely aggregated through the 3D convolutions. Each feature extraction block (shown in gray) consists of two padded 3 × 3 × 3 convolutions, followed by batch normalization and a leaky ReLU nonlinearity. Due to the shallow nature of the network (18 layers) no residual connections are utilized. The initial number of 26 feature maps is doubled (halved) with each of the 4 pooling (upscaling) operations, resulting in a maximum of 416 feature maps at the bottom of the U-shape. Deep supervision (as in [10]) is implemented by generating low resolution segmentation outputs via 1 × 1 × 1 convolutions before each of the last two upscaling operations, which are upscaled and aggregated for the final segmentation.
x-y-dim. To accomplish the training of a well generalizing model on limited data, we used a broad range of data augmentation techniques, such as mirroring along the x and y axes, random rotations, gamma-correction and elastic deformations. Due to the low z-resolution all data augmentation was performed only in the x-yplane. To account for the presence of slice misalignments, we artificially increased the number of misaligned slices by motion augmentation: All slices within the training batch were perturbed with a probability of 10% and a random offset drawn from N (0, 20).

Cardiac Disease Classification
Feature Extraction We extract two sets of features from the previously segmented structures to perform disease classification. All features were designed to quantify the traditional assessment procedures of expert cardiologists by describing static and dynamic properties of the structures of interest (see Table  1).

Instant features
Extracted from the two labeled ED and ES time instants as provided by the ACDC dataset, these features cover local and global shape information (circumference, circularity, LVM thickness, etc.), local variations (size of RVC at the apex, LVM thickness between RVC and LVC), simple texture descriptors (mass) as well as additional meta information (body mass index, weight, height). Notably, all thicknesses, circumferences and circularities are computed on the individual x-y-planes and aggregated over the z-dimension. The body surface is estimated from weight and height using the Mosteller formula.
Dynamic volume features We deployed the trained segmentation model to predict the anatomical structures in all time steps of the CMRI. This allows for exploitation of volume dynamics throughout the entire cardiac cycle independent of the predefined ED/ES. These volume dynamics are quantified in form of first order statistics (median, standard deviation, kurtosis, skewness) complemented by characteristics of the cardiac cycle's minimum and maximum volumes: We found the time instants of these extrema to not match the predefined ED/ES instants in the majority of patients. This finding is accounted for by computing volume, volume ratios and ejection fractions based on the determined actual minimum (v min ) and maximum (v max ) volume of the cardiac cycle. Finally, the synchrony of contraction between LVC and RVC is measured in form of the time step differences between their corresponding v min and v max .
Ensembling The features described in section 2.3 were used to train an ensemble of 50 multilayer perceptrons (MLP) and a random forest for pathology classification. The MLP's architecture consists of four hidden layers, each containing 32 units, followed by batch normalization, leaky ReLU nonlinearity and a Gaussian noise layer (σ = 0.1). Each MLP was trained on a random subset of 75% of the training data, while the remaining 25% were used for epoch selection. Further regularization was provoked by only presenting a random subset of 2/3 of the features to each MLP. We trained all MLPs for 400 epochs (with a patience of 40 epochs) using the ADAM solver with an initial learning rate of 5 · 10 −4 , decayed by 0.97 per epoch. An epoch was defined as a set of 50 batches containing 20 patients each. Additionally, we trained a random forest with 1000 trees. During testing, the softmax outputs of all MLPs were averaged to obtain an overall MLP score, which was recombined subsequently with the random forest output to obtain the final ensemble prediction.

Results
Segmentation With regard to the expert segmentations on the original ED and ES phase instants, individual dice scores of 0.930 for the LVC, 0.873 for the LVM and 0.879 for the RVC (see Table 2) were achieved in a the 5-fold crossvalidation. Note that cardiac phase instances between ED and ES were not considered in the scores due to unavailable ground truth labels. Qualitatively, the 4D segmentation yielded convincing results, which were smooth and robust in time for all substructures (see Figure 3). CMRI with slice misalignments were segmented successfully by the model when using motion augmentation. Based on the cross-validation, we observed only little overfitting, most of which occurred for the RVC region. The main mode of failure was the basal part of the RVC region, where the model struggled to distinguish between the right atrium and instant features RVC LVM LVC max thickness * x min thickness * x std thickness * x mean thickness * x std thickness of LVM between LVC and RVC * mean thickness of LVM between LVC and RVC * mean circularity * x x max circumference * x x mean circumference * x x RVC size at most apical LVM slice * RVC to LVC size ratio at most apical LVM slice * volume per m 2 body surface x x x mass x patient weight patient height patient body mass index x x x volume ratio vmin,LVC/vmin,RVC volume ratio vmin,LVM/vmin,LVC volume ratio vmin,RVC/vmin,LVM time step difference t(vmin,LVC)-t(vmin,RVC) time step difference t(vmax,LVC)-t(vmax,RVC) Table 1. The two sets of features extracted for disease classification and the corresponding cardiovascular structure (RVC, LVM, LVC). All instant features (except for additional patient information) are extracted on both ED and ES. *this feature was calculated in the x-y-plane and aggregated over slices in z. ** vmin,LVM is determined at t(vmin,LVC). the right outflow tract or the RVC. This occurred mostly in ES images, resulting in a lower average dice score of 0.834 compared to 0.925 in ED. Other failures occurred in the LVC region, where papillary muscles were anatomical correctly classified as LVM but should have been classified as LVC to meet the convention of the challenge. This was especially observed in HCM patients. Simultaneous segmentation of multiple structures in a 3D volume of size 320 × 320 × 10 voxels took 420 ms on a Pascal Titan X GPU. Hence, 4D segmentation of up to 40 time steps took less than 17 s.
Classification We trained the classification ensemble (see Section 2.3) on the ACDC training data using the features described in section 2.3. In a five fold cross-validation, a classification accuracy of 94% was achieved. The individual t=0 t=9 t=18 t=29  Table 2. Dice scores of the segmentation model after cross validation. performance of the MLP ensemble and random forest were 93% and 92%, respectively. A confusion matrix is provided in Figure 4, which shows that the proposed ensemble performed equally well in all classes. On average, 19 of 20 patients were classified correctly. Feature computation took 15 s for instant features and less than one second for the dynamic volume features. Together with the segmentation this adds up to a processing time of less than 40 s per patient.

Discussion
In this paper we presented a fully automatic processing pipeline for pathology classification on cardiac cine-MRI. First, we developed an accurate multistructure segmentation method trained solely on 3D ED and ES phase instances, but capable of processing the entire cardiac cycle. The model is robust against slice misalignments, different CMRI protocols as well as various pathologies.
Comparing the results to the model recently proposed by Zotti et al. [4], we achieve only marginally lower dice scores. Considering that their model was trained and evaluated on the entire ACDC data set (100 training + 50 test patients instead of a cross-validation on 100 training patients), we expect our scores to improve once entering the test phase of the challenge. Furthermore, our experimental setup does neither require shape priors nor an elaborate training scheme which makes it easier to adopt in practice. Based on the segmentations generated by our model, geometrical features are extracted and utilized by an ensemble of classifiers to predict the diagnosis, yielding promising outcomes (94% accuracy). This fully automatic processing pipeline constitutes an attractive software for clinical decision support due to the visualization of intermediate segmentation maps, the comprehensive quantification of cardiologic assessment and the rapid processing speed of less than 40 s. Note that this paper presents work in progress of the ongoing ACDC 2017. Possible future improvements of the model concern data augmentation and the architecture of the segmentation network as well as a regularization objective as used in [5]. Training the pipeline end-to-end in a multitask architecture could yield further improvement.