{'fraction_non_alphanumeric': 0.024578222660839324, 'fraction_numerical': 0.011513698952018422, 'mean_word_length': 4.974171813587872, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Hospital Readmission Prediction - Applying Hierarchical Sparsity Norms for Interpretable Models

Hospital readmissions have become one of the key measures of healthcare quality. Preventable readmissions have been identified as one of the primary targets for reducing costs and improving healthcare delivery. However, most data driven studies for understanding readmissions have produced black box classification and predictive models with moderate performance, which precludes them from being used effectively within the decision support systems in the hospitals. In this paper we present an application of structured sparsity-inducing norms for predicting readmission risk for patients based on their disease history and demographics. Most existing studies have focused on hospital utilization, test results, etc., to assign a readmission label to each episode of hospitalization. However, we focus on assigning a readmission risk label to a patient based on their disease history. Our emphasis is on interpreting the models to improve the understanding of the readmission problem. To achieve this, we exploit the domain induced hierarchical structure available for the disease codes which are the features for the classification algorithm. We use a tree based sparsity-inducing regularization strategy that explicitly uses the domain hierarchy. The resulting model not only outperforms standard regularization procedures but is also highly sparse and interpretable. We analyze the model and identify several significant factors that have an effect on readmission risk. Some of these factors conform to existing beliefs, e.g., impact of surgical complications and infections during hospital stay. Other factors, such as the impact of mental disorder and substance abuse on readmission, provide empirical evidence for several pre-existing but unverified hypotheses. The analysis also reveals previously undiscovered connections such as the influence of socioeconomic factors like lack of housing and malnutrition.


External cause status
Activity Railway Accidents . . . on readmissions [10]. Most interesting findings from our study reveal surprising connections between a patient's non-disease background and the risk of readmission. These include behavioral patterns (mental disorders, substance abuse) and socio-economic background.
We believe that such findings can have a significant impact on how healthcare providers develop effective strategies to reduce readmissions. At present, the healthcare efforts in this context have been two fold. First is the effort to improve the quality of care within the hospital and the second is to develop effective postdischarge strategies such as telephone outreach, community-based interventions, etc. The results from this study inform the domain experts on both fronts.
Organization The rest of the paper is organized as follows. We review existing literature on readmission prediction in Section 2. We describe the data used for our experiments in Section 3 and formulate the machine learning problem in Section 4. We discuss the classification methodology in Section 5. The results are presented in Section 6. The analysis of the resulting model in the context of readmissions is provided in Section 7. We present analysis of experiments done on subpopulations corresponding to major chronic diseases in Section 8.

Related Work
Coincident with the rising importance of readmissions in reducing healthcare costs, there have been several papers that use clinical and insurance claims information to build predictive models for readmissions. We refer the readers to a recent survey on the topic [12] for a comprehensive review. Most of these models use machine learning models such as Logistic Regression [8,4,7,15] and Support Vector Machines [19]. Futoma et. al [8] provide a comparison of several machine learning methods including Logistic Regression, SVM, Random Forests for readmission prediction using features such as the diagnosis codes, procedure codes, demographics, patient's hospitalization history. However, the focus of most of these papers has been on improving accuracy of the classifier and not on interpreting the models to improve the understanding of the readmission problem. Moreover, many of these studies have either focused on a specific patient cohort or patient data from one or few hospitals [19]. For example, there have been several studies that focus on patients with acute heart conditions [2]. Recently, healthcare community has begun to study the impact of behavioral and socioeconomic factors on readmissions [11]. However, none of the data driven predictive models exploit this aspect, primarily because such data is challenging to obtain. However, in this paper we show that the diagnosis codes in the claims data contains valuable non-disease information about the patient which can be leveraged to better understand the readmission issue. Finally, the hierarchical relationship has never been exploited for building predictive models for readmission. Singh, et. al, [17] have presented a similar approach in the context of predicting disease progression, however, the authors focus on using the disease hierarchy to come up new features that are fed into the classifier.

Data
The data is obtained from the New York State Medicaid Warehouse (MDW). Medicaid is a social health care program for families and individuals with low income and limited resources. We analyzed four years (2009-2012) of claims data from MDW. The claims correspond to multiple types of health utilizations including hospitalizations, outpatient visits, etc. While the raw data consisted of 4,073,189 claims for 352,716 patients, we only included the patients in the age range 18-65 with no obstetrics related hospitalizations. The number of patients with at least one hospitalization who satisfied these conditions were 11,774 and had 34,949 claims.
For each patient we have two types of information. First type of information includes demographic attributes (age, gender) and the type of insurance. The second type of information is a patient history extracted from four years of claims data represented as a binary vector that indicates if the patient was diagnosed with a certain disease in the last four years.
Insurance Plan Information: Patients covered through Medicaid insurance can enroll into one of the two plan options. First option is to enroll in a Managed Care Organization (MCO). The MCO takes care of the delivery of the medical care to the patient and gets paid per person. The other option is called Feefor-service (FFS) in which the healthcare provider gets paid for each service performed for the patient. While there has been a gradual transition from FFS to MCO style of insurance, the quality of care and costs under each plan has always been an important issue. In the context of readmissions it is important to understand how the readmission rate is impacted by the type of plan. V Diagnosis Codes: Disease information is encoded in insurance claims using diagnosis codes. The International Classification of Diseases (ICD) is an international standard for classification of disease codes. The data used in this paper followed the ICD-9-CM classification which is a US adaptation of the ICD-9 classification. Conceptually, the ICD-9-CM codes are structured as a tree (See Figure 1 for a sample 4 ) with 19 broad disease categories at level 1. The entire tree has 5 levels and has total of 14,567 diagnosis codes. While the primary purpose of ICD taxonomy has been to support the insurance billing process, it contains a wealth of domain knowledge about the difference diseases.
Readmission Risk Flag: For each patient in the above described cohort, we assign a binary flag for readmission risk. The readmission risk flag is set to 1 if the patient had at least one pair of consecutive hospitalizations within 30 days of each other in a single calendar year, otherwise it is set to 0.

Problem Statement
Given a patient's demographic information and disease history, we are interested in predicting the readmission risk (binary flag) for the patient. The problem formulation is different from many existing studies [8], where the focus is on assigning a readmission risk to a single hospitalization event. Our focus is on understanding the impact of socio-economic and behavioral factors on a readmission.
We denote each patient i as a vector x i consisting of 11,884 elements corresponding to 11,881 disease codes and three elements for age, gender, and plan type. Note that while ICD-9-CM classification contains 14,567 codes, only 11,881 codes are observed in the data set used in this paper. All elements in the vector are binary except for age which takes 10 possible values corresponding to 10 equal width partitions between 18 and 65. The readmission risk flag is denoted using y i ∈ {0, 1} where 1 indicates readmission risk.
From machine learning perspective our task is to learn a classifier from a training data set x i , y i N i=1 which can be used to assign the readmission risk flag to a new patient represented as x * . Note that the input vector x i is highly sparse for this data with nearly 36 non-zeros out of total 11,884 elements on an average.

Methodology
We use a logistic regression (LR) model [6] as the classifier, which, is the most widely used model in the context of readmission prediction [8]. The LR model, for binary classification tasks, computes the probability of the target y to be 1 (readmission risk), given the input variables, x as: 4 See http://www.icd9data.com/2015/Volume1/default.htm for complete hierarchy.

VI
Where β is the LR model parameter (regression coefficients). We assume that x includes a constant term corresponding to the intercept. Thus β has the dimensionality D + 1 where D = 11, 884. The model parameter β are learnt from a training data set ( ) by optimizing the following objective function: where the first term refers to the training loss and the second terms is a regularization penalty imposed on the solution; λ being the regularization parameter. Different forms of regularization penalties have been used in the past, including the widely used l 1 and l 2 norms [18]. While l 2 norm (Ω(β) = β 2 = ( j β 2 j ) 1/2 ) is typically used to ensure stable results, l 1 norm (Ω(β) = β 1 = j |β j |) is used to promote sparsity in the solution, i.e., most coefficients in β are 0.
However, l 1 regularizer does not explicitly promote structural sparsity. Given that the features used in predicting readmission risk have a well-defined structure imposed by the ICD-9 standards, we explore regularizers that leverage this structure for model learning: Sparse Group Regularizer This regularizer (also refered to as Sparse Group LASSO or SGL) assumes that the input features can be arranged into K groups (non-overlapping or overlapping) [3]. The SGL regularizer is given by: where β G k are the coefficients corresponding to the group G k . The above penalty function favors solutions which select only a few groups of features (group sparsity). For the task of readmission prediction, we divide the features corresponding to 11,881 diagnosis codes into 19 non-overlapping groups, based on the top level groupings in the ICD-9-CM classification (See Table 1). The demographic and insurance plan features are grouped into an additional group.
Tree Structured Group Regularizer This regularizer, also referred to as Tree Structured Group LASSO (TSGL), explicitly uses the hierarchical structure imposed on the features. The TSGL regularizer is given by: where G denotes the tree constructed using the hierarchy of the diagnosis codes. G i j denotes the j th node in the tree at the i th level. Thus G 0 1 denotes the root level, and so on. For readmission risk prediction, we consider a three internal level hierarchy with 1193 nodes at level 3, 183 nodes at level 2, and 20 nodes at level 1.

Results
In this section we present our findings by applying logistic regression classifier for the task of readmission prediction on the MDW data described earlier. The full data set consists of 11,774 patients with 4,580 patients with readmission risk flag as true and 7,194 patients with readmission risk flag as false. We first compare the performance of the different regularization strategies to the classification task using the F-measure (harmonic mean of precision and recall for the readmission = yes class) as our evaluation metric. We also report the area under the ROC-curve (AUC) for each classifier. For each strategy, we run 10 experiments with random 60-40 splits for training and test data, respectively. The optimal values for the regularization parameter for each strategy are chosen using cross-validation. We use the MATLAB package, SLEP [14], for the structured regularization experiments.

Comparing Different Regularizations
In the past, researchers have shown that logistic regression typically outperforms other methods for predicting readmissions. Here we compare the performance of different regularization methods discussed in Section 5. The results are summarized in Table 2 and Figure 2.
The results indicate that leveraging the structure in the features (disease codes) results in an improvement in the performance of the logistic regression   Table 2 shows a clear evidence that leveraging the structure in the disease codes allows for a better classifier to predict readmission risk. We now focus on the interpretability of the resulting model under the different regularization mechanisms. The weights for the logistic regression model learnt under the four different types of regularizers is shown in Figure 3. We first note that l2 regularizer, for obvious reasons, does not produce a sparse solution (45% zero weights), while the other three regularizers induce significant sparsity (l1 -92%, SGL -97%, and TSGL -94%). However, the structured regularizers are able to achieve structured sparsity which is consistent with the ICD-9-CM hierarchy. While SGL achieves higher sparsity, the TSGL solution "aligns" better with the ICD-9-CM hierarchy. To verify this, we measure the sparsity of the coefficients at different levels of the hierarchy, as shown in Table 3. We observe that the TSGL provides better sparsity at higher levels of the hierarchy.

Discussions
Section 6 shows that leveraging the hierarchical information in the ICD-9-CM classification improves the predictive capability of logistic regression while promoting the structural sparsity to allow for better interpretability. In this section, we study the learnt model from the healthcare perspective. The focus is to understand the factors that impact readmission risk using the coefficients of the model learnt using the TSGL constraint. As shown in Figure 3d, the model is well-informed by the ICD-9-CM hierarchy. In all, there are only 437 non-zero coefficients. We choose the top 40 diagnosis codes with highest absolute coefficients. The most important diagnosis codes are listed in Table 4. The codes are grouped by their higher order functions.
The top codes listed in Table 4 reveal several valuable insights into the issue of readmission. For instance, while the role of age is understandable (older patients tend to get readmitted more), the impact of the insurance plan supports existing belief that disease management and population health activities provided by MCO would be associated with readmissions [10].
The next group of diagnosis codes pertain to infections and complications that happen during the hospital stay of the patients. This information is especially useful for hospitals to identify the key improvement avenues in the hospital operations to reduce the readmission rates. For example, it is well known that post-operative infections result in the patients returning to the hospital shortly after getting discharged. We are able to identify the same issue through this analysis.
While the role of chronic diseases in readmissions is well-understood in the literature, our results indicate that the chronic diseases play less of role, compared to other diagnosis codes.
However, the most important findings of our study correspond to last three groups of diagnosis codes in Table 4. Diagnosis codes related to mental health related issues were some of the most important positive factors in predicting readmissions. This is a valuable insight for hospitals in creating post-discharge strategies for such patients. This includes post discharge counseling and home visits. Another key factor in readmission is substance abuse. In fact, the role of mental health and substance abuse in readmissions has long been studied in the healthcare community [13]. Many of the substance abuse issues might not be the direct cause of readmissions but might be indirect indicators of the lack of social and family support to assist patient recovery after the hospital discharge.
A direct evidence of the socio-economic effect on readmissions is given by the last set of diagnosis codes which includes lack of housing and malnutrition. These factors had very high positive weights indicating their importance. However, in most existing studies such factors were not considered, primarily because of the lack of relevant data. Through this study we discovery that insurance claims data itself contains elements that can be used to assess the socio-economic background of a patient.

Analyzing Sub-populations with Chronic Diseases
To further understand the role of various diagnosis codes in predicting readmissions in the context of major chronic diseases, we studied subpopulations of patients suffering from one of 9 diseases listed in Table 5. Most of these diseases have unique symptoms and treatments and many hospitals and other medical facilities are specialized in one of these diseases. Hence, it is important to under-    Table 6: Performance of TSGL based logistic regression classifier on chronic disease subpopulations Table 6 shows the average performance of the logistic regression classifier (using cross-validation) on subpopulations corresponding each disease code. The subpopulations are created by considering only those patients who have had at least one admission for a given chronic disease. The results match the values obtained for the entire population. Next, we study the diagnosis codes which are most important in predicting readmission risk. Generally, we observe that while a few codes are common to all disease types, there are certain disease codes which are exclusively unique to each of the subtype. The diagnosis codes that have a strong impact on readmissions, independent of the type of chronic disease the patient suffers from, include postoperative infection and suicidal ideation. On the other hand, for every chronic disease, there are certain diagnosis codes that are unique, i.e., they are not a factor in any other chronic disease. Such codes can be significant for organizations that specialize in such diseases. Some of these  codes are indicative of other chronic diseases or comorbidities. For example, for Diabetes patients, the presence of transient cerebral ischemia is an important predictor of readmission risk. Similarly, the presence of pancreatic disease codes in a COPD patient is an important risk factor. Interestingly, the unique diagnosis codes identified for each of the major chronic diseases primarily contain other chronic disease related codes, thereby indicating a strong impact of comorbidity in readmissions.

Conclusions
In the last decade, there have been numerous studies that link factors pertaining to a patient's hospital stay to the risk of readmission. However most studies have been on a focused cohort, limited to one or few hospitals. However, we show here that similar results can be achieved using claims data, which has fewer elements but provides a large population coverage; the entire state of New York for this study. Even with the large volume of data, the predictive algorithms are not accurate enough (∼ 0.60 F1-score) to be used as decision making tools. However, model interpretation can reveal insights which can inform the strategies for reducing and/or eliminating readmissions.
A patient's disease history is typically expressed using diagnosis codes, which can take as many as 18000 possible values, with many more possibilities in the next generation ICD-10 disease classification. With so many possible features, ensuring model interpretability is a challenge. However, using structured sparsity inducing models, such as the tree sparse group LASSO, used in this paper, one can ensure that the truly important factors can be identified. In this case study, we discover several such interesting factors.
In particular, we conclude that while in-hospital events such as infections are important, behavioral factors such as mental disorders and substance abuse and socio-economic factors, such as lack of housing or malnutrition at home are equally important. Targeted strategies, such as phone calls and home visits, will need to be developed to handle such situations. In Section 8, we analyze subpopulations specific to chronic diseases and show that similar methodology can reveral disease specific factors for readmissions.