{'fraction_non_alphanumeric': 0.03286934950385888, 'fraction_numerical': 0.0172271223814774, 'mean_word_length': 4.4303086997193635, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
GOLDmineR: improving models for classifying patients with chest pain.

The laboratory is dealing with reporting tests as information needed to make clinical decisions. The traditional statistical quality control measures which assigns reference ranges based on 95 percent confidence intervals is insufficient for diagnostic tests that assign risk. We construct a basis for risk assignment by a method that builds on the 2 x 2 contingency table used to calculate the C2 goodness-of-fit and Bayesian estimates. The widely used logistic regression is a subset of the regression method, as it only considers dichotomous outcome choices. We use examples of multivalued predictor(s) and a multivalued as well as dichotomous outcome. Outcomes analyses are quite easy using the ordinal logit regression model.

This study re-examines the approach to evaluating risk in the patient who presents to the emergency department with chest pain or other symptoms requiring the "rule-out" of acute myocardial infarction (MI)e. Goldman et al. [1] developed an algorithm using recursive partitioning and amalgamation for assessing chest pain based on clinical findings and EKG without use of a laboratory test, which proved unworkable as a sole tool for decisionmaking because the "posterior risk" was too high. Emergency Medicine physicians are faced with costly liability for these decisions and expect the error to be at a probability near 1 percent. However, the risk is substantially reduced by adding cardiac troponin (TnT) as a necessary step to relieve uncertainty. The Goldman algorithm, creatine kinase MB isoenzyme, and TnT taken at different times are used to classify patients with the tests ranked from most to least important in classifying the data taken in descending order. The classification is optimal, taking into consideration the Goldman algorithm is only optimized for rule-out MI. We also find that the TnT is as effective at 3 hours after initial presentation as the traditional creatine kinase MB (CKMB), measured sequentially for 12 hours.  Magidson (Statistical Innovations, Inc. Belmont, Massachusetts) (SPSS GOLDminer) has introduced a method for predicting an ordered response using multiple variables. In the case of the single variable the method is based on a regression of the predictor on the dependent variable and the predictor values are scaled. The model fit is measured by the chi-square (X2) for the regression, and the odds-ratios are calculated from the logit model of the response. A graphical presentation is generated, but the odd-ratios for each class is also available in table format. It is possible to compare the effect of variables used to classify patients, and to compare the variable combinations. The method is illustrated using chest pain, EKG features, creatine kinase MB, and troponins to assign patients classified into myocardial infarction, unstable angina, and other clinical findings. Table 1 is a multivariable model to predict myocardial infarction using chest pain and risk factors, then adding the laboratory tests to provide a definitive assessment of risk. The R2 and Phi (1) are the best measures of model fit. The X2, p-value, R2 and 4 for TnT at 3 hours are close to that at 12 hours.

TROPONIN-T RANDOMIZED TRIAL
Bridgeport Hospital carried out a randomized prospective trial of emergency department (ED) patients seen for symptoms suspicious for MI. The trial intended to determine whether a test for heart attack, cardiac troponin-T (Roche Diagnostics, Indianapolis, IN), has an effect on early release of patients without MI. Nine hundred and three consecutive patients who presented to the ED with a "rule-out" MI were entered into the study, and half were evaluated only by the standard CKMB testing protocol. The other half had the standard protocol and cardiac TnT. Eight hundred and sixty-six patients remained after patients with chronic renal failure, cardioversion, and those with ST segment elevation were excluded from the study. Tests were done at time of presenting, 3 and 12 hours later. The data collected on each patient included: age, sex, past medical history, risk factors, EKG findings, characteristic of the chest pain, cardiac

METHODS OF ANALYSIS
We visit the problem of interpreting a laboratory test. Using a 2 x 2 contingency table one calculates the sensitivity and specificity for any cutoff value. The cutoff value of the test will determine the falsepositive and false-negative error rates and, thereby, have an effect on the predictive values. It is again useful to refer to the standard 2 x 2 contingency table for a frame of reference (Table 2).
One can calculate the likelihood ratios and odds ratios from these tables [2,3]. The LR+ is obtained by dividing the first row, first column by the column total and the first row, second column, divided by its column total, and then taking the ratio of these ([a/a +c]/[b/b + d]). The LR-is obtained by dividing the second row, first column by the column total and the second row, second column by its column total, and then taking the ratio of these ([c/a + c]/[d/b + dl). This is the same thing as obtaining the ratio of true positive rate/false positive rate, and of the ratio of the false negative rate/true negative rate, respectively. The likelihood ratios are a ratio of two probabilities having a value from 0 to 1. They are, therefore, a ratio of probabilities. The odds ratio is derived from the LR+/LR-, but the column totals (a + c, b + d) drop out. This is of some interest. One can calculate an odds ratio from the standard 2 x 2 format as ad/bc, where: a = true positives; b = false positives; c = false negatives; d = true negatives. The odds ratio is obtained from the ratio of odds ([a/c]/[b/d]). In this case, the odds for a positive result in the disease population is a/c, and in the nondisease population is b/d. The odds and odds ratio can be inverted to: c/a, d/b, and (c/a)/(d/b) = cb/ad. The odds and odds ratios have values from 0 to mo. The probability can be converted to an odds by the calculation: p = odds/i + odds, and the odds can be converted to a probability. The introduction of likelihood ratio, odds and odds ratios introduces a concept of incremental risk.
In order to calculate odds ratios for a laboratory test, it is necessary to scale the data into discrete intervals. When the cutoff is not assumed to be known, the continuous values for the test are converted to several ranges. In the case of MI, the frequency of the values is used for two states: MI present and absent. The LR+ increases for each interval for MI present, as shown. Since there are several levels of cardiac marker, the sensitivity and specificity of the test can't be calculated in a 2 x 2 table. We can add the additional problem that there are more than two statessuch as -MI, acute coronary syndrome, and neither. This requires that we set up a table that is NxN in dimension.

Linear regression methods
Linear regression explores the fit of a line to a set of data represented by a response variable and a predictor variable [4]. The response (predicted) variable may be continuous or ordinal, but the predictor variable is usually continuous. When the response (predicted) variable is ordinal the linear model breaks down to a one-way analysis of variance model (ANOVA). When a continuous variable is fitted to a single predictor the simple linear regression is expressed by the equation: where a is the Y intercept, b is the slope of the regression line, and a or b is derived from the method of ordinary least squares (OLS).
The method of OLS uses the F test for the mean squared errors and requires normality of the errors. The measure of fit is

Logistic regression
The linear regression model is extended when the predictor variables are coded 0,1 and the response is a probability from 0 to 1 [4,5]. In the case of logistic regression the response variable is two valued or binary.
Graphical ordinal logit display Jay Magidson (Statistical Innovations, Belmont, Massachusetts) has developed a polytomous approach to probability estimation using ordinal variables whereby quantitative scores may or may not be assigned to the categories, but category spacing is assumed to exist [6][7][8]. The method assumes a monotonic relationship and uses a maximum likelihood estimation. It uses a log odds model fit and the odds ratio is obtained from the log (odds ratio). In the linear probability model, the coefficients (bi) are partial correlation coefficients. In the logit model, the coefficients are partial log (odds ratio). The results are expressed graphically in a GOLDminerTM (graphical ordinal logit display) interface. The method is polytomous because the outcome can have more than two values.
The method displays the observed frequencies, calculates the expected probabilities, the expected odds, and the expected odds-ratios. Baseline odds and baseline odds ratios are calculated by dividing each estimated expected frequency by the corresponding base frequency and by dividing each expected odds by the baseline odds, respectively, associated with the reference category. GOLDminer plots are effect plots that are obtained by taking the natural log of the expected odds-ratios.

RESULTS
The data are organized into a crosstable for classification purposes. Table 3 is a crosstable of TnT and disease classes with two configurations. The crosstable can be two or more columns and two or more rows, but it is essentially a two dimensional configuration. Chi-square (X2) is the measure of fit for the frequency of values in the cells compared with the expected frequency, which is calculated from the column and row totals, and the total count. The main limitation of this is that it has to be n-dimensional for more than one predictor and requires a single dependent variable. Rypka [9,10] has described a method for classification using a truth table. The method requires that the multiple variables are listed as discrete combinatorial classes. The combinatorial classes can be put in a truth table with the expected outcomes in columns and the classes in rows. We shall refine this concept with Magidson's universal regression [6][7][8]. Table 3

A CLOSER LOOK AT CALCULATIONS
We first take the regression using TnT taken at 3 hours after presentation (or 9 hours after the onset of chest pain) at a value of 0 or 1 determined at a cutoff of 0.1 mg/L to predict MI, unstable angina (UA), or other cardiac or noncardiac finding. The X2 for the fit is 98.89, significant at p = 3.4 x 10-22, with R2 = 0.337 and 1 = 0.4920. The frequencies are in Table 4.  6.0*10-21 CKMB (12 hr) 5.0*10-17 CKMB (3 hr) 1.4*10-9 Chest pain 6.9*10-4 The observed probabilities are obtained by summing the TnT frequencies in the assigned category and taking the ratio of each cell to its corresponding sum. Thus, the observed probability for TnT (-), other = 254/254 + 72 = 0.79, and that for TnT (+), other = 0.15. The expected probability is taken from the expected frequency based on the chi square distribution. Table 5 is the expected probabilities.
Given the probabilities, we can calculate the corresponding odds for any p using the following calculation: The odds for an event is expressed as o, but the odds against is o-1. An observed probability is not necessarily the same as an expected probability. A probability is adjusted when some evidence is introduced to change our estimation of the probability. The probability before is called "prior probability," and the new probability is called "posterior probability," or "estimated probability." The odds ratio is obtained by dividing the odds for the row used to demonstrate no effect, in this case other, by the rows -UA and MIwith the effect, where TnT negative is baseline odds. The odds-ratios are 192 for MI and 14 for UA with the 3hr TnT. An odds ratio of 1 indicates independence of the treatment and the outcome, or a measure of no effect. An odds ratio that is less than or greater than 1 is a measure of the degree of dependence between the categorical variables. In the example used, a contingency table is tested by the Fisher's exact test under the assumption that the odds ratio is 1. The row and column totals are held fixed, and the probability of the counts are compared with the expected counts under a hypothesis of independence. The Pearson X2 is an approximation of the X2 distribution. The differences between observed and the expected counts are used to calculate the chisquare [5]: X2 = X(observed countexpected count)2/(expected count) I have not calculated the expected odds, which can be done from tables as described above. The expected odds and expected odds ratios are calculated using the GOLDminer software.
The method uses multiple or a single variable to predict an ordered response. In the case of the single variable the method is based on a regression of the predictor on the dependent variable and the predictor values are scaled. The model fit is measured by the chi square for the regression, and the odds-ratios are calculated from the logit model of the response. The multivariable response model forms the classes of a truth table described by Rypka [9, 101.
The classes are fitted to a logit response curve in the same manner as the single variable model. A graphical presentation is generated, but the odd-ratios for each class is also available in table format. It is also possible to compare the effect of variables used to classify patients and to compare the variable combinations. Table 6 is a list of single variables used to predict MI-present vs. MI-absent, and the p-value of the regression ranked in descending order. Note that TnT at 3 hours is almost as good as obtained at 12 hours, and is superior to the CKMB at 3 or 12 hours. Table 7 is an expansion of the univariate predictors for MI versus UA or other in Table 6, which has only the p-value for the chi square. X2, R2, and 4 are important measures of model fit that are added to the table, which now includes troponin I at one (0.5 mg/L), and at two (0.5, 1.5 mg/L) cutoffs. The R2, as in linear regression, is a measure of the amount of variability explained. F is a better measure for the nonparametric function.

COMBINING VARIABLES
The GOLDminer model predicts using multiple variables. The same outcomes are modeled using TnT and chest pain as a predictor. Chest pain is typical, atypical, and none. The X2 for the regression is 138, significant at p = 9.2 x 10-25. The fit is measured by R2 = 0.392 and P = 0.6619, and 1 is a better measure than R2. Table 8 is the frequencies and probabilities for the model using TnT at 3 hours and chest pain characteristics. The six classes of TnT (0.1 mg/L cutoff) and chest pain are ordinal features used to predict.

MULTIVARIABLE RESPONSE
We have examined the basis for looking at the strength of an association using the N-by-N table. The method of ordinal the association, analogous to partial correlation coefficients, are beta. The model produces probabilities and odds-ratios. Table 9 is the probabilities and odds-ratios for the multivariable model for predicting MI using TnT as the first variable, scaled to 0 or 1 by the 0.1 ug/L cutoff, and chest pain as the second variable. Chest pain is scaled 1 to 3 as none, atypical, and typical. Chest pain is the most important variable in assigning patients to ACS or not. This produces a two-step model involving deselection of low risk patients without ACS, and ruling-out patients with ACS who exclude for MI. EKG is not a significant factor because ST segment elevation is removed. The number of classes is deter-mined by the number of variables and scaling.
The GOLDminer establishes the risks in an association model where there is a treatment and an effect. It is used to find an association for a drug compared to the placebo effect. A laboratory test is the treatment compared with clinical indicators in the example shown, and the association is MI. The classification is known, patients being assigned to MI or to ACS using defined criteria. Patients who have a positive test result may fail the criteria for UA or MI and they are assigned to a category other. The other category is absence or presence of cardiac disease. Patients with non-ischemic cardiac disease may have congestive heart failure or rhythm abnormalities. They don't have ACS. Table 10 is a multivariable model to predict MI using chest pain and risk factors, then adding the laboratory tests to provide a definitive assessment of risk. Note that the CKMB (with evolutionary    Figure 4. Joint regression for TnT, chest pain, and EKG with Ml, UA, and no ACS. Table 11 is the effect of scaling the continuous variable TnT for predicting MI or not into 7 discrete intervals (ordered) instead of a one or two cutoff partition (X2 = 112.49, p = 2.8 x 10-26, D = 0.5020). The lowest cutoff is at 0.04 mg/L, and the highest is at 0.35 mg/L. The table is based on only MI present and absent. The frequencies, probabilities, and odds ratios are shown. We find that with this partitioning, a finer definition of expected risk is captured than we previously identified. TnT has a significant risk of 10 percent probability for MI at a value between 0.08 to 0.099 mg/L, which corresponds to an odds ratio of 45.

GRAPHICAL DISPLAY OF RESULTS (GOLDMINER)
Table 11 is derived from a logs odds model. Figure 1 is a log odds model using 5 intervals, the scaling changed to: < 0.075, 0.075-0.099, 0.1-0.199, 0.2-0.05, and > 0.35 mg/L. The log (odds ratio) is converted to the odds ratio view in Figure   2. Figure 2 has equal spacing between variable intervals on the right that have odds-ratios on the left of the plot, and the outcome variable is described on the Xaxis. Figure 3 is the combined model Yview using the TnT scaled to 5 values with chest pain scaled to none (1), atypical (2), and typical (3), and EKG features (ST depression excluded) scaled to negative (1) or ST depression (2).
It is important to consider both the odds-ratios and the probabilities. An odds ratio of 50 to 1 is expected for a probability of less than 5 percent. If we look at the    Figure 3, the risk increases at a lower rate than the odds ratios for the combined model with UA as a defined class.

COMPARING SIZE OF EFFECT AND CONTRIBUTION TO MODEL
We have to consider the effect of TnT with clinical variables, with and without UA as an outcome class. Figures 1 and 2 use UA as a class. Figure 4 is the joint regression for the combined variables, TnT, chest pain, and EKG using UA as a class. Figure 5 is a Y-view of TnT with the OR for MI/not MI dichotomous choice.
In the dichotomous model, chest pain and EKG drop out of the model. The determination of risk is weighted by the TnT. Table 13 is the probabilities and odds ratios for TnT with five intervals in the dichotomous outcome model (without the UA class). Compare this with Table 11 with seven intervals and a dichotomous choice.  Table 11 has a hidden risk of 0.10 with an odds ratio of 45 at TnT in the range of 0.08-0.99 mg/L compared with a risk of 0.10 and odds ratio of 41 at TnT in the range 0.10-0.199 mg/L in Table 13. There is also an effect of using an additional class, where the probability of class membership is not exactly known. This is shown by a TnT in seven intervals with UA, MI and other as the dependent variable (Table 14) (X2 = 132.28, p =1.3 x 10-30, 1 = 0.6659). The TnT interval of 0.08-0.099 mg/L has a risk of 0.16, accounted for by seven patients, with an odds ratio of 64.

MORE MODEL COMPARISONS
We want to consider the effect of the contribution of the clinical information versus the contribution of the TnT. The full model would have four categories: MI, UA, cardiac (nonischemic) and noncardiac. The combined model for chest pain and EKG results in the following: X2 = 167.3, p = 4.6 x 10-37, and 1 = 0.5106. The probabilities for chest pain and EKG are shown in Table 15.
The probabilities for TnT alone have already been discussed for a model with seven intervals. The TnT alone may have a better model fit than the chest pain and EKG combined based on: x2 = 114.3, p = 1.2 x 10-26, and '1 = 0.6861. When Chest pain and EKG were added to the TnT, EKG dropped out of the model. The com-bined TnT with chest pain model is described as follows: X2 = 169.8, p = 1.3 x 10-37, D=0.9732.

LATENT CLASS MODELS
We can model the data assuming that we don't know how the patients are to be classified. A Latent Class Model is used when the true classification isn't known, or when it is known, but it is of interest to know how the data fit the proposed classification. A Latent Class Modeling software has been developed by Magidson and Vermunt (Latent GOLD) [11]. We examine our proposed model using Latent GOLD and the tradition LCM (other choices are LCM factor analysis and LCM regression). The optimum classes were 3. Table 16 is a LCM using chest pain, risk factors, EKG, and TnT at 3 hours, ACS is a covariate. Those with ACS are in classes 1 and 3, and those without ACS are in classes 2 and 3.
Patients with ACS have MI or they don't. They are distinguished by evolutionary changes of CKMB or by TnT. Classes 1 and 3 had 44 percent and 97 percent probability of typical chest pain. The probability of TnT positive was 48 percent in class 1 and 0 percent in class 2. The LCM was done using the same variables, but with CKMB and MI/UA as covariates instead of ACS. There were three classes with 57 percent of patients in class 1, 10 percent in class 2, and 33 percent in class 3. The probabilities of the variables in the classes is shown in Table 17. The refined model using the covariates results in all of the MI and only MI in class 2. TnT positivity is 100 percent. Class 3 has a significant population with ACS without MI (TnT positivity is 0 percent). Class 1 is the other category.

DISCUSSION
We illustrate the application of a new method, GOLDmineRTM, to the analysis of the clinical and laboratory evaluation of the patient presenting to the emergency department with features suspicious for MI. The method is nonparametric. It has none of the dependency of classical regression methods on assumptions of normality and distribution of the errors. The linear and logistic regression methods might be viewed as a special case of this regression method under limiting conditions.
The regression model has a dependent variable and independent or predictor variables, and is described by the equation: Where, e is the error, referred to as the residual.
The estimated equation is: + = a + YbnXn The error term I is not known until the equation has been solved for a and all of the b's [4]. The error term drops out and Y becomes+9. The error term e is Y-It is the observed value of the outcome variable for a given patient minus the predicted value for that patient. The best solution for the equation is that which minimizes e (e 0). The best estimate has been achieved when the sum of the squared error term has been minimized: The error term is called the residual. The values of a and the b terms, which give the smallest value for e2, are the best estimates for the data set. The method of least squares produces the estimates a, bn. The general linear model has variations, depending on the types of variables used for the Y and X terms. It is linear because it is a linear combination of the Xn terms.
There are certain assumptions in using the model. The F test is used for the regression, where: F = mean square (regression)/mean square error, provided bn = 0 We have to assume normality of errors. The correlation coefficient is r. We assume that r2 is the proportion of the variance that is explained. R2 =sum squares(SSreg)(regression)/sum squares (SStota0)(total) = I -SSe/SStotal The correlation coefficient, r, is the measure of linear association. It has the following properties: It is bounded in absolute value by 1. All points on a straight line imply correlation of 1.
Correlation of 1 implies all points are on a straight line.
The correlation coefficient is invariant to linear transformations of either variable.
Logistic regression is a linear probability model in which the standard regression model is applied to data for which the dependent variable is dichotomous (0, 1). The predicted values from the model are interpreted as a probability that the response is a 1. There are problems in using the linear probability model.
The residuals don't have a constant variance so that estimates from regression are not best linear unbiased and minimum variance.
Standard errors of regression coefficients are wrong giving invalid confidence intervals.
The predicted values from regression can range outside the interval (0,1).
The usual r2 measure is problematic. These problems are all inapplicable in the universal regression model, GOLDmineR, developed by Jay Magidson [6][7][8]. GOLDmineR and related methods are not constrained by the limitations imposed by the classical approaches to exploring data and testing hypotheses. The classical approaches are stretched whenever: (1) the law of the excluded middle does not apply (the usual 2 x 2 table is problematic); (2) the selection of a reference outcome variable is difficult to determine; and (3) assumptions about the distribution of the data and the errors are unwarranted. The methods we favor allow for: Determining medical decision values by exploring the information in the data set; (2) generating hypotheses supported by the data; and (3) constructing a selfclassifying table of ordered classes that have assigned measurable risks.
We have shown that there is no justifiable definition of normal reference in a population defined by a single attribute (or test) without referring to a supervisory classification [ 12]. Such an approach requires validation by use of a receiver operator characteristic curve. This condition is eliminated by treating diagnostic tests as a message transmission and the provisio of information as reduction of uncertainty [12]. The advantage of this is independence from any assumption about the nature and distribution of the data [11]. The process of maximizing information content and optimizing separation is tied to uncertainty in information theory [12,[13][14][15][16]. In this study, the definition of MI was dependent on the characteristic evolution of CKMB isoenzyme in the absence of definitive EKG changes. However, there is a latent class identified in the UA patients, with a significant risk of a cardiac event.
We examine the effects of scaling and of the number of outcome classes on the regression. The initial scaling of TnT to a single cutoff at 0.1 mg/L was determined at the maximum entropy decision point for discriminating MI from not MI is previous studies. Rudolph and Bernstein [12] describe the maximum entropy decision point as the level at which there is discrimination of two major classes with the fewest errors. The upper limit of normal may be distinctly different than the MEDP. Rypka [9,10] describes the formation of classes using variable combinations and the number of variables required to classify. We can define too few or more intervals than are needed for optimum classification. GOLDmineR permits us to explore the effects of scaling and to find suitable spacing and number of intervals. Unlike logistic regression, GOLDmineR easily permits the modeling of outcomes when the dependent variable is not dichotomous. This is very well illustrated by the understanding of a concept in cardiology of "acute coronary syndromes," which is a spectrum of unstable conditions between stable angina and MI which may culminate in "plaque rupture" in a coronary artery. The classical dichotomy doesn't accurately describe the actual events.
We also investigated the examination of risk for single predictors and variable combinations as the probability for MI within a class or feature. This is extremely important for presenting the options for clinical decision-making when the risk isn't well known, and might be overestimated. We identify a significant probability for MI with TnT in the range of 0.08 to 0.099 mg/L that wasn't clear from earlier studies, but which is consistent with the ACS concept in this study of non Q-wave MI.

CONCLUSIONS
The laboratory is dealing with reporting tests as information needed to make clinical decisions. The traditional statistical quality control measures which assigns reference ranges based on 95 percent confidence intervals is insufficient for diagnostic tests that assign risk. We construct a basis for risk assignment by a method that builds on the 2 x 2 contingency table used to calculate the X2 goodness-of-fit and Bayesian estimates. The widely used logistic regression is a subset of the regression method, as it only considers dichotomous outcome choices. We use examples of multivalued predictor(s) and a multivalued as well as dichotomous outcome. Outcomes analyses are quite easy using the ordinal logit regression model. REFERENCES: