# Can the lower/upper end of a confidence interval be interpreted separately?

A health insurance company here presents rating lists of hospitals based of their SMR (standardized mortality/morbidity ratio) values (observed/expected, i.e. the higher the worse). They use the confidence intervals of SMR as well, which are wide in a small number of cases and narrow in large samples. So far, so good. It follows: they consider the best hospitals as those with SMR below the 20 percentile with lowest values of the upper ends of their confidence intervals and the worst hospitals as those with SMR above the 80 percentile with highest values of the lower ends of their confidence intervals – regardless whether the whole confidence interval of a particular hospital includes 1 or not. As we asked about a rationale for this they argued that the higher value of the lower end of the confidence interval shows the higher probability of the undesirable outcome.

They even give an example. Two hospitals perform the same surgery with 96% success: the hospital A on 500 patients and the hospital B on 50 patients. They mean that the achieving the success value of 96% success is statistically more likely in the hospital A, because their confidence interval is 94 – 98% there. The hospital B would have the confidence interval of 84 to 100% - thus, not that nice.

I always thought that the confidence intervals just show the range where the true value lands with 95% chance (well, for 95% intervals) and that they should be always observed as a whole, not just lower end or upper end. As I read somewhere, confidence interval shows where the bomb lands with the 95% probability, not where the 95% of all the bombs land. For the latter there are tolerance intervals that are calculated differently. In example above there would be no difference between two hospitals on the basis of the mentioned confidence intervals. And it a confidence interval of a SMR includes 1 you cannot claim the result is better or worse as the expected value – regardless how high or low one end of this confidence interval is located.

I’m surprised however since it is a large company with the own statistic department. Please give me an advice whether I interpet the things correctly or miss something.

Thank you!

## 1 Answer

Your understanding of confidence intervals is a little flawed - they are not where the true value lands 95% of the time - its where, if we repeatedly resampled the population and conducted the same study, 95% of the estimates would fall.

There's no reason the lower or upper bound of a confidence interval couldn't be considered separately for some questions - you implicitly do so in this part of your question: "regardless whether the whole confidence interval of a particular hospital includes 1 or not", which in essence is the same as "Is the lower bound below 1 for estimates above one?" and "Is the upper bound above one for estimates below 1". That kind of ad hoc significance test is invariably only considering one of the two bounds.

As for the meat of your question, there are definitely some issues with using the confidence intervals the way they are, and I'm a little skeptical of their approach. It inherently penalizes small hospitals, as they have wider confidence intervals by virtue of their size and the number of events that occur. This may be an overcorrection for a number of rating schemes that penalize the opposite direction, favoring small hospitals (Consumer Reports, I'm looking at you). More generally, I'm skeptical about rating hospitals against each other generally.

• Thank you very much! I thought that large hospitals could be penalized as well: consider SMR 0.95 (0,65;1,25) and 1,05 (0,55;1,55) - with the above approach the first one would be (and is indeed) declared worse since the lower bound is higher. I would say that in both the results are not significantly different from expected value and are not different among themselves. I wanted to object their reading of higher lower bound as a proof of higher probability of undesirable outcome. Your remark in the first section of your answer resp. understanding of conf. intervals is of course correct. Commented Jan 29, 2016 at 22:22
The text passage discusses the interpretation and use of confidence intervals in evaluating hospital performance based on SMR values. It includes a specific example comparing two hospitals and their success rates, along with a critique of the method used by a health insurance company to rank hospitals.
no_question

the passage does not contain a direct mathematical question or calculation. instead, it presents a discussion and critique of a statistical method used by a health insurance company. the example provided is used to illustrate the critique rather than to pose a question or provide an answer to a mathematical problem.
Given a text passage, determine:
    1. If it contains a mathematical question (including word problems or calculations)
    2. If an answer to that mathematical question is provided in the text

First, describe the content structure you found in 1-2 sentences.

Then provide your analysis in the following format:
TYPE: <one of the following values>
- NO_QUESTION
- SINGLE_QUESTION
- SINGLE_QUESTION_WITH_ANSWER
- MULTIPLE_QUESTIONS
- MULTIPLE_QUESTIONS_WITH_ANSWERS
system
# Can the lower/upper end of a confidence interval be interpreted separately?

A health insurance company here presents rating lists of hospitals based of their SMR (standardized mortality/morbidity ratio) values (observed/expected, i.e. the higher the worse). They use the confidence intervals of SMR as well, which are wide in a small number of cases and narrow in large samples. So far, so good. It follows: they consider the best hospitals as those with SMR below the 20 percentile with lowest values of the upper ends of their confidence intervals and the worst hospitals as those with SMR above the 80 percentile with highest values of the lower ends of their confidence intervals – regardless whether the whole confidence interval of a particular hospital includes 1 or not. As we asked about a rationale for this they argued that the higher value of the lower end of the confidence interval shows the higher probability of the undesirable outcome.

They even give an example. Two hospitals perform the same surgery with 96% success: the hospital A on 500 patients and the hospital B on 50 patients. They mean that the achieving the success value of 96% success is statistically more likely in the hospital A, because their confidence interval is 94 – 98% there. The hospital B would have the confidence interval of 84 to 100% - thus, not that nice.

I always thought that the confidence intervals just show the range where the true value lands with 95% chance (well, for 95% intervals) and that they should be always observed as a whole, not just lower end or upper end. As I read somewhere, confidence interval shows where the bomb lands with the 95% probability, not where the 95% of all the bombs land. For the latter there are tolerance intervals that are calculated differently. In example above there would be no difference between two hospitals on the basis of the mentioned confidence intervals. And it a confidence interval of a SMR includes 1 you cannot claim the result is better or worse as the expected value – regardless how high or low one end of this confidence interval is located.

I’m surprised however since it is a large company with the own statistic department. Please give me an advice whether I interpet the things correctly or miss something.

Thank you!

## 1 Answer

Your understanding of confidence intervals is a little flawed - they are not where the true value lands 95% of the time - its where, if we repeatedly resampled the population and conducted the same study, 95% of the estimates would fall.

There's no reason the lower or upper bound of a confidence interval couldn't be considered separately for some questions - you implicitly do so in this part of your question: "regardless whether the whole confidence interval of a particular hospital includes 1 or not", which in essence is the same as "Is the lower bound below 1 for estimates above one?" and "Is the upper bound above one for estimates below 1". That kind of ad hoc significance test is invariably only considering one of the two bounds.

As for the meat of your question, there are definitely some issues with using the confidence intervals the way they are, and I'm a little skeptical of their approach. It inherently penalizes small hospitals, as they have wider confidence intervals by virtue of their size and the number of events that occur. This may be an overcorrection for a number of rating schemes that penalize the opposite direction, favoring small hospitals (Consumer Reports, I'm looking at you). More generally, I'm skeptical about rating hospitals against each other generally.

• Thank you very much! I thought that large hospitals could be penalized as well: consider SMR 0.95 (0,65;1,25) and 1,05 (0,55;1,55) - with the above approach the first one would be (and is indeed) declared worse since the lower bound is higher. I would say that in both the results are not significantly different from expected value and are not different among themselves. I wanted to object their reading of higher lower bound as a proof of higher probability of undesirable outcome. Your remark in the first section of your answer resp. understanding of conf. intervals is of course correct. Commented Jan 29, 2016 at 22:22
user
The text passage discusses the interpretation and use of confidence intervals in evaluating hospital performance based on SMR values. It includes a specific example comparing two hospitals and their success rates, along with a critique of the method used by a health insurance company to rank hospitals.

TYPE: NO_QUESTION

The passage does not contain a direct mathematical question or calculation. Instead, it presents a discussion and critique of a statistical method used by a health insurance company. The example provided is used to illustrate the critique rather than to pose a question or provide an answer to a mathematical problem.