{'fraction_non_alphanumeric': 0.033365477338476376, 'fraction_numerical': 0.01832208293153327, 'mean_word_length': 4.873159682899208, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Lost in translation, or overestimating administrative data for outcome control in colorectal surgery

A total of 52 544 case reports, 31 institutions and five countries were involved in an attempt to achieve an international benchmarking instrument for colorectal surgery [1]. The results are published in this issue. Is this mammoth work a breakthrough on the way to international, population-based comparisons? I am afraid not. The effort invested by the authors resembles the building of the Tower of Babel. Like the story in the Book of Genesis, this powerful instrument should enable universal control over the Earth. It was not successful then and it will not be successful now, but this time no divine intervention will be necessary. The Original Sin of the project lies in the use of administrative data. The temptation is enormous and many succumb to it; there are huge databases that can be effectively mined, producing many papers, as is the case in Medicare & Co.-derived publications from the USA. But these data are generated mostly for financial reasons and have little in common with the clinical picture. The outcomes analysed in the present paper include 30-day mortality, 30-day readmission rate, 28-day reoperation rate, length of stay and overall complication rate, all derived from administrative records. What do we know about their validity? Koch et al. [2] compared recent postoperative complications between US administrative and clinical data sources. They used a Cohen’s kappa (K) coefficient as a measure of agreement between the sources. A kappa of 1 indicates perfect agreement, whereas a kappa of 0 indicates agreement equivalent to chance [3]. For haemorrhage the observed K-values ranged between 0.08 and 0.27, for sepsis between 0.05 and 0.16 and for respiratory failure between 0.02 and 0.40, not reaching good concordance in any of these parameters. Similar disappointments were confirmed by Cima et al. [4] who analysed adverse events at single institutions recorded by the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) and by the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI). In this study ACS-NSQIP data were identified in 564 (7.4%) patients and AHRQ-PSI in 268 (3.5%) patients, but only 159 (2.1%) patients were identified by both systems. The poor concordance is not a US speciality. In 2008 Garout et al. [5] had compared administrative data (HES; hospital episode statistics) with the Association of Coloproctology of Great Britain and Ireland (ACPGBI) colorectal data base. The administrative data reported 12% more procedures and overall correlation between the reported procedures was 0.660. The reported mortality was also inconsistent, showing the poorest correlation for abdominoperineal excision (Pearson’s correlation coefficient r = 0.253, P = 0.038). And as stated by Lewis et al. [6], even the data recorded in the National Bowel Cancer Audit Project (NBOCAP) are far from clinical reality. In their analysis for 2008– 09, 75.8% of patients were reported as American Association of Anesthesiologists (ASA) grade 1; actually, only 6.1% were actually of this grade. The actual data showed the 66.6% of the procedures were elective, while the reported value was 90.3%. The reason for this discrepancy was very simple; the data were reported not by the clinical staff but by the administrative staff. Even such simple events as readmission cannot be reliably derived from administrative data, as proven by Hechenbleikner et al. [7]. Also, the crucial issue of cancer recurrence cannot be mapped by administrative billing and encounter codes [8]. But are these findings really surprising? What is the rational basis for the hypothesis that administrative data do reflect the clinical reality? We do not compare food quality in restaurants on the basis of the bills, so why do we have this expectation in surgery? A coding system is a way of communicating between the care provider (usually a hospital) and the payer (usually an insurance company or administrative body). Both structures have contradictory interests and administrative data illustrate the state of this conflict. Some systems even analyse what ‘depth of documentation’ produces the best financial outcome [9]. This is not the way to analyse clinical outcomes. Also, public calls for transparency and accountability in health care combined with an already fixed financial cut in health expenses [10] are not the way to control surgical quality. The current discussions underestimate the human factor and several basic instincts. One of the most powerful is a survival instinct. If the outcome data are used for decision making, i.e. which provider should be favoured, there is a strong bias towards ‘adaptation’ of the data. The same situation occurs where outcome is used as an advertisement to reach new customers and to win over competitors in the market. There is also a need to understand the difference between data collected at the level of a single institution, single surgeon or single trial on the one hand, and huge, national or international cohorts on the other. But the dimension