Toxic Numbers

Soon after the main PACE paper was published in 2011, there were numerous stories in the media about the dangerous militant ME patients who were harassing the researchers. It is known that the Science Media Centre was heavily involved in promoting this coverage. Many ME patients have suspected that these stories were part of a campaign to discredit critics of the trial. There is no question that it has affected how ME has been covered, part of what has been called the epistemic injustice (here and here) suffered by patients.

One of the main criticisms of the trial was that outcome thresholds were switched after the trial had finished. Some data were released after a First Tier Tribunal hearing. Reanalyses using the original thresholds found:



These findings raise serious concerns about the robustness of the claims made about the efficacy of CBT and GET. The modest treatment effects obtained on self-report measures in the PACE trial do not exceed what could be reasonably accounted for by participant reporting biases.

This week, three of the researchers responded to the reanalyses. This week, also, Reuters carried a story saying that one of them, Michael Sharpe, had been driven out of researching the illness because the field is ‘too toxic’. The timing did not seem coincidental.

The article claims that Professor Sharpe is not alone and that ‘there has been a decline in the number of new CFS/ME treatment trials being launched’. It quotes evidence from


I was surprised by that figure as research couple of years ago found an increase in the number of papers published on the illness. Along with the trend in medical science generally, more work was being done.

My first thought was that perhaps there was an innocent explanation for this decline: PACE was a massive trial, deemed ‘definitive’. The investigators were seen as the leading experts on this approach and the findings were said to be in line with other trials of the interventions. There wouldn’t seem much point in keeping on doing the same trial time after time.

I decided though, that I would also go to the website and check the numbers. And this is what I found:

ME-CFS trials 200-2019

The figures given in the article then are not false, but they would seem to be misleading. The overall numbers are very low, so a year when there are more trials than usual will have a large effect. Kelland chose to compare the last five years with the previous five years. But if you compare the last ten years (and, after all, the first stories about harassment came out in 2011, so ten years would seem just as reasonable), then there has in fact been a massive increase in trials being launched, not the decline described in the article. Or one could be very mischievous and point out that since Professor Sharpe announced he was quitting the field last year, the number of trials has gone up.

There is a further point. The article says that he and others are being driven out of research because some patients do not like their approach as it ‘suggests their illness is psychological’. No one researching ME as a pathological illness has made such complaints. It would be expected then that trials taking this behavioural approach should show a decline and that others, looking at possible drug treatments, for example, would be unaffected. As the table shows, though, there is no such trend.*

Finally, the number of trials listed seemed very low to me. I looked at some of them. They include acupuncture studies from China but as far as I could see they don’t show the PACE trial (or SMILE), so the numbers are probably not very reliable anyway.

An article standing up for the PACE researchers uses dodgy numbers.

As for the substance of Professor Sharpe’s accusations, he has himself described what he finds offensive: ‘attempts to damage his reputation and get his papers retracted’. Many would not agree that amounts to trolling.

It should also be pointed out that he himself has been described as ‘intemperate’; someone who ‘lashes out’ and who ‘cannot tell the difference between disagreement and defamation’.

Claims of bullying and harassment driving researchers from the field are often made by those with power against those without:

It’s a clever strategy, in its way. But it’s also the signet ring of weak people defending weak ideas.


Added 17/03/2019:

*It has been suggested that there is in fact a discernible trend in the lower numbers of behavioural trials, but there is no way of knowing whether there is a trend or whether there was a slight blip in 2009 or whether it is possible to know anything at all from such small numbers. It could be argued, for example, that one third of all clinical trials for ‘CFS/ME’ in 2018 were behavioural.

The same point about the reliability of the search applies, since neither PACE nor SMILE are listed.

And even if there were some sort of trend, there is no way of knowing the impact, for example, of the IOM report and the change in focus by the NIH.


Even the Ethics Committee says the PACE authors should share the patient-level data, so why does PLOS ONE not enforce its regulations and compel them to?

In 2012 a cost-effectiveness analysis of the PACE trial was published by PLOS ONE. One of the conditions of publication was an agreement by the authors ‘to make freely available any materials and data described in their publication that may be reasonably requested for the purpose of academic, non-commercial research’.

A number of requests have been made to the authors for the patient-level data. These requests have been rejected.

PLOS ONE has issued an Expression of Concern, but has not obliged the authors to make the data available. In response, the authors state they are ‘surprised by and question the decision by the journal to issue this Expression of Concern’. They continue to reject all requests for patient-level data. In their Statement, they make it clear why:

‘During negotiations with the journal over these matters, we have sought further guidance from the PACE trial R[esearch] E[thics] C[ommittee]. They have advised that public release, even of anonymised data, is not appropriate.’

The PACE trial REC is a public body and its advice to the authors was a public document covered by the FOIA, so I made a request asking for a copy (available here).

The REC does indeed advise against public release of the data, though for what many would consider a spurious reason and one beyond its remit, namely because of the controversy surrounding the illness (‘a contentious matter upon which there are divergent views’).

However, the REC also made clear that the data should be shared with an independent researcher or institution who/which would then ‘receive the anonymised data and produce a report’. In its response to my request the REC’s summarized its position as follows (email available here):

…from a transparency perspective the sharing of data would be the HRA’s normal recommended course of action. However as suggested in the attached email by the REC Chair, and from our discussions with other parties involved we understand the data may be made accessible via a secure site to other independent researchers wishing to analyse the data.

The REC’s view then is that there should not be public release of the data but that the data should be shared with independent researchers.

So why does PLOS ONE not enforce its conditions of publication and oblige the authors to provide the data to the researchers who have requested them or retract the paper?

Is PLOS ONE saying the researchers who have requested the data, such as Professor Coyne (see also here and here), are not ‘independent’?

If those researchers are not deemed sufficiently ‘independent’, then why doesn’t PLOS ONE nominate an independent scientist and instruct the authors to comply with the regulations and the wishes of the REC and provide that scientist with the patient-level data or retract the paper?

How does this refusal by PLOS ONE meet the claim by its editor to be committed to ‘full transparency‘?

Where does PLOS ONE’s failure to abide by its regulations leave all the academics who work for it as unpaid peer-reviewers, bloggers and editors because they support open science?

And why do the PACE authors continue to refuse access to the patient-level data even when the REC says they should provide it to independent researchers?


The PACE trial and why the data need to be shared

PACE was the ‘definitive’ trial of two interventions for myalgic encephalomyelitis (ME), sometimes known as chronic fatigue syndrome (CFS): cognitive behavioural therapy (CBT) and graded exercise therapy (GET). The main paper was heavily promoted in the media as providing evidence that CBT and GET could help patients ‘get back to normal’.

A number of criticisms have been made about the design of the trial, in particular: First, the trial used a very broad criteria for diagnosis of the illness which could include patients with general fatigue and/or mental health problems. Such patients are known to respond to the interventions. Second, the investigators failed to guard against the powerful distorting effect of researcher bias. Third, there was a heavy reliance on self-report measures in an unblinded trial (where both patients and therapists knew who was and who was not receiving the interventions). It has long been known that the use of subjective measures in an ‘open-label trial’ gives false results.

There has also been considerable criticism of the decision, which was taken after the trial had been completed, to deviate from the protocol and make a major change in the thresholds in the outcome measures. The investigators altered how they measured whether an intervention was effective or not. According to the measures they eventually used, a participant could actually get worse in the trial — their score could go down during the period when they were receiving treatment — and yet they could still count as evidence for the effectiveness of the treatment. 13% of patients on this new scoring were already ‘recovered’ on a measure before they had even entered the trial. In other words, they were both ill enough to qualify to participate in the trial and ‘recovered’.

The investigators, the responsible authorities and the main funder, the MRC, have always said they were prepared to provide access to the data to ‘bona fide researchers’. Patient-level data were shared with a few researchers, who are known to support the interventions, for a Cochrane systematic review (though note criticism of Cochrane reviews of interventins for this illness here, here and here, comments by among others Bob Courtney and Tom Kindlon, and reanalysis by Mark Vink and Alexandra Vink-Niese).

When researchers, patient organizations and patients have asked for the data in order to conduct independent analyses using the measures set out in the protocol, these requests have been rejected. These rejections led the First Tier Tribunal (FTT) in Matthees to express concern that the researchers may have been cherry-picking who analysed their data to only sympathetic researchers in an attempt to suppress criticism and proper scrutiny of their trial.

An open letter signed by many independent statisticians and scientists, along with MPs, lawyers and patient organizations, has been sent to The Lancet calling for the release of the data for independent analysis. There has been no response.

After a Freedom Of Information Act (FOIA) request by Alem Matthees, the FTT did order the release of some data. In the unanimous part of its ruling, the FTT was clear: there is no legal or ethical consideration which prevents release of the trial data; there is a strong public interest in release; making data available advances legitimate scientific debate; and where possible data should be released.

These data were analysed using the outcomes specified by the investigators in the trial protocol. It was found:

When recovery was defined according to the original protocol, recovery rates in the GET and CBT groups were low and not significantly higher than in the control group.



Since the decision in Matthees, the responsible authority, QMUL, has announced a trial data-sharing policy but has rejected a number of FOI requests for access to more data and continues to deny access to anyone critical of the trial, including highly qualified academics.

A Response to Fiona Fox




























1. Criticism of the trial here:


And here (pdf):




In particular these paragraphs:










The only one to benefit is Parker.






NHS and LP







Over £14,000 wasted on a junk intervention.


The only person to benefit from this costly disgrace is Parker.


See also:




A trial so flawed as to be worthless.












59 did not return consent forms.

These numbers are devastating and turn the trial into a farce:









Third, the trial was unblinded yet used subjective outcome measures:











The Advertising Standards Authority (ASA) guidance is clear:


























A Response to the blog by Puebla and Heber of PLOS ONE




Iratxe Puebla, Managing Editor for PLOS ONE, and Joerg Heber, Editor-in-Chief of PLOS ONE, have written a blog giving their view of the arguments involved and an insight into some of their thinking in issuing the expression of concern.






My response:





The court had two decisions to make:







A Response to Esther Crawley












Sense About Science, the PACE trial and ME.







I asked five questions of SaS:












A number of questions remain for SaS:





Why have they never welcomed the Tribunal decision?


Why will they not say they accept Butterworth’s criticism as valid?

Why do they still refuse to make a fuss about PACE?
