The Cochrane Collaboration has established a unique reputation for the publication of reports on a variety of topics in the medical field. This is an area where there are very powerful vested interests and the quality of much of the research is questionable. Therefore the contribution of the Collaboration has proved to be extremely valuable with respect to decisions on treatment and policy formulation.
In recent years the Collaboration has produced two reports which focus on the use of statins for the primary prevention of heart disease. In the first one it was concluded that caution should be taken in prescribing statins for primary prevention among people at low cardiovascular risk (1). However in the second on e published 2 years later, the conclusion was that the totality of evidence now supports the benefits of statins for primary prevention(2). In this blog I will attempt to discover why and how there has been such a radical shift in position.
At the outset I should emphasise, that I consider the critical information to assess the value of these drugs is all-cause mortality. First of all, there can be no argument about the validity of the “diagnosis”. Secondly, this is what most patients will wish to know. It not much comfort if a reduction in the risks of death from heart disease is counterbalanced by an increased risk of dying from another disease. It is also important to have reliable information on adverse side-effects. Many patients are likely to question the value of an extra few months on this earth if the quality of life is made miserable by the drugs.
The first report which was released in January 2013 included research up to September 2007. This was based on 14 trials (with 16 trial arms) involving 34,272 participants. The mean age of the participants was 57 years (age range 28-97), 65.0% were male. However data on all-cause mortality was only provided in 8 trials. Furthermore in 2 trials which accounted for over one quarter of the participants were stopped prematurely.
The report noted that all but one of the trials had some form of pharmaceutical industry sponsorship. Research has shown that the results of trials sponsored by the pharmaceutical industry- are more likely than non-industry-sponsored trials to report results and conclusions that favour drug over placebo because of biased reporting and/or interpretation of trial results (3). As a consequence:
“In primary prevention where world-wide the numbers of patients eligible for treatment are massive, there might be motivations to use composite outcomes and early stopping to get results that clearly support intervention.”
The authors concluded that the review:
“highlights the shortcomings in the published trials of statins for primary prevention. Selective reporting and inclusion of people with cardiovascular disease in many of the trials included in previous reviews of their role in primary prevention make the evidence impossible to disentangle without individual patient data. In people at high risk of cardiovascular events due to their risk factor profile (i.e. 20+% 10-year risk), it is likely that the benefits of statins are greater than potential short term harms although long-term effects (over decades) remain unknown. Caution should be taken in prescribing statins for primary prevention among people at low cardiovascular risk.”
As soon as the report was published, there was an objection from Rory Collins and Colin Baigent of the Cholesterol Treatment Trialists’ (CTT) Collaboration at the University of Oxford. Their concern was the report had stated that the CTT collaborators had not published information about the proportional and absolute benefits of statin therapy among people with no prior history of vascular disease. They referred specifically to a paper published in The Lancet in 2010.They also objected to a statement in the press release announcing the release of the report which was as follows:
“Given that low cholesterol has been shown to increase the risk of death from other causes, statins may do more harm than good in some patients”.
In reply the lead author Shah Ebrahim pointed out that the CTT Lancet paper of 2010 was not available at the time the review was completed. However he insisted that the estimate of the effects of lowering LDL cholesterol with respect to major vascular events, contained in the review, was similar to that of the CTT. He also re-iterated his surprise that the CTT did not provide more information on other outcomes among participants taking statins for primary prevention. He drew attention that others have raised the issue of all-cause mortality in primary prevention trials (4) and that there are particular concerns about increased risk of diabetes in people who take statins (5).
With respect to the press release, the point was accepted and a correction was issued.
Now we come to the second report which was published in January 2013 (2). This included research which was published up to the beginning of January 2012 and was based on 18 trials (with 19 trail arms) with 56,934 participants. The mean age was 57 years and 60.3% were men.
Thirteen trials with 48,060 participants recruited reported on total mortality. In the statin group 1077 out of 24,408 people died (4.4%) whereas in the control group there were 1223 deaths out of 23,652 (5.1%). This translates into an Numbers Needed to Treat (NNT) of 96 which means that 96 people have to be treated for 5 years for one to benefit. Furthermore it is crucial to appreciate that this means 96 people have to be treated for 5 years for one to benefit.
There was a reduction in various end-points related to cardiovascular disease (CVD) which were associated with falls in total cholesterol (TC) and LDL cholesterol in trials which reported these outcomes. The review concluded that adverse side-effects were not excessive although not all trials reported these fully. . Patients’ perception of quality of life was reported in only one trial and this showed limited benefit.
It is also interesting to note that in this report it was stated that:
“there was low risk of bias ….though all trials were either fully or partially funded by pharmaceutical companies”
What is especially noteworthy is that this review makes detailed reference to the work of the CTT.
- The CTT Collaboration has published analyses focusing on the comparison between high and low doses of statins which demonstrate that more intensive treatment lowers LDL cholesterol more, resulting in greater benefits with no excess risk of non-vascular mortality (6).
- Strong evidence of the absence of any adverse effects on cancer risk is also confirmed by a further CTT Collaboration report (7).
- The Cochrane estimates of the effects on all-cause mortality are in agreement with those of the CTT (8). Although it does accept that there is an increased risk of myopathy and rhabdomyolysis in those treated with statins especially in those treated with high (9). These benefits equate to an NNT of 167 for people with a <5% five year risk for heart disease and 67 for those with a 5-10% risk. This means that for those with the lower risk 167 people have to be treated for 5 years for one to benefit and 67 for those with the higher risk
The final conclusion is that benefits of statins outweigh any serious adverse effects. Hence earlier claims that statins provide no overall benefit in primary prevention in terms of all-cause mortality can no longer be substantiated.
This represents an about turn in the space of 2 years. Here are a few comments which I believe are worth addressing if you are mystified by this complete reversal in the position reached by the Cochrane authors.
It is evident that the research from the CTT has been highly influential with the authors when coming to their final conclusion. I just wonder if they were aware of the criticisms of the work of the CTT by Dr David Newman (10). He points out that the CTT does not compare those who were on statins with the corresponding controls. Instead they selected those individuals in which the statin treatment reduced the LDL Cholesterol 1 mmol/L or 40 points in US terms. As Dr Newman describes it:
“…they shifted the data so that their numbers corresponded precisely to patients whose cholesterol responded perfectly.”
“Patients whose cholesterol drops 40 points are different than others, and not just because their body had an ideal response to the drug. They may also be taking the drug more regularly, and more motivated. Or they may be exercising more, or eating right, and more health conscious than other patients. So it should be no surprise that this analysis comes up with different numbers than a simple comparison of statins versus placebo pills. Ultimately, then, this new information tells us little or nothing about the benefits someone might expect if they take a statin. Instead it tells us the average benefits among those who had a 40-point drop in LDL.”
Furthermore the LDL Cholesterol drop cannot be predicted because the effect of the statins is unknown. This means that the conclusions of this analysis have absolutely no relevance to patients and doctors who have to decide if statins should be prescribed.
Here is a final quote from Dr Newman:
“Perhaps never has a statistical deception been so cleverly buried, in plain sight. The study answers this question: how much did the people who responded well to the drug benefit? This is, by definition, a circular and retrospective question: revisiting old data and re-tailoring the question to arrive at a conclusion. And to be fair they may have answered an interesting, and in some ways contributory, question. However the authors’ conclusions imply that they answered a different, much bigger question. And that is not a true story.”
It seems to me that this is a very valid criticism because the CTT studies do not include those who were on statins but failed to lower their LDL Cholesterol. I am not aware that anyone has answered Dr Newman’s critique. Incidentally this would explain why the CTT does not agree with other analyses which find that the use of statins as a preventive measure does not reduce all-cause mortality.
The CTT data on side- effects are entirely dependent on the results which were obtained from trials. Serious doubts have been raised. Rather ironically the second Cochrane report accepted that the reporting of adverse events in these trials is generally poor, with failure to provide details of severity and type of adverse events or to report on health-related quality of life. It then went on to conclude rather complacently:
“However, it seems unlikely that any major life-threatening hazards associated with statin use exist”.
Recently Sir Richard Thompson , President of the Royal College of Physicians and colleagues wrote to NICE expressing concern about the proposal to extend the use of statins to those with a low risk of developing CVD (11). The letter commented specifically on side-effects as follows:
“I am in no doubt, both personally and from the reported experience of my patients, that statin side-effects are consistently under-reported. This occurs two ways; firstly, doctors airily dismiss symptoms reported by patients (“statins could not possibly cause that!”), and secondly it is unusual for well-recognised side-effects such as rhabdomyolysis or even non-specific muscle pains to be yellow-carded. In my view the potential severity of muscle symptoms justifies extreme caution”
The argument was supported by information on the number of adverse side-effects in various stain trials, which is shown here in Table 1.
Table 1 Adverse side-effects reported in various statin trial
|Trial,||Name of statin||Side-effects, statin treatment,%||Side-effects,control,%|
|JUPITER||Rosuvastatin||25.0,discontinuation,15.0,serious side-effects||25.0, discontinuation,!5.5, serious side-effects|
Note: The values shown refer to “total adverse effects” unless otherwise indicated
What is quite remarkable about these figures is the similarity between those for the group treated and those for the controls. Furthermore the variation is huge. In one the values are as low as 3% but in another it is over 80%. The reality is that the companies have little interest in collecting data on side-effects. The information shown above clearly demonstrates the information on side-effects obtained in clinical trials simply lack credibility.
The letter also referred to the fact that independent researchers do not have access to raw data and noted that:
“…the data driving NICE guidance on statins comes almost entirely from pharmaceutical company funded studies. Furthermore, these data are not available for review by independent researchers, only those who work for the Oxford Cholesterol Treatment Trialists Collaboration (CTT).
The CTT has commercial agreements with pharmaceutical companies which apparently means that they cannot release data to any other researchers who request to see it. Which, in turn, means that the latest reviews of the data by NICE and also by the Cochrane group are totally reliant on the CTT 20121 meta-analysis analysis of this concealed data?”
It is highly relevant that in the recent BMJ controversy it has emerged that the CTT at Oxford University has had funding from drug companies of about £268million as Zoe Harcombe has revealed (12).
There have to be very strong suspicions that the Cochrane authors have allowed themselves to be bullied into submission by the actions of Rory Collins and Colin Baigent. It is evident from the information I have presented here that they have ignored key considerations, which ought to have had an impact in reaching their conclusions. Perhaps they should have studied another Cochrane publication entitled “Eminence v Evidence” (13) which makes some extremely pertinent comments.
This is a great pity because if ever there was a need for a body which can conduct an authoritative evaluation of the evidence it is now. It is to be hoped that the powers that be in Cochrane will take these points on board. Above all its contributors must stand up to bully boy tactics of the very powerful vested interests. Many will be familiar with the fact that Rory Collins tried the same tactic with the BMJ. To her great credit Fiona Godlee , the editor-in-chief was not intimidated by Collins and effectively took appropriate action which put him in his place. Cochrane could well to learn some lessons from this.
Finally, even if the conclusions of the second Cochrane report are genuine, let us consider this from the perspective of those who will be prescribed statins in accordance with the latest NICE guidelines. These mean that those with a 10% risk of developing CVD will be advised to go on statins. Effectively this means everyone over 50, because no matter how fit and healthy a person is, when the time comes there must be at least a 1 in 10 chance that the cause of death will be heart disease. Cochrane states that 67 people will have to be treated for 5 years for one person to benefit. Although according to Mark Baker of NICE, 77 will have to be treated for 3 years. Whatever the figure the chances are very small. Cochrane comments:
“These NNTs are well within the range considered worthwhile in primary prevention”
Well I have got news for the authors and for the medical professions as a whole if this is statement is an accurate reflection of its opinion. It is certainly not acceptable to me and I strongly suspect to the vast majority of patients. In my experience most people believe that if they are prescribed a drug it will effective for them personally. They might be prepared to accept that there may be less that 100% certainty, possibly as low as 20%, which is an NNT of 5! It is crucial that we are concerned here with prevention. I recognize that it is rather different for a person with a serious illness, who will clutch at anything that may be effective. But prevention is a new ball game and in my opinion, there would be very few who would agree to statins if they were aware that there was only a 1 in 67 chance that it would do them any good.
For anyone who is concerned that their risk of CVD is high and they wish to lower them, what should they do? Here are some suggestions:
- If you smoke cigarettes, stop immediately
- Take regular exercise, If you can walk at least 1 mile per day, this will certainly imlrove your life expectancy
- Make adjustments to your diet. There is now very convincing evidence that restricting the intake of sugar and refined carbohydrates will reduce the risks not only of CVD but also of diabetes, cancer, Alzheimer’s Disease as well as lowering blood pressure and achieving weight loss
There is absolutely no doubt any benefits derived from statin treatment will be insignificant when compared those which can be achieved by implementing the measures outlined above. Furthermore, any side-effects will be minimal compared to those experienced by people on statins.
So whatever way you look at it, the case for statins just does not stack up. The truth is that the fundamental objective is the profitability of the drug companies and improving public health simply does not feature. In fact, if statins were only used to treat patients suffering from CVD, this could be a critical step towards good health. It would definitely release funds which could be better in spent in other ways, such as employing more nurses.
It is a matter of great regret that this particular Cochrane team did not follow the example of the one led by Tom Jefferson, which considered neuraminida inhibitors including Tamiflu (14). The background has been explained in an article in The Guardian by Ben Goldacre (15). An earlier Cochrane report had concluded that these ‘flu drugs do reduce the rate of complications of conditions such as pneumonia. However a paediatrician in Japan pointed out that the Cochrane report was based on the results of 2 published trials of which only one showed a positive response. In fact there were another 8 trials, for which the results had never seen the light of day and all of these had been negative!! As a direct result of this Tom Jefferson realised that in the 2008 report they had made a mistake in relying on the published data. So he decided to see if he could get hold of the information. There was a long battle with Roche, which by this time controlled the data, before the company eventually sent documents containing excerpts, which did not provide adequate information to assess the benefits and the adverse side-effects. It was also not possible to consider the details of the trial procedures, which is essential if the quality of the data is to be evaluated.
Consideration of the work of various regulatory agencies such as the FDA in the USA and EMEA in the EU indicated that there were serious flaws in the procedures. As Cochrane had not been provided with the detailed information it needed to do its review properly, it decide to exclude all this data from the analysis. Effectively it was unable to reach a conclusion and explained the position in a further review published in December 2009. This was followed by a long period of prevarication by Roche, including claims that some of the Cochrane researchers had made untrue statements about the drug and the company and allegations questioning their independence. Finally under pressure from the BMJ which supported the Cochrane investigators, Roche agreed to release all the data but it had taken 5 years for this to happen.
When this information was evaluated, the review was finally published in April 2014. Because the team now had the clinical study reports (CSRs) from the manufacturers it was able to compare the information with that which had been submitted to the regulatory authorities. Rather significantly it was decided not to use results which appeared in scientific journals because in previous versions of this review unresolved discrepancies in the data presented were identified in published trial reports. Substantial publication bias was also uncovered.
Based on these assessments of the regulatory documents (in excess of 160,000 pages), it was concluded that there were substantial problems with the design, conduct, reporting and availability of information from many of the trials. As a consequence, there were doubts about the reliability of the results. The review was based on 20 trials with zanamir (Relenza, from GSK) and 26 trials with oseltamir (Tamiflu fom Roche). It was found that:
“ both drugs shorten the duration of symptoms of influenza-like illness (unconfirmed influenza or ‘the flu’) by less than a day. Oseltamivir did not affect the number of hospitalisations, based on the data from all the people enrolled in treatment trials of oseltamivir. Zanamivir trials did not record this outcome. The effects on pneumonia and other complications of influenza, such as bronchitis, middle ear infection (otitis media) and sinusitis, were unreliably reported, as shown by the case report form in the trial documents. Some forms showed limitations in the diagnostic criteria for pneumonia. Regulatory comments noted problems with missing follow-up diary cards from participants. In children with asthma there was no clear effect on the time to first alleviation of symptoms.
Prophylaxis trials showed that oseltamivir and zanamivir reduced the risk of symptomatic influenza in individuals and households. There was no evidence of an effect on asymptomatic influenza or on non-influenza, influenza-like illness, but trial conduct problems prevent any definitive conclusion.
Oseltamivir use was associated with nausea, vomiting, headaches, renal and psychiatric events; these last three were when it was used to prevent influenza (prophylaxis). Its effect on the heart is unclear: it may reduce cardiac symptoms, but may induce serious heart rhythm problems. In adult treatment trials of zanamivir there was no increased risk of reported adverse events. The evidence on the possible harms associated with the treatment of children with zanamivir was sparse.”
This is not exactly impressive. Roche put out a press release contesting these findings but did not provide any reasons. When the benefits are balanced against the side-effects, it is difficult to justify their use. It was certainly a huge waste of money by the government to spend £500 million to stockpile supplies of this drug. But that is what happens when all the relevant data to make an informed decision is not available.
When this episode is compared with the performance of the team dealing with statins, a sad state of affairs emerges. In contrast to Tom Jefferson, the statin team seems to have just submitted to the power and influence of the industry and its supporters. There has been a total failure to insist on getting access to the primary data. As the Tamiflu review clearly demonstrates when the original reports become available for assessment, the perspective is completely different. In the light of this, there have to be very strong suspicions that if and when the relevant statin data is released generally, we will find that the benefit/adverse side-effects ratio is very much worse than we have been led to believe.
Commenting on the Tamiflu review, Dr David Tovey, Editor-in-Chief of The Cochrane Library said:
“We now have the most robust, comprehensive review on neuraminidase inhibitors that exists. Initially thought to reduce hospitalisations and serious complications from influenza, the review highlights that [NIs are] not proven to do this, and it also seems to lead to harmful effects that were not fully reported in the original publications. This shows the importance of ensuring that trial data are transparent and accessible” (17).
Let’s hope that one day he will be able to say the same about statins.
- Lancet 2012;378:doi:10.1016/S0140–6736(12)60367-5