EBM, SBM and Bias

EBM, SBM and Bias

In biomedical research there are two ongoing problems. The first is funding. For obvious reasons, many of the major drug trials are funded by the companies who developed the drug. Even assuming that the organizations are run by saints who only want to do what is best for patients, bias will seep into the studies. It is well known that Pharma funded studies show better results than independedly funded studies. That does not necessarily invalidate the study. It is rare that one study proves or disproves the efficacy of an intervention. But my rule of thumb is that in the real world an intervention is about half as good as the Pharma sponsored study would indicate.

The more troublesome problem is hidden data. All the studies that are not published and could help inform clinicians as to efficacy and side effects of medications. You can compensate for errors and bias if the data is available but not if it is unpublished. This problem is well described in the most excellent Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients.

A short review of both issues was recently published, How evidence-based medicine is failing due to biased trials and selective publication.

They offer six solutions to the problem, all of which are reasonable but highlight an issue of EBM vrs SBM:

1 The sensible campaign to formalize and enforce measures ensuring the registration and reporting of all clinical trials should be supported – otherwise trials that do not give the answer industry wants will remain unpublished.

2 More investment in independent research is required. As we have described, it is a false economy to indirectly finance industry-funded research through the high costs of patented pharmaceuticals.

3 Independent bodies, informed democratically, need to set research priorities.

4 Individuals and institutions conducting independent studies should be rewarded by the methodological quality of their studies and not by whether they manage to get a positive result (a ‘negative’ study is as valuable as a ‘positive’ one from a scientific point of view).

5 Risk of bias assessment instruments such as the Cochrane risk of bias tool should be amended to include funding source as an independent item.

6 Evidence-ranking schemes need to be modified to take the evidence about industry bias into account.

The problem with these EBM suggestions is they forget that there is a cadre of reseachers who work with the implausible and the impossible and have a profound bias in favor of their pseudo-medicine that is more powerful than financial influences:

Conflicts of interest are very common in biomedical research , and typically they are inadequately and sparsely reported. Prejudice may not necessarily have financial roots. Scientists in a given field may be prejudiced purely because of their belief in a scientific theory or commitment to their own findings. Many otherwise seemingly independent, university-based studies may be conducted for no other reason than to give physicians and researchers qualifications for promotion or tenure. Such nonfinancial conflicts may also lead to distorted reported results and interpretations.

 

The suggestions are incomplete.  I would add to 6 as follows: For the research in implausible pseudo-medical interventions,  evidence-ranking schemes need to be modified to take the evidence about industry bias,  prior plausibility and professional bias into account.

The risk of bias from study of homeopathy by homeopaths or acupuncture by acupuncuritsts should push a ranking for any such study into the fuhgeddaboudit category, espcially when prior plausibility is taken into account.

To modify their conclusion with an SBM perspective:

Through processes of selective publication and manipulation of study design, pseudo-medical-sponsored studies are weighted to be favourable to their product. Simply put, pseudo-medical–sponsored evidence is incomplete and biased. Most intervention studies are pseudo-medical- sponsored. This means that the overall evidence about many interventions is incomplete and biased.

EBM has been an important advance in medicine but often makes the understandable mistake of not including the alternative reality of pseudo-medicine.

 

Points of Interest 5/16/2014
Points of Interest 5/15/2014