Testimonials are the heart and soul of pseudo-medical treatments. Since they are not based in reality, pseudo-medicines always fail when examined by well designed clinical trials. But that doesn’t matter since there are always glowing testimonials as to the benefits of the pseudo-medicine. Every pseudo-medical web site has its testimonial page and a single testimonial will always be more highly regarded than the best clinical trial in the NEJM.

Who are you going to believe, a patient with a compelling story of illness and recovery or some know-it-all white coat smarty pants doctor? The patient of course.

Testimonials are powerful affirmations even with the topic is fantasy. These positive stories serve to perpetuate the nonsense of pseudo-medicines and the effect is perhaps magnified by the internet, as demonstrated in How Feedback Biases Give Ineffective Medical Treatments a Good Reputation

Background: Medical treatments with no direct effect (like homeopathy) or that cause harm (like bloodletting) are common across cultures and throughout history. How do such treatments spread and persist? Most medical treatments result in a range of outcomes: some people improve while others deteriorate. If the people who improve are more inclined to tell others about their experiences than the people who deteriorate, ineffective or even harmful treatments can maintain a good reputation.

Objective: The intent of this study was to test the hypothesis that positive outcomes are overrepresented in online medical product reviews, to examine if this reputational distortion is large enough to bias people’s decisions, and to explore the implications of this bias for the cultural evolution of medical treatments.

Methods: We compared outcomes of weight loss treatments and fertility treatments in clinical trials to outcomes reported in 1901 reviews on Amazon. Then, in a series of experiments, we evaluated people’s choice of weight loss diet after reading different reviews. Finally, a mathematical model was used to examine if this bias could result in less effective treatments having a better reputation than more effective treatments.

Results: Data are consistent with the hypothesis that people with better outcomes are more inclined to write reviews. After 6 months on the diet, 93% (64/69) of online reviewers reported a weight loss of 10 kg or more while just 27% (19/71) of clinical trial participants experienced this level of weight change. A similar positive distortion was found in fertility treatment reviews. In a series of experiments, we show that people are more inclined to begin a diet with many positive reviews, than a diet with reviews that are representative of the diet’s true effect. A mathematical model of medical cultural evolution shows that the size of the positive distortion critically depends on the shape of the outcome distribution.

Conclusions: Online reviews overestimate the benefits of medical treatments, probably because people with negative outcomes are less inclined to tell others about their experiences. This bias can enable ineffective medical treatments to maintain a good reputation.

This surprised me as I would have thought that it would have been the dissatisfied who would be more likely to write negative reviews. It probably depends on the reason for the review.  People tend to proselytize enthusiastically about perceived positive life changing events.

So the internet, by emphasizing the positive reviews leads to a

feedback bias (that) may be one reason that people have unrealistically high expectations of weight loss diets and other medical treatments.

Homeopaths are taking this to heart with the The Making Cases Count initiative.

They would like to utilize the case reports, espcially patient stories,  to support homeopathy as:

the patient's voice is the trusted, indisputable and easily understood common ground in homeopathy. Yet, the experiences of patients are rarely heard outside the profession of homeopathy. Homeopaths are in a unique position to make these voices heard by disseminating the results of their routine practice cases incorporating their patients’ voices.

Given homeopathies complete disconnect from reality, testimonials, the weakest and most prone to bias form of evidence, is all they have.

The goal is to collect all these case reports

in a format which will then be able to be transformed (i.e. anonymised, summarised and counted). This routine data requires numbers and categories to report the behavior and the perspective of patients receiving homeopathic treatment.

No surprise that homeopaths would be unaware the plural of anecdote is anecdotes, not data. Still, it will make an excellent advertising gambit. No more valid than the idea that a pretty woman will stoke my cheek after a close shave, but advertising is about image, not reality. Like homeopathy.

Points of Interest 8/29/2014