Information is Beautiful: Snake-oil or Supplement?

On Facebook, Highlight HEALTH points to this interesting data visualization. It's an interactive chart of health supplements and the evidence supporting their effectiveness for a given condition (e.g. green tea for cholesterol reduction). The higher the balloon "floats", the better the available evidence and the larger the balloon, the more popular (as measured in google hits) the intervention is. This is the statement on the evidence used to determine how high a balloon ranks.

We only considered large, human, randomized placebo-controlled trials in our data scrape – wherever possible. No animal trials. No cell studies. Many of the health claims made by the $23 billion supplements industry are based on non-human trials. We wanted to cut through that.

This piece was doggedly researched by myself, and researchers Pearl Doughty-White and Alexia Wdowski. We looked at the abstracts of over 1500 studies on PubMed (run by US National Library Of Medicine) and Cochrane.org (which hosts meta-studies of scientific research). It took us several months to seek out the evidence – or lack of.

You can see our key results in this spreadsheet. (It’s the same spreadsheet that generates the interactive image).

There's also a "worth it line" but no telling what that line actually means. It's a pretty cool way to visualize the data (personally I might have had the bubble size correspond to evidence rather than google hits since my eye is drawn more to that than the vertical scale). However, it suffers from the same problems as any attempt to simplify information... it's simple. Clicking on a bubble will take you to a single abstract supporting or refuting the particular supplement. Presumably more than one study is involved in each determination but nowhere, not even in the data used to make the chart (that is freely accessible on the site), is there an explanation of quantity or how quality was assessed. (Also, a nit-pick: while it says only randomized controlled trials (RCT) with placebo controls are considered, the first bubble I clicked on [green tea for cholesterol] took me to a non-RCT).

Complaints aside, it's a nice visualization, if limited. Clearly the designers aren't setting out to do formal systematic reviews for each indication, and by no means is it intended to be a final word resource for these things, but is more useful as an interesting starting point to think about these supplements. More importantly, it encourages thinking about what evidence exists, and highlights the fact that many of these things can and are tested - as any health intervention should be.