One of the first places that many supplement consumers turn to for guidance on product safety and efficacy is ConsumerLab. The company performs product and category reviews, and publishes research and product testing that is accessible with a paid subscription.
The purpose of this review is to examine the methodologies behind ConsumerLab’s ratings, and assess whether their ratings are trustworthy. You can check out our review of Labdoor, the other popular supplement ratings site, here. This review will cover five categories: credentials, testing methodologies, data quality, ease-of-use and business model.
Unlike Labdoor, ConsumerLab is actually run by credentialed scientists. Their About page informs that the founder Tod Cooperman is an M.D., and their VP has a PhD in pharmacology/toxicology.
Their whole team has impressive credentials which are relevant to their operating model. This is a good sign about their quality of work. We believe strongly that any company informing consumers about health decisions should have scientists or similarly credentialed researchers (PhD, MD) on staff.
ConsumerLab publishes their testing methodologies on their About page, and the specific methodologies used for each product on each product review page. This is important, because transparency is woefully lacking in the supplement space at large.
They primarily test supplements for identity, strength, and purity. These are the most important factors to determine the safety and efficacy of a supplement. ConsumerLab tests for disintegration as well, which is a more ancillary consideration that indicates how well a product dissolves into water but has no effect on health.
The testing methodologies used for their herbal supplement analyses all seem appropriate. ConsumerLab uses high-performance liquid chromatography (HPLC) for quantitative assessment of potency, and spectroscopy (ICP-MS) for heavy metals testing. These methodologies are the industry standard, and we commend them for using accurate scientific methods in their research and analysis.
One the whole, the data quality on ConsumerLab is excellent, well-referenced, and presented in a neat manner. However, in our opinion, there are some improvements that could be made.
In some of their herbal supplement category reviews, ConsumerLab only tests certain subset categories for heavy metal contamination. In their ashwagandha review, for example, they only test products containing “whole herbs” and not extract products for heavy metal contamination. You can see in the image below an “N/A” score for the heavy metals section of an ashwagandha extract product:
Lead is equally harmful whether it’s in an extract or raw powder product. We believe that it would be beneficial for consumers if ConsumerLab tested all products for heavy metal contamination and for pesticide contamination.
It appears there is no way to access the raw data, which could be useful to consumers. Two supplements that “passed” heavy metals testing may have wildly varying levels of heavy metals. Consider a pregnant woman who may want to reduce her lead intake as much as possible.
All things being equal, the health-conscious pregnant woman would prefer a supplement with half the lead as another supplement, even if both “passed” testing. This is why we urge ConsumerLab to publish the raw data.
The product category reviews on ConsumerLab are extensive. There is an entire section that sits above their product test results with background information and a review of published medical research on the supplement category, which often exceeds 1,000 words. This makes mobile use of their site somewhat frustrating.
This contextual information may be useful to some readers, but seems cumbersome in our opinion. The core value proposition of ConsumerLab is providing test results of dietary supplements, and their reviews may be more accessible if they were less text-heavy.
We believe that if ConsumerLab were to remove all of the information above the test results, or house it elsewhere on their site, the product testing pages would be easier to navigate and more useful to readers.
Based on the disclosures on their website, ConsumerLab seems to drive revenue in two ways: first by charging a subscription fee to users to access to their content, and second by charging a fee to manufacturers who want to take part in their certification program.
This business model creates no economic incentives for ConsumerLab to misreport test results, which is great for consumers. They make money by producing great content and test results which consumers pay to access, so it’s in their best interest to publish accurate and thorough test results.
ConsumerLab's economic incentives are aligned in a way that creates less bias than Labdoor's business model.