They Said What?

Home » Uncategorized » Are Livongo’s outcomes real?

Are Livongo’s outcomes real?

Do you know whether heartburn pills are safe for long-term use?

Quizzify knows. Click to learn more.

Breaking news first…

Sign up for the Validation Institute webinar and learn why Yale got sued for wellness because they made a series of rookie mistakes…and how you can avoid them.

Attention, Pennsylvanians: register now for the Pittsburgh Business Group on Health Symposium 9/12…and play in our trivia contest.


And, now, for the rest of the story.

Kudos to employers who have resisted the entreaties of their carriers to jump on the Livongo bandwagon. Resistance is not easy — carriers will pester employers endlessly because they get a nice commission every time an employer bites.

Instead, I would simply recommend waiting until Livongo answers these five questions and gets their results validated by the Validation Institute. One would think that if they are losing $30-million/year anyway, spending an extra $10,000 to see if their offering actually works shouldn’t break the bank.

Here are the five questions that Livongo apparently can’t answer. Or they can, but choose not to. Not sure which inspires less confidence.

The references below mention a “study.”,  By way of background, this study was conducted by Livongo’s employees, along with employees of its partnered diabetes supply company (Eli Lilly), which also funded the study. So there couldn’t possibly have been any conflict of interest, right? Right?

It was published in something called the Journal of Medical Economics (JME). And no, I hadn’t heard of this publication either. Turns out it’s an “open-access” journal offering “accelerated publication,” with an Impact Factor of 1.9.

Not familiar with the concept of Impact Factors? Those measure the influence of a publication. For instance, the New England Journal of Medicine tallies a 70.8. How hard is it to only get a 1.9? Even the American Journal of Health Promotion (AJHP), which recently proposed charging employees for insurance by the pound, scratches out a 2.6.  Possibly, that’s because AJHP does sprinkle more humor into its content than JME, like:

I guess that means 10 states both prohibit it and allow it.


Question #1 for Livongo

Even obscure journals need to do a little peer review (which itself is increasingly considered to be a joke), and this peer review was pretty basic: the authors were asked to disclose that the study couldn’t draw any causal relationship between the Livongo intervention and the results. The title of the article itself says that reduced medical spending is “associated with” their product. Later the authors say the results “imply” the product works.

So if the study showed only a correlation and not causation, why does Livongo’s press release announce:

“The findings showed that by using its remote digital health platform, the Livongo for Diabetes program delivered an $88 per member monthly reduction” ?


Question #2 for Livongo

Since these alleged findings are the opposite of Livongo’s initial claim below, featured in a recent Valid Points posting about diabetes vendors snookering purchasers, the second question is, how did the original 59% reduction claim (which basically requires eliminating every hospitalization not connected with births, trauma or cancer) get replaced by the opposite claim that large reductions in physician visits generate all the savings, while admissions increased?

A while back, I actually looked into the likelihood that tighter glycemic control, which itself is rather controversial, (“there is good evidence that intensive control of blood glucose increases patients’ relative risk of severe hypoglycemia by 30%”) could reduce hospitalizations. I used very optimistic assumptions (since I was consulting for a company making a glucometer with remote capabilities, not unlike the one Livongo pitches). Here’s what I came up with: savings of $27 per diabetic per year in inpatient admissions, as opposed to Livongo’s $88 per diabetic per month.

In all fairness to Livongo, they don’t promote that inpatient admissions result anymore (I may have missed the apology for fudging that outcome in the first place), focusing instead on getting doctors to do fewer visits by managing patients remotely.

Seems curious that physicians would be doing more work – more notifications from remote monitoring devices about blood sugar, more titrating dosages – and be perfectly fine making 26% less money. Plus, every other wellness vendor brags about how many more physician visits they generate, not how many fewer.


Question #3 for Livongo

If I’m seeking a vendor to reverse diabetes in my population (assuming that is even possible on a broad scale), I would look for weight loss (without which diabetes reversal is basically impossible, but then again sustained weight loss itself is pretty close to impossible) as the leading indicator.

As an intermediate indicator, I would measure units of insulin use across the entire population. That should decline if people are eating much better an exercising much more and losing weight.

The end-point indicator would be a decrease in admissions for diabetes. Of course, since diabetes admissions generally increase following retirement, it isn’t exactly easy to save money in decreased admissions for diabetes, for the simple reason that there hardly are any.

Specifically, for the last year in which a full set is available (2014),158 million commercially insured <65 people generated 126,710 admissions for diabetes. Meaning that in the commercially insured population, the admission rate is so low that a 10% reduction in admissions (which has never been achieved in any population health program) would mean that an employer with 10,000 employees would avoid — get ready — 1 admission.

We suspect Livongo knows this because they listed the diabetes diagnosis codes in their appendix, and it takes about 5 minutes to tally the US admission rate for those codes using the federal database designed for that purpose. And if they don’t know it, they should. Any health services researcher should be aware of this database.

So why didn’t Livongo measure any of those three outcomes? Or did they measure them and decide not to report them?

I’m not sure which answer is “right”: While I would be very concerned if they were suppressing data, I think I would be even more concerned if they didn’t know enough about diabetes to measure the key outcomes.


Question 4 for Livongo

Why did Livongo measure participants against non-participants, when that study design is known to be completely invalid?  Benchmarking a par-vs-non-par result has been done five times, including three times by wellness promoters hoping to prove validity of the design  The conclusion in each case: 100% of difference in outcomes between the two groups is attributable to the study design, and 0% to the intervention. The design is 100% invalid.

There was actually a case in which the two groups were separated and “match-controlled” in 2004, but the program didn’t start until 2006. During the 2-year period following separation but before the program became available to participate in, the would-be participants nonetheless dramatically outperformed the non-participants…despite not having a program to participate in. (This result caused quite a ruckus in the wellness industry once they realized what they had done.)

Further, more than 15% of the initial Livongo participants dropped out. Assuming dropouts largely failed, aren’t the authors of the Journal article overstating the outcomes for the participant group as a whole by not counting dropouts? Isn’t that like on-time performance statistics not accounting for planes that crashed?


Question 5 for Livongo (really multiple questions)

Livongo offers “free unlimited glucose test strips” to try to reach a “target level of glucose control of Hb A1c <6.5%.”

It is not quite clear that either unlimited free strips or a 6.5% Hb A1c target are good for diabetics. Why, one might ask, does the American College of Physicians propose 7% to 8% instead of 6.5%? While it is true that the American Diabetes Association is sticking with its much lower blood sugar goal, isn’t it also the case that the ADA is largely funded by companies that make products to help lower blood sugar?

And what is the rationale for encouraging more use of glucose strips while Choosing Wisely recommends less testing for many Type 2 diabetics (in JAMA Internal Medicineimpact factor 20)?  Exact words:

This recommendation is based on robust evidence, including a Cochrane review of 12 randomized clinical trials with more than 3000 patients, showing no statistical difference between patients who do not self-monitor their blood glucose multiple times per day and those who do self-monitor their blood glucose multiple times per day in glycemic control, nor evidence of effects on health-related quality of life, patient satisfaction, or decreased number of hypoglycemic episodes.


So it could be that this is just me, since their revenues are doubling every year. Maybe, like in the rest of wellness, answering questions about efficacy is beside the point. Maybe the point in HR is just to do something, regardless of whether it works, in case someone in the C-Suite asks. Or in the immortal words of the great philosopher Yogi Berra: “We don’t know where we’re going, but we’re making good time.”


In the immortal words of the great philosopher Pat Benatar, hit me with your best shot.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: