Is Katherine “3.27-to-1” Baicker Coming in from the Dark Side?

In the workplace wellness epidemic, Harvard School of Public Health Professor Katherine Baicker is Patient Zero. However, she may soon come up with the cure for the disease she spread. This would be like when the inventor of the PSA test said it shouldn’t be used.

By way of background, her original dalliance with wellness in Health Affairs–timed and designed to get the Business Roundtable’s “Safeway Amendment” allowing a 30% to 50% clawback of insurance premiums into the Affordable Care Act–introduced the infamous “3.27-to-1 ROI” into wellness vendor vernacular.  That was almost seven years ago, but wellness vendors continue to cite this figure as gospel even though RAND demolished it and Professor Baicker herself walked it back four times.

For good reason, as it turns out — this Pulse posting says it all.  The 10-to-30-year-old studies comprising her meta-analysis are even more amusing in retrospect.  Lots of overcreening, overreliance on now-discredited BMIs, and of course recommendations for low-fat diets–even for employees with metabolic syndrome. Most published in third-tier journals, by the Wellness Ignorati and their friends.

And almost every one of them compared active motivated participants to non-participants.  6 years ago we suspected that design was invalid.  Today we know for certain, thanks to Health Fitness Corporation and Aetna. Two more elegant studies to prove that point could not be imagined. No “investigator bias” or “publication bias” there, since both were hoping for the opposite finding.

However, mindful of her reputation, Professor Baicker recently announced (in as many words) that she is going to redeem herself.

Wellness Workdays

Her protestation that she is no longer interested in wellness appears to have taken a backseat to her desire to remove this one blot–granted, a blot of Lady MacBeth proportions–on an otherwise excellent curriculum vitae.

She is going to do a study for BJ’s Wholesale Club, in conjunction with an outfit called Wellness Workdays.  Wellness Workdays is a classic wellness vendor. That is to say, they won’t be winning a Nobel Prize anytime soon, or even a spelling bee. Let’s start by examining their analytic and clinical prowess.

To start with, their “White Paper” doesn’t just quote the infamous 3.27-to-1. They’ve upped the ante to 6.00 to 1, maintaining the two significant digits while almost doubling the savings.  How? They’ve added the 3.27-to-1 for healthcare savings to the 2.73-to-1 for absenteeism reduction from that same 2010 study. Those two separate conclusions were reached from almost totally different studies. Anyone can tell that from reading the original.  Anyone, that is, except Wellness Workdays.

Their analytic qualifications are matched only by their clinical qualifications. One member of their medical advisory board is Chief of Allergy and Clinical Immunology at the Indian River Medical Center in Vero Beach, Florida.  While this expertise is not exactly central to the mission of the pry,poke, and prod industry, in all fairness it should be noted that the Indian River Medical Center runs one of the best allergy programs in all of Vero Beach.

Another is an OB-GYN in Colorado.  Perhaps this advisor will develop a protocol for employees who want to be screened and induced at the same time.  A third consults to orthopedists at “Lennox Hill Hospital,” a role that probably doesn’t require too much heavy lifting, because there is no hospital by that name.

This guy is also an expert on steroids and other performance-enhancing products, and has “published rseveral esearch studies.”

So they can’t spell, can’t proofread, can’t understand study design, and can’t cobble together a qualified advisory board.  In other words, to paraphrase the immortal words of those great philosophers Gilbert & Sullivan, they are the very model of a modern clueless wellness vendor.

Katherine Baicker’s proposed study design

We are confident she is going to get it right this time and “discover” that wellness loses money. By selecting a vendor of the caliber of Wellness Workdays, she isn’t leaving anything to chance.

Having learned her lesson — and I’ve been pretty conscientious about forwarding helpful study design materials to guide her — she will certainly tally wellness-sensitive medical events across the entire population. More importantly, here’s what she is not going to do with this study:

• “Match” volunteer participants to — you guessed it — non-participants. This nonsense that keeps the wellness industry afloat, and Professor Baicker no doubt sees right through it;
• List the “unobservable differences” between participants and non-participants as a “limitation.” It’s not a limitation. It’s a baldfaced lie.  It is now known that this face-invalid study design is truly invalid. She wouldn’t lie, right?  That would damage the reputation of the entire Harvard School of Public Health in what will certainly be a high-profile study;
• Show a high ROI even though the change in health risk factors is trivial. This of course is the stock-in-trade of the Koop Award committee, but a real academic researcher would know better;
• Compare the costs to “trend” and say “costs were projected to rise by [this amount] but they only rose by [this lower amount], and therefore a huge pile of money was saved”;
• Attribute all cost differences to the program, whereas a real researcher would look only at utilization differences that could actually be attributed to urging employees to eat more broccoli. (By contrast, one of the studies in her previous meta-analysis credited the wellness program with a reduction in cat scratch fever.)

How do we know she isn’t going to make up phony outcomes again?  She “tipped her hand” with the Oregon Medicaid lottery study.  A terrific natural experiment, her study emphasized the value of a “lottery control,” meaning every subject had the same intent-to-treat. Exactly the opposite of the wellness participants-vs-non-participants study design.

Further, who wants to be known as the Typhoid Mary of workplace wellness? The Oregon study was an excellent one, so naturally it showed exactly the opposite of what wellness studies show. Specifically, facilitating access to care doesn’t reduce the cost of care or improve physical health status. And that was for people — newly minted Medicaid recipients — who didn’t have any insurance to begin with.  Wellness, of course, takes employees who already have plenty of access to care and drown them in even more, unwanted and largely unneeded, screenings and checkups.

As for BJ’s Wholesale Club, I suspect they got suckered into this. Who volunteers to become the next Pepsico, a case study of how wellness programs fail?  BJ’s obviously isn’t studying their own competitors: Target has one of the best programs in the country — precisely because they are far too smart to use a vendor like Wellness Workdays.

In any case, we look forward to her research study. Or perhaps, since this is in conjunction with Wellness Workdays, to rseveral esearch studies.