Home » Posts tagged 'Katherine Baicker'
Tag Archives: Katherine Baicker
We’ll go walking out
While others shout of war’s disaster
Oh, we won’t give in
Let’s go living in the past
A delegation from the Validation Institute attending the World Health Care Congress time-traveled over to the US Chamber of Commerce Monday to watch Ron Goetzel and a few acolytes try to re-create the shiny new wellness object from the 1980s. Just like in the 1980s, the theme was, and these are the exact words of one of the speakers: “Why aren’t more employers seeing the value in wellness?”
Um, because there isn’t any? 7 years after I first proved, and guaranteed, that none of this “pry, poke and prod” nonsense could possibly add up, the evidence has finally caught up with the conclusion.
Of course, there is value in wellness done for employees, rather than to employees. And far be it from me to discourage anyone from being screened according to guidelines. But that’s not what the US Chamber is all about. The US Chamber is about pressuring the EEOC into restoring the punitive “voluntary” wellness penalties that a federal judge put the kibosh on in December 2017, effective this year.
The irony is, the Chamber is its own worst enemy. If the EEOC ever got hold of a recording of that session, it would constitute an irrefutable argument for the EEOC not to change the rules. The speakers other than Ron were all about improving employee well-being through autonomy, independence, responsibility for projects from beginning to end, and creating a culture of health. All those are worthy goals. Goals which, more than coincidentally, have nothing to do with screening the stuffing out of the workforce and penalizing employees 30% for not losing weight. (Shame on Oprah Winfrey!) Quite the contrary, any organization trying to achieve those goals would never dream of manhandling their employees like that.
Ron Goetzel: Katherine Baicker is living in the past
If a fortune-teller had predicted the following as recently as three weeks ago, I would have demanded a refund: Ron Goetzel dissed Katherine Baicker., Specifically, he accused Professors Baicker and Song, along with BJ’s and their vendor, of “doing a program out of the 1980s,” saying that their outstanding study could safely be ignored. (These are the same 1980s in which wellness, according to its promoters, couldn’t fail.)
What horrible 1980s elements did the BJ’s program have? Coaching, screening, learning “modules” and HRAs. Of course, no one has included those things in a wellness program for at least 30 years. (not)
Curiously, Ron didn’t have any issue at all with Baicker’s original 2010 “Workplace Wellness Can Generate Savings” article, which was chock full of programs literally from the 1980s. Only 2 of the programs analyzed in that study were conducted in the 21st century.
If that fortune-teller had also predicted that Ron would start telling the truth someday, I would have demanded double my money back. But no fortune-teller would ever predict that. Fortune-telling industry malpractice insurers will no longer cover lawsuits arising out of that prediction.
And for good reason. It’s generally a safe bet, given Mr. Goetzel’s tenuous grasp on integrity, that he’s going to misrepresent something. Yesterday, as is his wont, he deliberately misrepresented the position of The Incidental Economist (TIE) on the subject of wellness. He quoted TIE as saying that wellness “usually” doesn’t save money. (As a sidebar, I’d note that it is a bit of a head-scratcher that even the misrepresented TIE position makes Goetzel’s position look bad.)
Perhaps I need new glasses, but I can’t seem to find the word “usually” in the most recent TIE article on wellness savings:
There’s another reason Ron doesn’t quote TIE, which is they completely dissed him and his Koop Award friends in that article. Totally worth a read.
What does a Goetzel program look like?
So how do you succeed? How do you avoid programs “from the 1980s”? Maybe you avoid that by using a vendor where Ron has a “strong working relationship,” like Vitality. And just to be on the safe side, maybe you report only the results from Vitality’s own program for Vitality’s own employees. That way you can’t fail…
…and yet fail they did…
Even with the investigator bias and self-selection and “home court advantage,” Vitality employees got 2% fatter and their eating habits got worse after being coached.
Although maybe that counts as a success in the wellness industry, because employees weren’t too badly harmed, as they are in a typical Koop-Award winning program. Congratulations to Mr. Goetzel!
Part 2 picks up where Part 1 left off, as coincidence would have it.
Soeren Mattke (as mentioned in the last installment) and I were quite relentless in trying, quixotically, to get Professor Baicker to explain her results. Its popularity could have landed her many profitable speaking and consulting gigs, but she evinced no interest in cashing in, or even in defending her position. Indeed, the four times she spoke publicly on the topic, she didn’t do herself, or her legions of sycophants in the wellness industry, any favors. In each interview, she distanced herself more and more from her previous conclusion. Here are her four takeaways from her own study “proving” wellness has precisely a 3.27-to-1 ROI:
- It’s too early to tell (um, after 30 years of workplace wellness?)
- She has no interest in wellness anymore
- People aren’t reading her paper right (Shame on us readers! We’re only reading the headline, the data, the findings and the conclusion, apparently)
- “There are few studies with reliable data on the costs and the benefits” (um, then how were you able to reach a conclusion with two significant digits?)
Individually or in total, these comments sounded an awful lot like retractions, but she (and her co-author and instigator, David Cutler) claimed those comments didn’t constitute retractions. Whatever they were, she wasn’t exactly doubling down on this 3.27-to-1 conclusion.
The industry at this time — 2013 and 2014 — came under a lot of criticism for basically being somewhere between hilariously worthless and a fraud. (Here is some comprehensive documentation of the fraudulent and harmful activity in the wellness industry. Wellness is a classic example of the observation that every cause starts out as a movement, becomes a business, and degenerates into a racket, the poster company for the last being Interactive Health.)
Here’s where it gets interesting. Due to this relentless criticism, instigated and sometimes written by me (most of which is catalogued here) with credit to Soeren Mattke and Jon Robison as well (with Pulitzer Prize-winning LA Times columnist Michael Hiltzik piling on multiple times), Ron Goetzel and his colleagues urged Professor Baicker — who had previously claimed to have no more interest in wellness — to get a grant from Ron’s friends at the Robert Wood Johnson Foundation and do an actual controlled study of workplace wellness to set the record straight. As a strategic gambit, this showed a level of professional judgment worthy of Gary Hart.*
Obviously if you are perpetrating a fraud and you know it, you don’t urge people to investigate it. Mr. Goetzel at this point (or at least shortly thereafter) knew that his industry was a fraud, going so far as to erase some accurate data and disown inconveniently accurate conclusions he had forgotten to suppress.
He even forged a letter from the governor of Nebraska to cover his tracks. (To be completely accurate, he did not forge the signature on the letter. He merely doctored the wording in a letter that was already signed to say something much different than the letter said when it was actually signed. Maybe that’s not technically forgery. Who knows?)
And yet that’s exactly what Mr. Goetzel did. Maybe he thought she was as corrupt as he is, but she has an excellent reputation in this industry. My suspicion is that she accepted his offer for exactly the opposite reason that he made it — to clear her name after her ill-considered 2010 meta-analysis.
Way back in 2016, I predicted how this gambit would turn out — that Katherine Baicker was not going to mail it in this time but rather do a high-quality study of the type she is known for.
Her now-famous outcome was preordained for four reasons:
- In this industry, as Michael O’Donnell pointed out in the trade journal where he formally served as Prevaricator-in-Chief, the higher quality the study, the lower the ROI, to the point where RCTs invariably show negative ROIs. (Owing to the embarrassing accuracy of this unintended finding, Mr. O’Donnell walked it back in a subsequent editorial.)
- Kate Baicker was not going to sully her legacy by being known as the Typhoid Mary of wacky wellness programs, just to please her sycophants.
- She picked the most clueless vendor imaginable to carry their water. This selection couldn’t have been an accident.
- No wellness study since Johnson & Johnson (which was probably also wrong–just not obviously from the data presented) had shown even a remotely positive outcome. This list includes PepsiCo, Barnes Hospital, Nebraska, Vitality, Newtopia, Health Fitness Corp, Health Fitness Corp again, Boise School District/Wellsteps, McKesson, Connecticut, and the University of Illinois. All were epic fails, according to their own data. Not to mention that the wellness trade association itself published a case study showing wellness loses money and then ran away from it.
Where does the wellness industry go from here? They’re hoping for Tiger Woods, wiser souls are channeling Gary Hart, and probably what we’ll get instead is Harold Stassen. (Look it up.)
*For those of you who aren’t old enough to remember, here’s what happened. Dogged by rumors of philandering, the front-running Democratic presidential candidate urged reporters to follow him around. It took about a day for reporters to discover he was indeed philandering. It only took another few weeks for him to withdraw from the presidential race.
There is a lot more to this study than meets the eye.
Some tourist attractions feature an “A” tour for newbies and then a “behind-the-scenes” tour for those of us who truly need lives. For instance, I confess to having taken Disney’s Magic Kingdom underground tour, exploring, among other things, the tunnels through which employees travel so as not to be seen out of costume in the wrong “Land.”
Likewise, there have been many reviews of the recent wellness study conducted by the National Bureau of Economic Research (NBER), the first-ever randomized control study of a wellness program. This, however, is the first review to go beyond the “A” tour of the headlines.
By way of background, the headline is that the mainstream wellness program the investigators examined at the University of Illinois did not noticeably move the needle on employee health. They didn’t address return-on-investment (ROI), because there obviously was none. Achieving a positive ROI would require moving the health risk needle—not just by a little, but rather by enough to significantly improve the health of many employees. Then, since wellness-related events, such as heart attacks, would not otherwise have befallen these employees immediately, this improvement would have to be sustained over several years before there was a statistical chance of some events being avoided.
Finally, the magnitude of this improvement would have to be great enough to violate the rules of arithmetic, because it is not mathematically possible to avoid enough medical events to break even on wellness. For instance, it actually costs about $1 million to avoid a heart attack through a screening program.
This finding, therefore, represents an existential threat to conventional wellness programs.
It all boils down to: why would an associate professor (Damon Jones) publicly humiliate his own dean (Katherine Baicker — yes, the very same Katherine Baicker who always seems to be on the wrong side of every wellness debate) …unless he is absolutely sure he is right?
She can’t fire him now because that would get picked up by the lay media. Perhaps she should have paid him $130,000 not to disclose the results.
OK, this time I’m not the one causing the kerfluffle in the wellness industry, though I will confess to being a force multiplier.
Not since 2014, when the very unstable morons at the Incidental Economist made fun of the very stable geniuses who give out the Koop Award and also unequivocally concluded wellness loses money — combined with continued fallout from the Penn State debacle and the Nebraska scandal — has the wellness industry had such a bad year. And it’s only February.
Let’s review what’s happened so far in 2018. First, a federal judge ruled that voluntary wellness programs need to be — get ready — voluntary. The EEOC’s responded with the legalese equivalent of: “Fine, be that way.”
Next, WillisTowersWatson did something that might get them in hot water with the very stable wellness industry leaders: they were honest. They published a study revealing that employees hate wellness even more — way more — than they hate waiting for the cable guy to show up.
Finally, the very unstable National Bureau of Economic Research conducted a controlled study finding basically no impact whatsoever of a wellness program. More importantly, they specifically invalidated the “pre-post” methodology. Even more importantly, they specifically invalidated 78% of the studies used in Kate Baicker’s “Harvard Study” meta-analysis.
Here is an interesting piece of trivia. The lead researcher is an assistant professor at the Harris School of Public Policy. Why is this interesting trivia? Because Katherine Baicker — the Typhoid Mary of Wellness, whose THC-infused 3.27-to-1 ROI is the basis for essentially every subsequent genius wellness outcomes claim — is now the dean of that very same Harris School. I’m just guessing here, but I’d say it’s gotta be a trifle embarrassing when your own subordinate publicly disproves your own study. I mean, it’s one thing for me, RAND, Bloomberg, and anyone else with five minutes, internet access and a calculator to do it, but…your very subordinate?
On the other hand, the researcher, Damon Jones, just demonstrated not just amazing competence, but amazing integrity as well. In other words, he has no future in wellness.
The Wellness Empire Strikes Back
How does the wellness industry respond to these smoking guns threatening their entire revenue stream? Apparently, there is little cause for concern on their planet.
Let’s start with America’s Health Insurance Plans (AHIP), the health insurance industry lobbying group. Here is AHIP’s oxymoronic Wellness Smartbrief (January 26), on the NBER research. Yes, it summarizes the same wellness-emasculating study as the one above, though you could never guess it from the headline:
Continuing, AHIP said:
Offering incentives for completing wellness activities might be more cost-effective than offering incentives for wellness screening, a recent study of a comprehensive program found.
Perhaps AHIP has been infiltrated by Russian trolls, because here’s what the NBER article actually said about “completing wellness activities”:
We…do not find any effect of treatment on the number of visits to campus gym facilities or on the probability of participating in a popular annual community running event, two health behaviors that are relatively simple for a motivated employee to change over the course of one year.
Wellness programs might attract mostly employees who are already fitness-conscious, but the potential to attract healthy employees whose medical spending is already low could nonetheless be a boon to employers, the researchers found.
And on the subject of “the potential to attract healthy employees” being a “boon to employers,” the authors actually said:
We further find that selection into wellness programs is associated with both lower average spending and healthier behaviors prior to the beginning of the study. Thus, one motivation for a firm to adopt a wellness program is its potential to screen for workers with low medical spending. Considering only health care costs, reducing the share of non-participating (high-spending) employees by just 4.5 percentage points would suffice to cover the costs of our wellness program intervention.
In other words, you can apply some workplace eugenics to your company by using wellness to weed out obese employees, employees with chronic or congenital diseases, and so on. Good for you!
Soon, if AHIP and others have their way, there will be no need for guesswork in eugenics: employer wellness programs will be able to screen these employees out based on their actual DNA.
AHIP’s take on AARP v. EEOC
And now, AHIP’s take on this landmark case, their ace reporters scooping everyone with this February 2 headline on the December 20th court ruling:
Here are more typical headlines on that court ruling, headlines that came out the same month that the court ruling came out. Perhaps AHIP used the interim six weeks to focus-group various verbs until they settled on…tweak???
AHIP: It’s not just the headlines
One prominent healthcare executive recently attended an AHIP conference and reports:
I just returned from one of the dumbest meetings I’ve ever attended in Washington. Report of a new “study” by AHIP. Turns out people don’t mind health costs all that much, they just want more benefits. And everything is hunky-dory with their health plans, people like them so much. They love wellness benefits and crave more. Prescription drug prices have been nicely controlled thanks to the competitive marketplace (no, I am not making this up or exaggerating for drama). For every $1 employers spend on benefits workers get $4 in value. Priorities for SHRM rep: Fitbits for all employees, solving the outrage that only 20% of her employees got an annual physical. 85 cents of every dollar spent on health care goes to chronic disease.
Over these same two hours, I’d estimate about a thousand employees were misinformed, harmed or harassed by wellness vendors, roughly equal numbers of employees got useless annual checkups, employers spent about $200-million on healthcare and 40 people died in hospitals from preventable errors. But I’m being such a Debbie Downer! I’m going home to read Why Nobody Believes the Numbers to remove myself from this alternative universe.
Enter the Health Enhancement Research Organization (HERO)
HERO’s Prevaricator-in-Chief, Paul Terry, is demonstrating his usual leadership abilities in this crisis, of course. After all, HERO is the wellness industry trade association and these three items — the NBER invalidating their product, employees hating their product, and a federal judge forbidding them to force employees to use their product — represent existential threats to his “pry, poke and prod” members.
Teddy Roosevelt said, “complaining about a problem without posing a solution is called whining.” It’s a quote that also reminds me why I’ve not thought of angry bloggers who target health promotion [vendors] as bullies. Though they relish trolling for bad apples, their scolding is toothless, more the stuff of chronic whiners.
I suspect he is talking about me here as the “chronic whiner” who is “scolding” them. Or perhaps he is referring to the “angry bloggers” at the Los Angeles Times, the New York Times, Slate, or STATNews, since those “toothless” publications seem to be scolding wellness vendors more than I ever have. For instance, I’ve never called wellness vendors’ offering a “scam” or a “sham.” I simply quote these very stable wellness geniuses verbatim, as above or below, or last week.
Being quoted verbatim, not angry bloggers, is their worst nightmare. (One thing I would concede, though, is that “Paul Terry and the Angry Bloggers” would be a great name for a rock band.)
Yep, looks like the implosion of his industry all my fault. Otherwise, I’m not quite sure who is the “angry blogger” he is referring to, other than to note that Mr. Terry himself seems to blog a tad angrily himself, both above, and here…
Why I choose to ignore the blogger critics: We’re fortunate to work in a profession with a scant number of vociferous critics. My take is that there is one thing these few angry loners [Editor’s note: the complete “scant list” of the 220 “few angry loners” who have been “vociferous critics” can be found here] want more desperately than attention: that’s to be taken seriously. What they fail to comprehend is that as they’ve gotten ever more farfetched and vitriolic in search of the former, they’ve cinched their inability to attain the latter.
Baiting people with misinformation and offensive insults (but just a tad under highly offensive) is a pesky ploy that trolls hope will eventually land a bite that confers credibility where there is none. Even reading such drivel is a form of taking the bait; responding is swallowing it whole. Some say dishonesty should not go unchallenged and I respect their view; nevertheless, I’m convinced responding to bloggers who show disdain for our field is an utter waste of time. I’ve rarely been persuaded to respond to bloggers, and each time I did it affirmed my worry that, more than a waste, it’s counter-productive.
…and especially here, a seemingly incongruous decision to “act out” by someone who claims to be “choosing to ignore the blogger critics.”
Having read years of my “drivel” alongside Mr. Terry’s posting explaining why you shouldn’t “swallow this bait,” perhaps readers might opine here: which of us, exactly, is the “chronic whiner”?
Coincidentally, when I run live health-and-wellness trivia contests, the first of our 3 rules is: No Whining. Seems to me that he would have just violated it. Indeed the only rule HERO hasn’t violated so far is #3 below. Not that I want to put ideas in their head.
Not to be confused with those immortal words often attributed to the great philosopher Yogi Berra, a big joke among economists is: “An economist is someone who upon learning how something works in practice, wonders how it will work in theory.”
That joke morphed into reality last month — though it was a controlled study testing a theory, rather than the theory itself. The “theory” would be that inactive unmotivated non-participants can be used as a control for active motivated participants. Ironically, this study design has never been proposed as legitimate, even in theory. Wellness “researchers” like it because it always show savings, even when nothing happens. For example, even when there was no program for the participants to participate in.
Obviously if this were a legitimate design, the FDA would approve it for clinical trials, saving a ton of time and money vs. having to do controlled trials.
To wit, the National Bureau of Economic Research (NBER) just published a study showing that the participants-vs-non-participants (“par-vs.-nonpar”) study design, used extensively by the very stable geniuses in the wellness industry to do their alt-research to fabricate their alt-findings, is completely invalid.
Highlights of NBER study for the wellness industry: bad news first
First the bad news. Fasten your seat belts and be shocked, shocked to learn that the researchers could identify:
- No noticeable change in health behaviors due to the wellness program
- No noticeable change in health outcomes due to the wellness program
- Clear self-selection bias among participants opting into the wellness program
Lest anyone think we are taking this out of context, here are their exact words:
We do not find any significant effects of treatment on total medical expenditures, employee productivity, health behaviors, or self-reported health measures in the first year following random assignment. We further investigate the effect of our intervention on medical expenditures in greater detail, but fail to find significant effects on different quantiles of the spending distribution or on any major subcategory of medical expenditures (pharmaceutical, office, or hospital). We also do not find any effect of treatment on the number of visits to campus gym facilities or on the probability of participating in a popular annual community running event, two health behaviors that are relatively simple for a motivated employee to change over the course of one year.
This of course merely confirms what we observe in outcomes reports published by wellness vendors, including the two most recent proud recipients of the Deplorables Awards. Actually, in the case of Wellsteps there was indeed a noticeable change in health outcomes among program participants — they got worse.
Also, the authors — no doubt anticipating the objection from the very stable geniuses at the Health Enhancement Research Organization — specifically note that nothing in Year One’s results presage any step-function improvement in Year Two. So the specious “Wait ’til next year” argument is off the table.
You might be thinking, “Another nail in the wellness industry coffin.” True, except that there almost isn’t room for any more nails. Soon the coffin will have enough nails to create its own gravitational field.
Next, the good news
Here are the two pieces of good news for the wellness industry. One finding was:
Our 95% confidence intervals rule out 78 percent of previous estimates on medical spending and absenteeism.
That means that it is possible that 22% of previous estimates may conceivably not be completely invalid. This is not to say that 22% are valid, just that they aren’t automatically invalid. That is great news for the wellness industry, where clearing the bar for not being automatically invalid is cause for celebration. As Dave Chase says, the bar for wellness is so low a snake could jump over it.
However, the 78%-totally-invalid figure specifically invalidates Katherine Baicker, author of the so-called “Harvard Study.” Depending on whether you are a wellness vendor or an oppressed employee, she is either the Johnny Appleseed or the Typhoid Mary of wellness. Her famous 3.27-to-1 ROI was tallied entirely from par-vs.-nonpar studies, exactly the methodology that the NBER just invalidated, citing exactly the studies she cited. (The three examples in my study referenced above were also part of her meta-analysis.)
So perhaps she might now make a formal statement regarding par-vs-non-par as a study design? Either a retraction or a defense. Just something that clarifies her previous statements, which seem to be neither retractions nor defenses but rather more like excuses:
- It’s too early to tell (um, after 30 years of workplace wellness?)
- She has no interest in wellness any more
- People aren’t reading her paper right (we’re only reading the headline, the data, the findings and the conclusion, apparently)
- “There are few studies with reliable data on the costs and the benefits” (um, then how were you able to reach a conclusion with two significant digits?)
The irony is that Kate Baicker has otherwise done outstanding research. Her study on Oregon Medicaid is a classic. In Oregon at the time, Medicaid eligibility was determined by lottery amongst applicants. That meant that — quite the opposite of wellness control groups — the control group of people not picked in the lottery had equal motivation to seek insurance coverage as people who were picked. After following both groups going forward, her finding was that obtaining insurance to access basic medical care did not change outcomes. (Having insurance did bring peace of mind, though.)
And yet somehow in “Workplace Wellness Can Generate Savings,” she was quite comfortable reaching a conclusion that was completely inconsistent with her Oregon finding, not to mention the Law of Diminishing Returns: throwing additional unrequested, generally unwanted, and largely misdirected medical interventions and advice at employees who already have insurance — and recall that most insured Americans are drowning in medical care — could dramatically improve their outcomes enough to calculate not just a massive ROI, but an ROI precise to two significant digits.
What she will hopefully learn through the NBER study is something that I learned 11 years ago: when the data proves you wrong, fess up. Then people like me have to find someone else to blog about. Fortunately, in wellness, that is not a heavy lift.
The second piece of good news for the wellness industry
To quote the study:
…wellness incentives may shift costs onto unhealthy or lower-income employees if these groups are less likely to participate in wellness programs. Furthermore, wellness programs may act as a screening device by encouraging employees who benefit most from these programs to join or remain at the firm…
To be clear, this calculation does not imply that adoption of workplace wellness programs is socially beneficial. But, it does provide a profit-maximizing rationale for firms to adopt wellness programs, even in the absence of any direct effects on health, productivity, or medical spending. [emphasis theirs]
In other words, employers can use wellness programs to subtly discriminate against unhealthy — read, older and poorer — workers. Many of the very stable geniuses in the wellness industry will be happy to hear this. The exceptionally stable genius who will be most thrilled to hear this is Michael O’Donnell. Mr. O’Donnell is the former Prevaricator-in-Chief of the wellness industry trade publication and a current member of the Koop Award cabal. These are excerpts from one of his editorials:
First, he says prospective new hires should be subjected to an intrusive physical exam [editor’s note: notwithstanding the fact that this is totally illegal], and hired only if they are in good shape. OK, not every single prospective new hire needs to be in good shape — only those applying for “blue collar jobs or jobs that require excessive walking, standing, or even sitting.” Hence he would waive the physical exam requirement for mattress-tester, prostitute, or Koop Award Committee member, because those jobs require only excessive lying.
Second, he would fine people for not meeting “outcomes standards.” In an accompanying document, he defines those “outcomes standards.” He specifies fining people who have high BMIs, blood pressure, glucose, or cholesterol.
In other words, Mr. O’Donnell wants to charge for insurance by the pound, as that accompanying document says. Actually, by BMI, which of course is of dubious value as a measure of weight, let alone health.
Here is his actuarial formula:
Although having read his very stable arithmetic elsewhere in this same document, I’d worry about the accreditation status of any actuarial school, or for that matter any school of any kind within the 50 states, that would accept him:
Thirty-one states have no laws that prohibit employers from using smoking status as the reason for not hiring… In the remaining 29 states…smoking status cannot be used as the reason for not hiring.
I’m not waiting around for a retraction from this genius either.
Here’s a similar apology, for Professor Baicker’s infamous “Workplace Wellness Can Generate Savings” meta-analysis claiming the 3.27-to-1 ROI from wellness that RAND also eviscerated:
I had been quite adamant in the previous post that this meta-analysis was likely just a gold-plated package of garbage case studies. I compared it to packaging subprime home loans into AAA-rated collateralized mortgage obligations (CMO).
That was before I looked at the individual studies comprising this meta-analysis. I realize now that comparing the Harvard Study to the CMO scam was unfair. So I owe an apology to Bear Stearns, Lehman Brothers, and Countrywide.
Next, I apologize for pointing out that Ron Goetzel, as recently as last week, is still quoting this very same thoroughly discredited 7-year-old study, as well as many other outdated analyses. For example, he insists on continuing to quote the New York Times economists’ September 2014 analysis that wellness programs “generally” don’t work, even though they subsequently made their conclusion much clearer: “We’ve said it before, many times and in many ways: workplace wellness programs don’t save money.” They then specifically criticized Mr. Goetzel’s own methodology (“industry studies based on study designs that cannot produce valid causal estimates”).
I apologize for thinking that this deliberately selective misinterpretation of these economists’ previous conclusions makes him sound deceitful. And I apologize for being sure that the people who forwarded me this slide would agree with that assessment. And I apologize for once again making phony apologies.
The real apology
Now that I’ve checked off the usual Wellsteps-and-Goetzel-integrity boxes, it’s time to step out of character and seriously apologize. Here’s what I did that I really do need to apologize for: not cutting wellness professionals enough slack on giving them enough time to learn what took me several years to learn about wellness losing money.
I was once guilty myself of believing this 3-to-1 ROI nonsense, specifically about disease management (DM). Why DM in particular? I had some ego wrapped up in it because I am actually credited with inventing DM. Really. Just google on “invented disease management.” Whether or not I did (and plenty of others could share the credit), I was not just drinking the DM Kool-Aid. I was mixing it up and selling it to others.
Then, 10-12 years ago, a few people told me none of the DM savings numbers added up. I didn’t believe them. I thought it was sour grapes because they missed the boat.
True, I had enrolled a few of my own extended family members and friends into DM programs. Vendors were more than happy to offer me their best nurses, VIP treatment, you name it. Yet no one I referred thought these free programs were even worth a second free phone call.
Nonetheless, I was sure that somewhere there existed a whole lot of employees who did benefit from DM. Yes, in my fantasy world tons of people really appreciated these unsolicited calls from their health plan offering to help. After all, who among us doesn’t trust an unsolicited caller from their health plan offering to help?
In my worldview, the lucky recipients of these calls would respond: “You’re right. I should be taking my pills. Hey, thanks. I never would have thought of that on my own. And I was just about to have a heart attack, so you saved a ton of money.”
Yes, I realize this made no sense. Yet I never questioned my own findings. Basically, I ignored the warning signs about DM’s sketchy economics for years. When I finally had the epiphany, it got quite the headline:
Once I questioned my own figures, the rest because immediately obvious to me — one after another after another, sets of numbers in this field simply did not add up. Wellness was a far worse offender than DM, which does appear to roughly break even or better. No surprise about wellness. Screening costs about 10 times as much as DM and whereas people who qualify for DM are already well down the slope to infirmity, screens are performed mostly on healthy employees, who can’t generate any savings. (We of course support screenings according to guidelines, for the health of employees rather than an ROI. But most vendors ignore guidelines and screen the stuffing out of employees.)
Still, it had taken me years to have the initial epiphany…and yet now I was quite curt and dismissive with other people who didn’t immediately get it, and were defensive in support of their lifelong assumption. I was basically saying to others in the field: “You know all those savings claims? Total malarkey. Prying, poking and prodding doesn’t save any money.” I’d expect everyone else to get this right away. When they didn’t, I was not gracious with them in many cases.
Over time, a large number of folks have come around. They did it at their own speed, same as I did. Take WELCOA, for example. They were among the worst (and God bless ’em, funniest) offenders…and yet now you won’t find an organization more committed to getting wellness right, helping employees, and being honest than WELCOA. (In their case, a night-and-day change of leadership helped.) I’m not just saying this–I’m walking the walk. Quizzify is joining WELCOA’s Premier Provider Network for 2017, joining vendors I have a lot of respect for, like It Starts with Me, Populytics, and SelfHelpWorks. The first two, also like Quizzify, are validated by the Validation Institute.
Making good on the apology
A good apology comes with an offer to make it up. So if you feel like I dissed you prematurely, while you were learning wellness economics on your own, you can have one of:
- a free pdf of any of my books — Surviving Workplace Wellness, Cracking Health Costs, or Why Nobody Believes the Numbers
- half-price on a hardcopy of those books (order direct, not through Amazon, of course)
- a free analysis of any outcomes report
- a free month of Quizzify with no commitment (there is a minor asterisk on this one, please inquire)
In the workplace wellness epidemic, Harvard School of Public Health Professor Katherine Baicker is Patient Zero. However, she may soon come up with the cure for the disease she spread. This would be like when the inventor of the PSA test said it shouldn’t be used.
By way of background, her original dalliance with wellness in Health Affairs–timed and designed to get the Business Roundtable’s “Safeway Amendment” allowing a 30% to 50% clawback of insurance premiums into the Affordable Care Act–introduced the infamous “3.27-to-1 ROI” into wellness vendor vernacular. That was almost seven years ago, but wellness vendors continue to cite this figure as gospel even though RAND demolished it and Professor Baicker herself walked it back four times.
For good reason, as it turns out — this Pulse posting says it all. The 10-to-30-year-old studies comprising her meta-analysis are even more amusing in retrospect. Lots of overcreening, overreliance on now-discredited BMIs, and of course recommendations for low-fat diets–even for employees with metabolic syndrome. Most published in third-tier journals, by the Wellness Ignorati and their friends.
And almost every one of them compared active motivated participants to non-participants. 6 years ago we suspected that design was invalid. Today we know for certain, thanks to Health Fitness Corporation and Aetna. Two more elegant studies to prove that point could not be imagined. No “investigator bias” or “publication bias” there, since both were hoping for the opposite finding.
However, mindful of her reputation, Professor Baicker recently announced (in as many words) that she is going to redeem herself.
Her protestation that she is no longer interested in wellness appears to have taken a backseat to her desire to remove this one blot–granted, a blot of Lady MacBeth proportions–on an otherwise excellent curriculum vitae.
She is going to do a study for BJ’s Wholesale Club, in conjunction with an outfit called Wellness Workdays. Wellness Workdays is a classic wellness vendor. That is to say, they won’t be winning a Nobel Prize anytime soon, or even a spelling bee. Let’s start by examining their analytic and clinical prowess.
To start with, their “White Paper” doesn’t just quote the infamous 3.27-to-1. They’ve upped the ante to 6.00 to 1, maintaining the two significant digits while almost doubling the savings. How? They’ve added the 3.27-to-1 for healthcare savings to the 2.73-to-1 for absenteeism reduction from that same 2010 study. Those two separate conclusions were reached from almost totally different studies. Anyone can tell that from reading the original. Anyone, that is, except Wellness Workdays.
Their analytic qualifications are matched only by their clinical qualifications. One member of their medical advisory board is Chief of Allergy and Clinical Immunology at the Indian River Medical Center in Vero Beach, Florida. While this expertise is not exactly central to the mission of the pry,poke, and prod industry, in all fairness it should be noted that the Indian River Medical Center runs one of the best allergy programs in all of Vero Beach.
Another is an OB-GYN in Colorado. Perhaps this advisor will develop a protocol for employees who want to be screened and induced at the same time. A third consults to orthopedists at “Lennox Hill Hospital,” a role that probably doesn’t require too much heavy lifting, because there is no hospital by that name.
This guy is also an expert on steroids and other performance-enhancing products, and has “published rseveral esearch studies.”
So they can’t spell, can’t proofread, can’t understand study design, and can’t cobble together a qualified advisory board. In other words, to paraphrase the immortal words of those great philosophers Gilbert & Sullivan, they are the very model of a modern clueless wellness vendor.
Katherine Baicker’s proposed study design
We are confident she is going to get it right this time and “discover” that wellness loses money. By selecting a vendor of the caliber of Wellness Workdays, she isn’t leaving anything to chance.
Having learned her lesson — and I’ve been pretty conscientious about forwarding helpful study design materials to guide her — she will certainly tally wellness-sensitive medical events across the entire population. More importantly, here’s what she is not going to do with this study:
- “Match” volunteer participants to — you guessed it — non-participants. This nonsense that keeps the wellness industry afloat, and Professor Baicker no doubt sees right through it;
- List the “unobservable differences” between participants and non-participants as a “limitation.” It’s not a limitation. It’s a baldfaced lie. It is now known that this face-invalid study design is truly invalid. She wouldn’t lie, right? That would damage the reputation of the entire Harvard School of Public Health in what will certainly be a high-profile study;
- Show a high ROI even though the change in health risk factors is trivial. This of course is the stock-in-trade of the Koop Award committee, but a real academic researcher would know better;
- Compare the costs to “trend” and say “costs were projected to rise by [this amount] but they only rose by [this lower amount], and therefore a huge pile of money was saved”;
- Attribute all cost differences to the program, whereas a real researcher would look only at utilization differences that could actually be attributed to urging employees to eat more broccoli. (By contrast, one of the studies in her previous meta-analysis credited the wellness program with a reduction in cat scratch fever.)
How do we know she isn’t going to make up phony outcomes again? She “tipped her hand” with the Oregon Medicaid lottery study. A terrific natural experiment, her study emphasized the value of a “lottery control,” meaning every subject had the same intent-to-treat. Exactly the opposite of the wellness participants-vs-non-participants study design.
Further, who wants to be known as the Typhoid Mary of workplace wellness? The Oregon study was an excellent one, so naturally it showed exactly the opposite of what wellness studies show. Specifically, facilitating access to care doesn’t reduce the cost of care or improve physical health status. And that was for people — newly minted Medicaid recipients — who didn’t have any insurance to begin with. Wellness, of course, takes employees who already have plenty of access to care and drown them in even more, unwanted and largely unneeded, screenings and checkups.
As for BJ’s Wholesale Club, I suspect they got suckered into this. Who volunteers to become the next Pepsico, a case study of how wellness programs fail? BJ’s obviously isn’t studying their own competitors: Target has one of the best programs in the country — precisely because they are far too smart to use a vendor like Wellness Workdays.
In any case, we look forward to her research study. Or perhaps, since this is in conjunction with Wellness Workdays, to rseveral esearch studies.