They Said What?

Home » Uncategorized » Busted! Another mistake found in my analysis

Busted! Another mistake found in my analysis

Do you know whether heartburn pills are safe for long-term use?

Guilty as charged. Someone called me out on yet another mistake buried in my 500,000 words published to date.

Yep, the number of members in the most exclusive club in healthcare outcomes analysis just rose by 33%, as Tom Milam of TrueLifeCare joins Corey Colman and Keith McNeil in justifiably calling me out.*

To put this track record in perspective, Ron Goetzel has been caught 14 times. You might say, well, 14 isn’t that much different from 3 in absolute terms.  (In percentage, it is, but we’ll let that slide.)

Except that I needed an entire decade to rack up 3, while Ron needed only 45 minutes to tally 14.  Over the decade, his number would be more like approximately eleventy zillion. It depends how you count the ones where he doctored numbers that were phony to begin with and then doctored them back again to the original phony numbers, after insisting that the doctored numbers were real. If you’ve lost track on all the doctorings that I just now published a companion blog post on it.


So what was the mistake? 

[SPOILER ALERT:  The rest of this post is boring.]

It’s kind of anticlimactic, and quite obscure. By way of background, I routinely analyze wellness-senstive medical event (WSME) rate trends for large employers and health plans. It’s not rocket science, but it’s totally valid. Indeed, it’s the only population-based observational analysis that is valid. (RCTs are not observational. But you knew that.). It was even embraced by Ron Goetzel’s very own outfit: the Health Enhancement Research Organization — before they realized that valid measures are the wellness industry’s kryptonite.

The WSME tally is also the only observational methodology accepted by the Validation Institute for employers and health plans.

Here’s what the national WSME rate looks like. (I think there was a reporting or transcribing error by one of the reporting states in 2005-2006, to the extent anyone noticed the inflection in the graph, or cares.) This graph of WSMEs shows that, over the decade+ period of the greatest growth of workplace wellness, that there was no improvement in event rates relative to the US population that would not have had access to workplace wellness — Medicare, Medicaid and the uninsured. Obviously their raw rates were higher. This is a difference-of-differences analysis.

Quite the contrary, it appears that if anything the employer-insured cohort trended worse than the control.

Tallying this rate requires our data extraction algorithm to collect ER and IP events primary-coded both to the disease in question, or else are common complications of the disease in question. We pick common complications based on two factors:

  1. How likely is someone with the disease in question to get the complication?
  2. How likely is it that the complication in question occurs in someone with the disease?

Remember, we only tally primary codes because we want to simplify the analysis enough that we can be 100% sure of comparability between any given payor and other payors comprising the benchmark. So we look for an “80-20 rule” in what we include in the primary code data extraction.

Our diabetes rate includes quite a number of complications that fit that description., one of which is cellulitis. Diabetics are much more likely than non-diabetics to get cellulitis in their extremities — feet in particular — because they often can’t feel a cut. (Also the skin on their feet can be thinner than it should be.) Likewise, cellulitis of extremities is much more likely to be diagnosed in diabetics than non-diabetics.

If you can’t feel it, you won’t treat it. And therefore your odds of cellulitis in your foot are high. However, cellulitis in non-extremities would correlate much more loosely with diabetes, since diabetics can still feel and see skin issues elsewhere on their bodies. Therefore, not all cellulitis codes, by a longshot, are included in our analysis.

While we included cellulitis of the foot (and leg, also common enough), we somehow — despite having done these analyses 20 to 30 times a year for 15 years — omitted cellulitis of the toes. Sort of like the Matisse painting hanging upside down in the Museum of Modern Art for 47 days, no one else noticed either. Yet even the most intellectually challenged members of the wellness industry’s self-anointed awards committee understand the anatomical fact that, technically speaking, the toe is part of the foot.


Vintage HENRI MATISSE Original Lithograph 1958 Color Art image 0

Le Bateau, Henri Matisse


Honestly, when all is said and done, this won’t change anyone’s results much, and all the changes will be in the same direction vs. history (which is also going to be recoded) and vs. the benchmark/average, likewise recoded. This is especially true in the working-age population, which comprises most of our analysis. Nonetheless, kudos to Tom Milam for becoming the third member of this most exclusive club.


*Your chances of joining this club are quite remote, statisically speaking. They are even more remote if you didn’t notice the arithmetic error just now. n increase in membership from 2 to 3 is a 50% increase, not a 33% increase. And the painting is still upside down…


In the immortal words of the great philosopher Pat Benatar, hit me with your best shot.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: