A few months ago I published an analysis about whether generalized mask wearing “worked”. That analysis showed, to my surprise, that if a state population consistently wore masks at a rate greater than the median, they would slow the speed of spread over time by about half versus states that had below median mask usage.
There were several caveats to this analysis. The mask usage was self reported on social media and very likely exaggerated. My hope was that on average all states exaggerated about the same. Anyway, it had to be better than earlier published studies, which were based only on whether states had or had not a mask order. The other problem was that my curve was synthetic based on comparing a theoretical state that was always above median verse always below. That may not reflect the real world because it is possible if a state did that the effects would extinct over time.
This analysis made very happy, people who were already mask fanatics. They claimed a certain amount of smug victory over us mask skeptics because I “finally admitted masks work”. Yeah that’s not how science works but I guess it is how cognitive bias works.
Enough time has passed, and I really do think we are over the real meat of this pandemic in the US, so it’s time to looks at ways to judge state success at fighting the pandemic. I wanted to create a way to rank states based on available data on how well they did against the pandemic. Some success may be due to different policies (and I hope this inspires fights and online flame wars) and some of it may just be dumb luck. Like the stars and stripes guy said in “Full Metal Jacket”, “well, that’s why god passed the law of probability”.
I also wanted to check if mask usage had an effect on mortality. I already showed that it had a likely effect on speed of spread, which is handy for not overwhelming medical facilities. But that doesn’t mean lower deaths. It could be you get the same deaths in slow motion. In fact, that is what the experts were telling us last spring when the focus was on preserving capacity and before we collectively lost our shit.
Spoiler alert, mask fanatics are about to be pissed. Sorry kids, argue the numbers and analysis and forget what you desperately want to be true.
I thought a lot about how to compare the success of states in COVID battling/luck. There is no reason to assume that absent both, COVID19 deaths would be evenly distributed. States have many differences. Some have higher population density. One super dense city that is a focus of international travel could doom your whole state. On the other hand, being a small island might be helpful. Different states have different environments. You expect colder states to have worse respiratory virus seasons than mild states. Different states have different distributions of ages. Older states would be expected to have greater mortality.
My best ideas were to normalize by age by dividing deaths by the over 65 population and then multiplying by 100,000 to give a standard COVID mortality per 100k seniors. Not perfect but pretty good.
Then to eliminate all the different X factors that might help a state fight a respiratory virus epidemic, I normalized again using influenza deaths from the previous six flu seasons. If you just exclaimed out loud “This isn’t just the flu!” you are too stupid to read this blog post and I know I would have no success explaining to you why. Just stop. I hear Kanye West is getting a divorce. Maybe go read about that and leave the critical thinking to the rest of us.
In order to do that I summed the flu deaths per 100k seniors for each state over the previous 6 seasons and then divided the number of COVID deaths they’s had so far per 100k seniors by that number. This is the death multiple. A very low multiple would mean the state is doing well while a very high one would mean they did worse than expected compared to other states. For example Hawaii has actually had few COVID deaths per senior than their previous 6 season of flu deaths. Nebraska has had about 5 times as many COVID deaths as six seasons of flue deaths. Here is the list in order from best to worst:
|State||6 years flu deaths per 100K seniors||COVID deaths per 100K seniors||multiple||cum mask use claim|
My state of California has been (so I’m told) one of the strictest over-reactors to the plague and we landed pretty solidly in the middle with respect to the multiple. Florida which was beating us on deaths adjusted for age, is doing much worse on this measure. Of course they have a very low mortality per senior for flu. I wonder if they are really good at vaccination which skews them a little lower? Texas was doing ok on age adjusted mortality but for a state that doesn’t have much flu death on average they are doing pretty poorly. But so is Massachusetts which has worse flu seasons (all those noreasters I suppose).
Is there any correlation between average flu season death and COVID death? There is, I did a standard linear regression and found the following relationship between flu deaths and covid deaths:
Slope standard error:1.6800903849590512
r value of 0.32 isn’t super solid but definitely a positive correlation between past flu season deaths and COVID deaths. This is what you would expect if many of the unchanging factors that affect states’ mortality are similar between the two diseases. This gives me some confidence this is a reasonable analysis, so you can all start fighting about which states got it right. To me it looks pretty random but maybe you can find a pattern.
Oh and how did masks (or claimed mask usage on surveys) correlate with COVID mortality? I’m so glad you asked. I summed all the mask use claims to arrive at a comparison number. Basically just added the percent number from every day for each state. So a state that had 100% claimed mask wearing for half the days would get the same number as one with 50% over all the days. A simplification but it seems fair to me. If you think otherwise please tell me why.
So did higher claims of mask wearing correlate with lower COVID19 deaths per senior? Nope. Here is a regression analysis comparing the multiple to cumulative claimed mask usage. I think it is fair to compare this multiple because in a regular flu season mask wearing hardly registers so it should be expected to show as a positive effect this year compared to past flu seasons.
Slope standard error:0.019114887544216856
r-value is close to 0 which is about as uncorrelated as things get. You can complain and rationalize all you want. Frankly I see too many claims over 90% to be believable but, as above I hope the degree of self delusion is similar in all states. Look at the plot. It’s random as can be:
So there you have it kids. We are almost done (or at least to the point where vaccination effects will swamp policy) and it looks like states successes were pretty random and masks don’t look like they had much effect on mortality.
As always I appreciate feedback on my methods and interpretation. For this particular post if anyone sees a potential pattern in states’ performances thus far, I would love to discuss.