From Proposal to Policy: Does Policy Reflect the Quality or the Quantity of Evidence?

Last year, CMS proposed new Minimum Staffing Requirements for nursing homes, a topic I covered in the post titled Minimum-Staff Requirements Will Help Some Nursing Home Residents; Payment and Regulatory Reform Would Do More. 

There's no doubt that residents in nursing homes with more staff experience better outcomes—the literature on this topic is vast. The CMS-commissioned ‘Staffing Study’ includes a review of over 30 studies from 2019 to 2022 alone. However, most studies on staff and outcomes are descriptive, illustrating that more staff correlates with better outcomes, but stops short of asserting a causal relationship. 

But if the goal is better policy, the quality of the evidence should be the primary consideration. The sheer number of papers, whether 100 or 200, shouldn’t matter much, if what they predominantly show that healthier, financially better off residents are able to get care in nursing homes with more staff. 

Studies attempting to estimate the causal effects of increasing staff through minimum staff policies present mixed findings: one study (Lin 2014) shows a positive relationship for nurses, but no effect for nurse aides; another finds no effect from adding more social workers. Evidence suggests that while total staff hours increase, an unintended consequence is a reduction in registered nurses. Not surprisingly, several studies also conclude that the added costs negatively impacts the financial performance of nursing homes, potentially leading providers to exit and/or deter entry. 

An important question, therefore, is how policymakers respond to the quality versus the quantity of evidence. An interesting policy experiment is currently underway:

The proposed Minimum Staffing Rule was announced September 1, 2023 with a comment period of 60 days. In that time, the federal government received over 46,000 comments (!), which CMS now has to sift through and, theoretically, use to inform their next steps. The sheer amount of comments has already delayed the expected implementation date until 2025 (I call this as a point for team-quantity.)

Research shows that policymakers do make changes based on comments, so it will be interesting to see how the final rule differs from the proposed version. While comments on proposed rules is not where you'd expect high-quality research evidence to have the most influence on policymakers, the mere four mentions of "Lin 2014" among 46,000 comments is nevertheless  interesting.

Given the high-quality evidence, it's clear that strict enforcement of the rule by CMS would necessitate lowering the requirements, or risk triggering massive bankruptcies throughout the industry. The research also shows far more careful consideration is necessary regarding the impact on staff composition. Is CMS sure patients will benefit from more nurses and nurse aides, given the very predictable reductions that will occur to licensed practical nurses, a category not covered under the proposed rule and the obvious place nursing homes will cut costs to stay afloat? So what do you think, will CMS consider the insights from high-quality studies, or will the sheer volume of public feedback steer the course of policy?