By Sasha TerMaat

At the end of November 2017, JAMIA published the article, “Are all certified EHRs created equal? Assessing the relationship between EHR vendor and hospital meaningful use performance.” The authors, A Jay Holmgren, Julia Adler-Milstein, and Jeffrey McCullough, performed a statistical analysis of publicly available data sets on Meaningful Use EHR Incentive Program performance, stratifying based on the developer of the EHR product used by the Meaningful Use participant.

It’s wonderful to see the data sets published by CMS and ONC used for insightful research. I know from personal experience doing data analysis of CMS and ONC published data sets that a lot of effort goes into data normalization, and the authors took a thoughtful and careful approach.

However, I was surprised by the authors’ conclusions and policy recommendations at the close of the article. Having found EHR developer-correlated variability in performance on certain activities measured in the Meaningful Use incentive program, the authors state that is undesirable, and write recommendations to standardize. The authors say, “Our results suggest that policy-makers should pursue modifications to the EHR certification process to decrease such variation across EHR vendors and improve EHR systems.”

I conclude the opposite. Differentiation in performance is an indicator of a vibrant marketplace of electronic health record (EHR) options where innovation is taking place. Therefore, the policy goal should be to encourage ongoing improvement and continued differentiation.

I am a wholehearted supporter of ongoing innovation in EHRs, continually making them better at supporting clinicians and patients. But having worked on the certification of EHRs to ONC criteria since the inception of that program in 2010, I know from personal experience that the certification process didn’t dramatically improve products on the market.

The authors are correct that certification promotes standardization. Certification by its nature identifies a way to test how something is expected to work. While this can be beneficial for some targeted use cases (such as checking conformance with interoperability standards), in many areas it has the unintended side effect of prohibiting creative approaches. It causes developers and implementers to think about the features encapsulated in certification narrowly, and discourages truly innovative and unique solutions.

In a certification demonstration, as you wait nervously for the proctor’s decision on if your software meets the requirements, it’s terrifying to hear, “Hmm, I’ve never seen that before.”

But when you are showing software users a new feature that you think can bring more joy to their practice, it’s exciting to hear, “Wow, I’ve never seen it that way before!”  Those are the moments that can lead to dramatic improvements in efficiency or satisfaction with the system.

ONC has repeatedly indicated that certification and other regulations are intended to be a floor, not a ceiling. Certification as a floor means it assures purchasers of a base set of functionality.  Certification not being a ceiling means it should not limit or impede adding other features or competitive differentiators.

The authors of the JAMIA article have shown through their data that the EHR systems they’ve analyzed have that floor, as users were able to achieve Meaningful Use requirements with a variety of certified systems. Their policy recommendation to change certification into a ceiling, with the purpose of minimizing variability in the marketplace, is not the direction our industry needs and would be detrimental to future innovation and usability of healthcare information technology.​

Sasha TerMaat is Chair of the EHRA Executive Committee and Director at Epic.