2015 Edition EHR Certification Criterion
(3) Safety-enhanced design.

User-centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria: § 170.315(a)(1) through (4), (8) through (10), and (18) and (b)(2) and (3).

Public Comment Field:

First, we should comment that from our experience that the usability test effort is not a small undertaking for EHR vendors. In fact, it was essentially one of the most difficult criteria to complete, although the vendor was not “judged” on the usability results. Privately, we had several vendors indicate they were glad they had to do this testing as it revealed key findings of their system which they were not aware.

Therefore, we support expanding the scope of the “Safety-Enhanced Design” to include additional certification criteria, but we recommend focusing only those highly used/highly critical criteria, like problem list or those with a more open design aspect, like electronic notes. If the scope expanded to most or nearly all of the criteria, it would be an unnecessary burden.We believe formative testing can be very valuable, but we do not believe it needs to be used as a substitute for the summative testing required in this criterion. The reality is vendors who have an established user base are not re-designing their system’s layout/design very often, and the usability changes are more gradual. Thus, there will not be sufficient opportunities to do formative usability testing in a typical certification cycle.

We believe the current process of not requiring explicit usability tests has been acceptable, but we do think there is value in at least offering suggested usability tests to be performed, although stopping short of absolutely requiring them. Many vendors, especially smaller ones, are seeking guidance as how to best conduct viable usability testing.

Internally as on ACB and ATL, we decided to require a minimum of 5 clinical-type users (e.g. doing CPOE) and 2 administrative-type users (e.g. configuring CDS activation) for our vendor submitted usability reports although we were open to vendors using less than those numbers if they could clearly show why that was warranted. We believe establishing an general guideline for minimum number of participants would be of benefit, but we would caution flexibility should still be considered and also that the number needs to be relatively low for small vendors who would struggle with obtaining a large number of users given their lack of resources.

Finally, we do ask for guidance from ONC as to how much of the usability testing conducted for 2014 Edition can be inherited into 2015 Edition or if the entire usability tests must be re-administered. Basically, what is the “shelf life” of a usability test report.

For expert usability evaluations of your companies EHR: Contact The Usability People