A while back I mentioned that HIT usability was so problematic, the Government was starting to get involved (See post, "War on EMR usability").  Now, we have a much better ideas of what this mean with the recent release of the NIST document "Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records".

After reading this, I must start with a question.  What is the purpose of this document? Is the goal of the article to educate vendors on best practices? Will this report serve as a foundation for evaluating usability of EMR systems that will be published to consumers? Without this end point in mind, it’s hard to judge the merit of the report.

Waterfall design, really?

With that said, the methodologies outlined in the report contain little groundbreaking information, but is a rehash of what is considered best practices in the field of human-computer interaction. My major problem with the report is the emphasis placed on a “waterfall model of design” (see Figure #1).

First and foremost good design is a process. It is not something that can done once or injected at the very end of the development cycle. You can’t put “lipstick on the pig” and call your software usable. The best design companies have taken these principles to heart very early in the design process by constantly testing and modifying their product. Design is iterative. When you wait until the very end of the development process to run usability tests with your users you’re going to find problems – lots of problems. At that point, what is a vendor going to do? Miss a shipment deadline or send less “usable” code? I’m going to guess the former. Add to that, the usability testing will uncover major code rewrites often times requiring starting from scratch. Don’t make the mistake of assuming usability is going to uncover only aesthetic problems. This report needs to place a greater emphasis on iterative, rather than a once-through (waterfall) design process.

Developers != Designers

If you’re going to improve vendor design, it has to begin with an internal commitment to value designers and what they contribute to the product development process. They cannot be an afterthought. There are very few companies with this mindset. Almost all HIT companies are developer-driven, so the first thought, and one that is promoted in this report, is to turn developers into designers. This will not work! A developer and a designer require two fundamentally different skill sets that are not easily transposed. Developers are trained to think rationally and analytically; design requires empathy for the user.

Trained designers have a hard enough time in HIT because most of them are not clinicians. It’s much easier for a designer to create a new social network because we have used them. It’s much hardly for HIT designers to judge the value of the solution. That is why it is crucial to have a clinician apart of the design process. They can add the insights that the designer could never see. It’s hard enough for a trained designer to develop useful solutions in the HIT space, yet alone asking a developer to do the same.

I don’t think that vendors are going to change to this designer-centric development process voluntarily, so how is this report going to push them?

Some other thoughts

Why test applications in a sterile user testing environment? One of the biggest problems in HIT is that software is not designed with the clinical workflow in mind. How often have you seen a software tool that supports the interruptions that are common in a clinician’s day? By keeping the software in a controlled environment everything will look fine, but once it's in the wild the problems will show themselves.

The highlight of the report for me, were the HIT examples in the Appendix. They did a great job of capturing healthcare use cases.

When testing task competition times, why not assume expert users? If expert users are assumed then software can be used to model the time to competition based on spacing of buttons, number of clicks, system lag, keystrokes, etc. This will give an accurate, reproducible competition time estimates across vendors. Right now, untrained EMR users will be the test subjects, but as stated in the report EMRs are not “walk-up-and-use applications”. My guess is without training they will have difficultly even completing the tasks!