EHR Usability

EHR Usability


A few days ago, I wrote about Innovation, a term being overused in the EHR industry to the point where it lost all meaning. Here is another such term: Usability

Just like Innovation, Usability is the weapon du jour against the large and/or established EHR vendors. After all, it is common knowledge that these “legacy” products all look like old Windows applications and lack usability to the point of endangering patients’ lives. On the other hand, the new and innovative EHRs, anticipated to make their debut any day now, will have so much usability that users will intuitively know how to use them before even laying their eyes on the actual product. With this new generation of EHR technology, users will be up and running their medical practice in 5 minutes and everybody in the office will be able to complete their tasks in a fraction of the time it took with the clunky, legacy EMRs built in the 90s. And all this because the new EHRs have Usability, not functionality, a.k.a. bloat, not analytical business intelligence and definitely not massive integration, a.k.a. monolithic. No, this is the minimalist age of EHR haiku. Less is better, as long as it has Usability.

Usability, according to the Usability Professionals Association, is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [ISO 9241-11]”. Based on this definition, it stands to reason that any EHR prospective buyer should want a product with lots of Usability. Everybody wants to be effective, efficient and satisfied. So how does one go about finding such EHR?

Well, as always, CCHIT picked up the glove, and as always, CCHIT will be criticized for doing so. The 2011 Ambulatory EHR Certification includes Usability Ratings from 1 to 5 stars. The ratings are based on a Usability Testing Guide. Jurors are instructed to assess Usability of the product during and after the certification testing based on three criteria: Effectiveness, Efficiency and the subjective Satisfaction, as required by the ISO standard.  The tools for this assessment consist of 3 types of questionnaires:

  • After Scenario Questionnaire (ASQ) –jurors rate perceived efficiency (time and effort), learnability, and confidence after viewing scenarios

4 questions after each scenario –16 overall

  • Perceived Usability Questionnaire (PERUSE)–jurors rate screen-level design attributes based on reasonably observable characteristics

20 questions divided among each of the scenarios;

  • System Usability Survey (SUS) –jurors rate the assessment of usability, and satisfaction with the application

10 questions after all four scenarios have been demonstrated

The questions range from general subjective assessments in the ASQ, to very specific inquiries in PERUSE, like whether table headers are clearly indicative of the table columns content. Following the certification testing, results from all jurors are combined and weighted with more weight to specific answers and less to subjective overall impressions. The final result is the star rating, ranging from 1 to 5 Usability stars.

As of this writing, 19 Ambulatory EHRs have obtained CCHIT 2011 certification and all of them have been rated for Usability presumably according to the model described above. Of those, 12 achieved 5 stars, 6 have 4 stars and 1 has 3 stars. Amongst the 5 stars winners, one can find such “legacy” products as Epic, Allscripts and NextGen. The 4 and 3 stars awardees are rather obscure. So what can we learn from these results?

The futuristic EHR movement will probably dismiss these rankings as the usual CCHIT bias towards large vendors. Having gone through a full CCHIT certification process a couple of years ago, I can attest that the only large vendor bias I observed was in the functionality criteria, which seemed tailored to large products. Big problem. However, the testing and the jurors seemed very fair and competent. Looking at the CCHIT Usability Testing Guide, I cannot detect any bias towards any type of software. I would encourage folks to read the guide and form their own unbiased opinions. Are we then to assume that the 5 Stars EHRs have high Usability and therefore will provide satisfaction?

I don’t have a clear answer to this question. Obviously these EHRs have all their buttons and labels and text conforming to the Usability industry standards, and obviously a handful of jurors watching a vendor representative go through a bunch of preset tasks on a Webex screen felt comfortable that they understand and could use the system themselves without too much trouble. Many physicians feel the same way during vendor sales demos. However, efficiency and effectiveness can only be measured by repetitive use of the software in real life settings, for long periods of time and by a variety of users. Measuring satisfaction, the third pillar of Usability, is a different story altogether. There isn’t much satisfaction about anything in the physician community nowadays and when one is overwhelmed with patients, contemplating pay cuts every 30 days or so and bracing for unwelcome intrusion of regulators into one’s business, it’s hard to find joy in a piece of software, no matter how  well aligned the checkboxes are.

The bottom line for doctors looking for EHRs remains unchanged: caveat emptor. The footnote is that the bigger EHRs are as usable as the Usability standards dictate, just like they are as meaningful as the Meaningful Use standards dictate and when all is said and done it is still up to the individual physician user to pick the best EHR for his/her own Satisfaction.


Leave a Reply

41 Comments on "EHR Usability"

Jun 14, 2010

You are obviously correct, Richard. there is no turning back and we shouldn’t look for a way back either. We just need a good and responsible way to move forward.
The more folks get on the software, the faster it will improve because there is nothing more effective than customer input and customer complaints.
However, since other than inconveniencing early adopters, these software products can adversely affect patient care, we also need a well regulated way of moving forward.

Richard Castro
Jun 13, 2010

You made some very intereting points in your blog. What most people don’t realize is that EMR software has been around for more than a decade, but that most healthcare professionals have not used it as of yet. Now that EMR software is becoming mainstream, healthcare professionals are beginning to analize,and critize it every chance that they get. Like always no one likes a change, especially healthcare professionals, and they most realize that this change is inevitable. They should learn to use the software that their facility has to its fullest, and realize that better software (updates, etc) will be on the way. EMRs are basically still in their infancy, and they will “grow up” rapidly.
CCHIT on the other hand was placed in a position to rate and accredit these EMRs by the government. This position has only been for only about a year now, and again, it will eventually progress into its place. All new fields have found there ways in the past, as will EMRs.
As for the usability of EMRs, it to will take time for healthcare professionals to be able to learn to use a EMR in a matter of 5 mins. EMRs are not as simple as are internet browsers, or word processors. They are complex systems that deal with not only very confidential information, but also many other systems like x-ray images, lab systems and others that interface into the EMR. EMRs are not plug and play type of software. As for the healthcare professionals complaints, in the past they would have training on the paperwork used in the facilty for at least 4-8 hrs as well.
We need to continue to go forward into the future of EMRs, and not look back to the days of patient’s charts overfilled with handwritten notes, labs, and unorganized paperwork.


I’m working my way through the supplement from the bottom up, but one sentence caught my eye:
“Future PCMH recognition and certification processes should focus more on patient-centered attributes and the proven, valuable key features of primary care than on the features of disease management and information technology.”
There have been studies before that showed indication of what they called “high touch” having higher correlation with outcomes than high tech had.
I never really understood why the PCMH criteria were so heavily tilted to EHRs. I do understand the basic need for having a comprehensive record stored at the MH, as it was always intended, but I think they went overboard with specific requirements.
Anyway, maybe we can learn something going forward.

Jun 8, 2010

Yes, there is value in data collection and population management, but those are very expensive activities and cannot/should not be allowed to interfere with one-on-one patient care.
Did you see that the AAFP just released the independent evaluation of the medical home National Demonstration Project? In a word: disaster.


Your description of what passes as CDS is indeed useless and unfortunately all too prevalent.
If we assume, just for the sake of this discussion, that reimbursement is available for both just talking to the patient and data collection, would you see any value in population management? Would registries help in managing a large panel of folks with multiple chronic conditions?
What do you think about Dr. Wagner’s chronic care model?

Jun 7, 2010

The devices are impediments to safe cost effective care.
They are clinical disrupters with inexpicable idiosynchracies that impede comparison of multiple data points at the speed with which the clinicians’ mind operates. They are disruptive to the creativity that is required to evaluate and manage multi-organ disease in patients on multiple medications. They are good for sandbox care, eg a simple one problem surgical situation, but otherwise, rigidly require workflow adjustments that are counter intuitive to effective care. They facilitate errors.

Jun 7, 2010

“However, I am looking at all the requirements for “meaningful use” and most things in there seem to go to patient care, such as registries, CDS, sharing information with patients and other providers of care.”
When I say patient care, I mean me sitting in the exam room, face to face with the patient. I see nothing in the meaningful use standards that would improve that interaction. All I see is a tremendous administrative load that means I need to see more patients to cover the overhead!
To be more specific, my day typically begins with seeing five or six Type II diabetics, usually poorly controlled. I want to use my time with them talking about diet and exercise. The last thing I want to do is, 30 times every week, reading a prompt from the EMR about the latest data on diabetes, clicking to indicate that I’ve read the prompt, and then clicking to acknowledge that it is or is not relevant to the patient I am trying to talk to. What a waste of time and energy, and what a disservice to the patient.

bev M.D.
Jun 7, 2010

I don’t think any physician who gets his patients’ lab results and radiology reports off the hospital computer (which is most physicians these days) can claim EHR’s are mostly made for billing. But I worked mostly in the lab, so what is it you clinicians really dislike – is it CPOE? Having to document your notes in the computer? What? Just come out and say it.
Now physician office systems are another issue and one I know nothing about. But again, I would ask the doc commenters to be specific about what they don’t like. EHR’s have multiple different components. What is it you do like and what you don’t?


pcp, you are making a point frequently made by many docs (EHRs are designed for optimizing billings, not patient care).
However, I am looking at all the requirements for “meaningful use” and most things in there seem to go to patient care, such as registries, CDS, sharing information with patients and other providers of care. Even recording of orders and receipt of electronic results can be viewed as an attempt to improve patient care.
So what would you, or anyone else that cares to comment, consider a worthwhile feature in an EHR, a feature that will improve patient care?

Dr. Seker
Jun 7, 2010

Tsuris and pcp are spot on. These devices should be recalled until there is proof of efficacy and safety. The data at present prove no better outcomes and no cost savings.
Equipment is being sold illegally. Best not to purchase.

Jun 7, 2010

The problem with EHRs is that they’re not designed to improve patient care: they’re designed primarily to deal with coding, billing, documentation, insurance issues, data collection, and so on.
Good products sell themselves. Bad products are subsidized and mandated.

Jun 6, 2010

In response to your questions, Margalit,
“So, tsuris, let’s take this a bit further. How do you propose the FDA test these software tools?”
HIT caused cases of failed care and lost specimen, lost results, hidden results, death and other, as reported on MAUDE and in Florida are frequent and give the equipment a rightfully deserved “black eye”. The user is not in error, but is, instead, bewildered and ostracized by the unusable equipment.
There needs to be candid in situ evaluations of CPOE and EMR equipment spanning the training, implementation, complications, defects, flaws, cognitive disturbance, usability, and chronic efficacy with efficient after market surveillance (if any of these devices would meet the safety and efficacy requirement of the FDA). Since the systems controlled by these HIT devices alter the entirety of communication and care of the hospitalized patient, their impact on each and every component of the complex interdigitated system of care requires detailed evaluation.
Most importantly, the patients and doctors must provide consent to have their care and workflow altered and affected by these systems.
The gag clauses between vendors and hospitals must go away, and all complaints, adverse events, confusion, disrupted communications be vetted.
When compared to the systems of care these non approved devices are replacing, they may produce worse outcomes and more complications. No one knows but it is likely according to the front line users, doctors and nurses.
They may be better, also, but why spend $ billions on devices that are of unporven safety and efficacy? HIM$$ and C$HIT propaganda is insufficient to support unfettered use.


Dr. Rosen,
What I find most disturbing about the Florida incident is the following quote from Dr. Rosenberg:
“This year we have seen as many if not more medical records violations from electronic medical records as we saw from hand-written records violations.”
As to good EMRs, there are several small EMRs, some written by their users, some commercial, that consistently manage to satisfy customers. I just reviewed one on my blog and I was fairly surprised by the scarcity of complaints. Maybe because these small software packages have no pretenses of saving the world or reducing the US deficit.
smart patient, I cannot speak for the HIT industry and the many appearances of impropriety it displays.
In my opinion, the industry avoided and is trying to avoid even now any FDA involvement because the high road is more expensive than the low one. The most well intended argument against FDA oversight is about “stifling innovation”. I don’t buy that either. Innovation for the sake of innovation is silly, and harmful innovation is not a desirable thing to have.

smart patient
Jun 6, 2010

If this is true–“I also think that the FDA has a role in ensuring patient safety.”
Margalit, why have the HIT industry’s devices avoided any and all scrutiny?
Were the authorites duped by the C$HIT certification or were there greenbacks being exchanged. The government is supposed to protect the patients, but the government has been commandeered by the self interested “intellectuals” from HIMSS and C$HIT amd others with a revolving door between ONCHIT, HHS and elsewhere. Is not N. deParle tagged to Cerner?

Jay Rosen, MD
Jun 6, 2010

You said: “I don’t think all EMRs out there are bad, some are actually pretty good, but this indiscriminate forced shopping spree will inevitably create large numbers of dissatisfied customers.”
What proof do you have for this statement?
Was the EMR involved in the missed uterine cancer case in Florida a “pretty good” EMR? See HIStalk report on this.