How Worst mHealth Apps Usability Restricts Patient Engagement

mHealth apps usability may not be all they are chalked up to be, particularly in the realm of patient engagement and chronic disease management.

In a data brief issued by the Commonwealth Fund, researchers describe how most mHealth apps usability have low design quality, making them significantly unusable for several patients.

The researchers, supervised by Urmimala Sarkar, MD, MPH, performed an observational analysis utilizing the 11 high-rated diabetes, depression, and caregiver apps. The group inquired twenty-six patient and caregiver participants to complete a set of activities proposed to demonstrate the app usability.

Overall, these apps indicated important interface design issues. The buttons were not huge enough for sufferers, and most applications lacked instructions for convenient navigation.

The applications also lacked contextual data elaborating to patients why certain data entry points were primary.

Limited patient education mostly keeps sufferers from completely understanding their chronic sicknesses, thus keeping them from full engagement. When the tested applications didn’t explain to diabetics why they might need to review a past meal, the applications kept those sufferers from entirely understanding the implications of their illness.

The applications also presented various data entry and retrieval complications. Most of the apps needed extensive data entry, inquiring users to input several pages of data points. On average, sufferers were only capable to get about halfway through the data entry prompts without expert help. Many sufferers also reported unclear explanations of what information the app needed.

Sufferers also reported problems with data retrieval, like data about upcoming doctor’s appointments or consultations. On average, sufferers completed 79 out of 185 tasks without help.

In the end, the researchers recognized 2 huge obstacles to app usability: a deficiency of patient confidence with technology, and poor app interface design usability.

The researchers cited 2 potential solutions to these issues.

“To harness the potential of mobile applications, developers might require engaging a diverse set of sufferers in the design and testing of their products,” Sarkar and coworkers wrote.

“Additionally, the applications should be capable to remind users of the rationale for each task and should integrate information from other sources, like pharmacies, to decrease the requirement for manual data entry,” they sustained.

Not only the mHealth apps usability are very low, but research indicates that many of them are also unsuccessful at empowering patient engagement. In accordance to researchers Karandeep Singh and David Bates, MD, most mHealth apps usability fail to fulfill 8 central sectors of mHealth patient engagement.

These factors included offering educational data, reminding or alerting consumers, recording and tracking health data, reflecting and summarizing health data, giving guidance deployed on user activity, enabling communications with contributors and family members, offering support through social networks, and motivating behavior change in rewards (like a points system).

Singh and Bates discovered that most applications fell into one of 2 categories: educational applications or engagement applications. Educational tools best suited sufferers with lower levels of engagement and who were merely initiating to learn about their conditions.

Engagement applications were better at reminding and prompting sufferers, and gave little data about chronic illnesses. These applications were better suited for the already fully engaged sufferer.

Overall, it is a long road ahead for mHealth app developers, but professionals say the road is worth it. When developers consult sufferers about usability requirements and take into account significant aspects of patient engagement, their programs can show promise.

“Chronic disease self-management and promotion of patient engagement are necessary to victorious care management programs targeting sufferers with high requirements and high costs and are linked with improved quality of life, functional autonomy, and decreased hospital use,” Singh and Bates described.

“Mobile health applications are designed for smartphones can assist to empower high-need high-cost sufferers to self-manage their health.”

 

ONC Posts Updates on Health IT Certification Transparency

Updates to certified health IT product list now involve the details about the costs and limitations of their implementation and utilization.

The Office of the National Coordinator for Health Information Technology is beginning to make good on its promise to increase transparency of information on certified EHR technology and health IT systems.

Before the beginning of the 2^nd day of its annual meeting, the federal agency declared the updates to its certified health IT product list (CHPL) to involve disclosures of information — costs and limitations related to implementing and using various certified technology.

“These new attempts to provide more and easier-to-understand information are critical to helping clinicians find the right tools to provide better care and make better the health of their patients,” said National Coordinator Karen DeSalvo, MD, MPH, MSc, said in a public statement Wednesday.

“This information and our new websites," she sustained, "will make the process of comparing and buying certified health IT simpler and better, discourage information blocking, and create clear incentives for developers to focus on the quality and usability of their products."

Why EHR Clinical Decision Support Devices Require More Research

Research indicates that effective, complex clinical decision support tools have lower EHR integration rates due to serious hurdles in research.

Real-time, patient-specific clinical decision support tools should be integrated into physician workflow, particularly given the near-ubiquitous adoption of EHRs. Despite this, physicians seen several hurdles to integrating CDS tools into their EHRs, according to a recent study published in the Journal of Medical Internet Research.

In accordance to a research team led by Thomas McGinn, MD, MPH, CDS that is most successful in making better the patient care are those that provide real-time prompts to providers and that are hyper-specific to an individual patient’s medical history.

Although, the most commonly utilized CDS are one-dimensional, the team claims, and only provides physicians with general prompts.

“Most of the CDS tools being introduced are uni-dimensional and not incorporated into the physicians’ workflow,” McGinn and colleagues say. “[These are] one-dimensional alerts that are generally triggered by one or two EHR components like an element of patient history. Examples include flu-shot reminders at annual visits or reminders for colon-cancer screening triggered by patients’ age.”

While it is significant to remind sufferers to get their annual flu shot, these kinds of reminders have little effect on patient care because they rarely relate to the individual patient during the appointment and because the prompts are not presented to providers in an organic way.

EHR Analysis Improves Complication Tracking

August 23, 2011 — Analysis of electronic medical records (EMRs) with natural language processing shows an improved ability to identify postoperative surgical complications compared with the standard method of relying on administrative data codes, according to a new study published in the August 24/31 issue of JAMA.

In efforts to improve patient safety, hospital administrative data are typically screened for codes that may reflect potential adverse events during hospitalization, and a quality surveillance tool developed by the Agency for Healthcare Research and Quality has refined that process to focus on a set of 20 patient safety indicators used in screening the data.

However, the system has some drawbacks, including some uncertainty about the validity of administrative codes and the inability of discharge codes to distinguish whether a disease existed before a patient's admission or was acquired during hospitalization, according to Harvey J. Murff, MD, MPH, lead author of the study from Tennessee Valley Healthcare System, Veterans Affairs Medical Center, and Vanderbilt University, Nashville, TN, and colleagues.

The emergence of EMRs, combined with the development of automated systems such as natural language processing, however, allows for screening of more extensive medical data and documents and extraction of specific medical concepts, as opposed to simply searching for potentially unreliable discharge codes.

In an effort to compare the 2 approaches, the researchers evaluated data on 2974 patients undergoing inpatient surgical procedures at 6 Veterans Health Administration medical centers from 1999 to 2006.

During this period, percentages of patients with postoperative acute renal failure requiring dialysis was 2% (39/1924 patients); pulmonary embolism, 0.7% (18/2327 patients); deep vein thrombosis, 1% (29/2327 patients); sepsis, 7% (61/866 patients); pneumonia, 16% (222/1405 patients); and myocardial infarction, 2% (35/1822 patients).

Natural language processing was able to correctly identify 82% (95% confidence interval [CI], 67% - 91%) of acute renal failure cases, whereas screening using patient safety indicators only correctly identified 38% (95% CI, 25% - 54%).

The results were also more accurate for natural language processing compared with patient safety indicators for venous thromboembolism (59% [95% CI, 44% - 72%] vs 46% [95% CI, 32% - 60%]), pneumonia (64% [95% CI, 58% - 70%] vs 5% [95% CI, 3% - 9%]), and sepsis (89% [95% CI, 78% - 94%] vs 34% [95% CI, 24% - 47%]). Comparison of the 2 methods for postoperative myocardial infarction, however, showed similar results (91% [95% CI, 78% - 97%] vs 89% [95% CI, 74% - 96%]).

Both approaches were highly specific for the diagnoses.

"In general, using a natural language processing–based approach had higher sensitivities and lower specificities than did the patient safety indicator," the authors write.

"The increase in sensitivity of the natural language processing–based approach compared with the patient safety indicator was more than 2-fold for acute renal failure and sepsis and over 12-fold for pneumonia. Specificities were 4% to 7% higher with the patient safety indicator method than the natural language processing approach."

The authors noted that a greater ability to refine and vary search strategies allowed for greater sensitivities with natural language processing, with only a small reduction in specificities in all areas except postoperative myocardial infarction.

"In contrast to the patient safety indicator approach, for which test characteristics are fixed, the natural language processing approach offered a wide array of search strategies with varying test characteristics," they add.

"Nevertheless in some cases, specifically postoperative myocardial infarction, the patient safety indicator algorithm had excellent test characteristics that were not improved through the natural language processing approach."

The study's limitations include that patient safety indicators were not originally designed for Veterans Health Administration data; however, the researchers noted that the patient safety indicator rates appeared similar between the Veterans Health Administration and non–Veterans Health Administration populations. In addition, institutions that have not adopted EMRs yet would obviously not benefit from the approach.

The results suggest, however, that natural language processing systems should be considered as EMRs become more widely implemented and evolve.

"As additional institutions develop fully integrated EMR, electronic chart reviews for quality purposes should be further developed and evaluated," the authors state in their conclusion.

In an accompanying editorial, Ashish K Jha, MD, MPH, from the Department of Health Policy and Management, Harvard School of Public Health, Division of General Medicine, Brigham and Women's Hospital, and the VA Boston Healthcare System, Boston, Massachusetts, states that "[d]espite the promise of [EHRs], recent data on their benefits have been disappointing." The data to date, Dr. Jha explains, have shown that EHRs can be a useful tool to help clinicians adhere to guideline-based care and to reduce medication errors. "[B]eyond these narrow benefits, there is little evidence that EHRs improve patient outcomes and even less evidence that they improve the efficiency of care," he states.

However, "in this sea of disappointing data about EHRs comes some good news."

Dr. Murff and colleagues "push beyond the traditional uses of the EHR by demonstrating that natural language processing, when applied to electronic data, can help clinicians track adverse events after surgery," writes Dr .Jha. Although this benefit may seem "esoteric," its value should not be "underestimated," according to the editorialist. "Their value as quality measurement tools will improve substantially when EHRs can automatically generate quality measures that account for the reasons guideline-driven care is adhered to or, if not, why not.

"Currently, the EHR remains a tool with vast potential but a limited set of current capabilities. Natural language process has the potential for many new applications such as automated quality assessment to assisting in the performance of comparative effectiveness research," concludes Dr. Jha. "The study by Murff et al suggests that these benefits may be closer than ever, but only if the power of computing is harnessed to understand the vast amount of written data that currently needs a pair of eyes and a human brain to comprehend." He emphasizes, however, that federal funding is needed to propel research in this area forward.

The study was supported by a grant from the Department of Veterans Affairs. One author is supported by the Veterans Health Administration Career Development Award, and 2 authors by Veterans Health Consortium for Health Informatics Research awards. Dr. Jha reports that he serves on the scientific advisory board of Humedica Inc, which pools clinical data to provide clinical intelligence to physicians and hospitals.

JAMA. 2011;306:848-855, 880-881.

Abstract

E-Prescriptions Just as Error-Prone as Paper Scripts

July 1, 2011 — Government and the healthcare industry have placed big bets on digital technology, and electronic prescribing in particular, for the sake of patient safety, but a new study reports that the error rate with computer-generated prescriptions in physician offices roughly matches that for paper scripts: about 1 in 10.

However, results from the study, published online June 29 in the Journal of the American Medical Informatics Association, are not as damning as they may initially appear. Error rates varied widely depending on the type of e-prescribing software used, with some programs outperforming pen and paper. In addition, software improvements could eliminate more than 80% of the mistakes, most of them involving omitted information.

In 2010, an estimated 190,000 physicians were electronically prescribing, the technical term for transmitting scripts directly to a pharmacy computer, according to a pharmacy industry group called Surescripts. That number does not include physicians who create a prescription with computer software and then either fax it to the pharmacy or give patients a printout.

Since 2009, the federal government has been paying hundreds of millions of dollars in Medicare bonuses to physicians and other clinicians who electronically prescribe. The government operates an even pricier incentive program for electronic health records, and e-prescribing is one of the prerequisites for earning a 6-figure bonus.

The new study study examined nearly 3900 computer-generated prescriptions received by a pharmacy chain in 2008 in Florida, Massachusetts, and Arizona, regardless of whether they were faxed or electronically transmitted to pharmacies or were printed out. Of those prescriptions, 11.7% contained at least 1 error. Researchers did not ascertain whether errors were corrected by the pharmacy chain or whether they led to an actual adverse drug event. Lead author Karen Nanji, MD, MPH, writes that the 11.7% figure is "consistent with the literature on manual handwritten prescription error rates."

Roughly one third of the errors represented potential adverse drug events, none of them life-threatening.

Software Improvements Must Be Physician-Friendly

Omitted information such as drug dose, duration, and frequency accounted for almost 61% of the errors detected by the authors. The rest of the errors stemmed from unclear, conflicting, or clinically incorrect information.

Software improvements, Dr. Nanji and coauthors write, could eliminate the vast majority of these mistakes. E-prescribing programs can incorporate so-called forcing functions that would prevent physicians from completing a prescription unless they enter required information, including complete drug names and proper abbreviations. Likewise, decision-support tools can issue alerts about a wrong drug dose or frequency. However, the authors note, physicians may rebel against e-prescribing software if antierror safeguards make it too slow or annoying to use.

Some e-prescribing programs included in the study appeared to give users a technological edge. The error rate associated with one such program was only 5.1% compared with a whopping 37.5% for another. However, the study did not assess whether the root cause was system design or how well or poorly the systems were implemented in physician offices. Training physicians and staff on new software systems, the authors note, is often given short shrift.

The study was supported by the federal Agency for Healthcare Research and Quality and the Harvard Risk Management Foundation. The authors have disclosed no relevant financial relationships.

J Am Med Inf Assn. Published online June 29, 2011. Abstract