Research Questions Accuracy of Consumer-Facing mHealth Resources

Researchers aren’t too keen on WebMD’s mHealth platform for diagnosing eye issues, reporting that the digital health symptom checker offered an accurate summary less than 40 percent of the time.

In a study recently posted in JAMA Ophthalmology, a team of researchers led by Carl Shen, MD, of Ontario’s McMaster University found that WebMD’s online symptom checker displayed a correct primary diagnosis in only 26 percent of the 42 cases submitted to the connected health resource. A triage emergency was correctly recommended in 39 percent of the cases in which an emergency would be diagnosed, and in 88 percent of the cases that would have been deemed non-emergent.

The study adds fuel to the debate that consumer-facing mHealth and telehealth services, though easy to use and popular, aren’t sophisticated enough to render an accurate diagnosis, often missing crucial information or sending the person to the doctor and hospital for unnecessary treatment.

“The most popular online symptom checker may arrive at the correct clinical diagnosis for ophthalmic conditions, but a substantial proportion of diagnoses may not be captured,” Shen and his colleagues wrote in the study and an accompanying brief. “Until the accuracy and information provided … are improved, there is a risk of unnecessary use of health care services and, conversely, missed opportunities for appropriate intervention when seeking care is delayed.”

The study, conducted in October 2017, involved 42 vignettes entered into the online symptom checker by a mixture of medically trained personnel and those without medical experience, divided into 24 non-emergent and 18 emergent cases.

According to the study, the primary diagnosis by the symptom checker was correct in 11 of 42 cases (26 percent), while the correct diagnosis was included in the top 3 diagnoses in 16 of 42(38 percent) cases; the correct diagnosis was not included at all in 18 of 42 (43 percent) cases. In addition, triage urgency based on the top diagnosis was appropriate in 7 of 18 (39 percent) emergent cases and 21 of 24 (88 percent0 non-emergent cases.

That low accuracy rate may be causing more harm to patients than good, leading to “inappropriate use of health resources and poor outcomes,” according to Rahul N. Khurana, MD, of the Northern California Retina Vitreous Associates Medical Group in Daly City.

“Empowering patients with more information leads to greater engagement and better health outcomes,” he wrote in an accompanying commentary. “However, the accuracy must be validated before trusting these online resources. Incorrect diagnosis can create unnecessary anxiety for the patient and inappropriate triage advice, which can compromise care.”

Originally published in mHealth Intelligence by Eric Wicklund on May 28, 2019