Back to search

University of Washington

Performance evaluation of a natural language processing tool to extract infectious disease problems

Abstract

dc:description.abstract

Use of a complete problem list can benefit patient care, quality improvement initiatives, and research activities. However, it can be time consuming for physicians to enter the correct encoded problem from a standardized terminology. I evaluated Discern nCode, the natural language processing (NLP) system embedded in Cerner Powerchart at Harborview Medical Center (HMC), for its utility to add Infectious Diseases (ID) problems to the electronic medical record problem list, in comparison with the usual practice of physicians adding problems unaided by NLP. 74 ID consultation notes were annotated by human experts to create gold standard problem lists. NLP-extracted problems and problem list entries were recorded for each note. Recall, precision and f-measure were calculated for nCode and the problem list, and an error analysis was performed to describe false positives and missed concepts. Discern nCode's recall was .65 and precision was .14. Problem list recall was .10 and precision was .43. Many false negatives resulted from partial matches between NLP-extracted and reference standard problems. The majority of false positives were due to inclusion of past medical problems and non-ID problems; nearly 20% of false positives should not have been extracted. Discern nCode had significantly higher recall for ID problems than the problem list. Recommendations are provided for increasing system sensitivity and recall. Overall, nCode could be a useful facilitator of problem entry and result in higher problem list completeness, but recall should be increased.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Mandel, Hannah Lily
Advisor dc:contributor.advisor
  • Payne, Thomas H

Subjects

dc:subject × 4

Rights

dc:rights
Statement dc:rights
  • Copyright is held by the individual authors.
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1773/24103

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Mandel, Hannah Lily. Performance evaluation of a natural language processing tool to extract infectious disease problems. 2013. http://hdl.handle.net/1773/24103