Abstract
BACKGROUND: Oncology practice is increasingly aiming to be patient centric. Imaging is a decisive part of the management of cancer patients and with the introduction of Digital Health Records (DHR) patients have the possibility of accessing their imaging results independently, yet the optimal way of doing so is still not clear. The introduction of Large Language Models (LLM) offers the potential to turn radiology reports into a clearer, accessible and unambiguous format and to democratise patient’s access to their own medical records. METHODS: A multi-reader retrospective Service Evaluation (SE) conducted at a tertiary oncology hospital aimed to assess the capability of an LLM to generate two versions of simplified oncology imaging reports. The SE assessed Patient and Public Involvement (PPI) representatives and healthcare professionals’ (HCP) preferences using original radiology reports from two cohorts, colorectal (n = 30) and lung (n = 30) cancer. A Prompt-development phase created two prompts to generate the summarised (version A) and the full-length (version B) report versions. The review was performed by radiologists with 360 reads and PPI representatives with 180 reads. RESULTS: Radiologists scores between summaries and full-length reports differed per cohort. In the lung cohort, version A was rated higher for factual correctness (P = 0.001), completeness (P < 0.0001), accessibility and readability (P = 0.026), and benefit to patients (P < 0.0001). The opposite was seen in the colorectal cohort, version B achieved consistently higher scores (P < 0.002). When the two cohorts were combined, median scores for version A and B did not differ significantly (all P > 0.057). PPI reviews indicated that full-length reports were favoured significantly (P < 0.0001). Qualitative results from radiologists and PPI identified incorrect statements (n = 28), complex terminology (n = 18), addition of confusion (n = 10), and missing information (n = 10). CONCLUSIONS: LLM simplified reports have the potential to improve patient accessibility in oncology imaging. PPI and HCP preferences for summarised versus full-length reports vary. Findings suggest these outputs are likely to benefit from appropriate adjustments to individual patient needs and clinical context. Reports with incorrect, confusing and missing content, highlight that LLM need improvement, ahead of potential clinical use in this setting.