Large language models (LLMs) are being tested to see if they can capture aspects of chronic pain that standard medical scoring fails to detect, according to a study published October 6 in npj Digital Medicine. The research examined how well AI could interpret patient narratives to quantify sensory, emotional, and psychological dimensions of pain that are often lost in standard clinical assessments.
Limitations of Current Pain Scoring
Standard pain assessments typically ask patients to rate their pain on a scale of 0 to 10, but the report notes these scores often miss critical details. Ira Hofer, MD, an anesthesiologist and clinical informaticist at the Icahn School of Medicine at Mount Sinai, said current tools do not capture whether pain interferes with daily activities, coexisting issues like depression, or treatment progress. These scores are also subjective and vary from person to person.
Dr. Hofer, who was not involved in the study, stated that an LLM-based assessment could enable consistent analysis of a larger set of information. Such a system would reliably capture not only the severity of pain but its overall effects on a patient’s life, which is difficult to achieve with manual chart review.
AI Analysis of Patient Narratives
In this cross-sectional observational study, researchers used LLMs to analyze transcripts from semi-structured patient interviews. The model was prompted to act as a clinical evaluator, assigning scores from 0 to 10 across nine domains. These domains included physical pain, emotional pain, quality of life, catastrophizing, depression, anxiety, rumination, agency deficit, and narrative fragmentation.
The study included 131 participants over the age of 40, divided into three groups: 67 with chronic low back pain, 33 with major depressive disorder (MDD), and 31 pain-free healthy controls. Interviews were conducted via Zoom between December 2022 and June 2025 using open-ended questions to elicit patient-centered pain narratives. The researchers used OpenAI’s Whisper base model to transcribe the recordings and Meta’s Llama-3-405B-Instruct model to analyze the text through IBM Watson’s API on IBM servers.
Read Also: Tirzepatide Reduces Asthma Flare-Up Risk in Diabetes Patients
When researchers compared the LLM-derived measures against interview word counts, the AI scores clearly separated the three groups, while word counting showed significant overlap. A model combining agency deficit and rumination number achieved an F1 score of 0.851, indicating very high discrimination between groups.
Comparing Clinical Patterns
Patients with chronic low back pain reported the highest physical pain scores. Meanwhile, those with MDD had higher scores for impaired quality of life, emotional pain, depression, and agency deficit. Researchers also compared the LLM scores against patient-reported scores from numeric rating scales and visual analog scales. Overall, 85 of 117 correlations (73%) were statistically significant after correction, suggesting the AI metrics reflect established clinical measures.
Analysis of Reddit posts found similar clinical patterns: depression-related posts had higher psychological scores, while chronic pain posts had higher physical pain scores. However, scores were generally higher in online communities, suggesting that writing style, context, and unverified diagnoses can influence LLM-generated assessments.
Challenges and Risks
Rohan Jotwani, MD, an anesthesiologist and pain management specialist at Och Spine at New York-Presbyterian, said that if an LLM can reliably flag patterns suggesting catastrophizing or social isolation, it could help triage patients. He added a concern about patients unconsciously reshaping how they understand their pain to match what they sense the system expects.
Dr. Jotwani, who was not part of the study, also noted that if clinicians routinely defer to LLM narrative summaries, there is a risk of eroding the deep listening and interpretive skills that create the therapeutic alliance. He stated that physician empathy directly influences chronic pain outcomes, and that relationship cannot be outsourced without cost. If used to guide treatment decisions, these tools function as clinical decision support and should be evaluated with the same rigor as any intervention brought into the room.
