Introduction
In early 2025, our residency implemented the AI note-writing service “Abridge” without prior evaluation of effects on clinical documentation. There has been concern that AI designed to emphasize billable text would lack details important for Relationship Centered Care. As our program wishes to emphasize human interaction and relationships, we chose to look at how the adoption and use of AI technology may be affecting the process of creating clinical documentation. The hope is that the results of this project will be used to improve how not only we as a residency, but how we as a profession utilize AI technology in our day-to-day practice and documentation habits.
Methods
We compared: (1) faculty dictated narrative and (2) AI-generated progress note, each from the same visit. These two notes were compared to a visit transcription recorded by the AI. A panel of a senior physician, a clinical psychologist, and two residents scored based on two factors: Whether the visit contained any Relationship Significant Events (RSEs: bereavement, disclosure of sexual orientation, narcotics diversion, divorce, emotional abuse, etc.); and how much the AI and dictated notes documented these events (0=absent, 1=minimally present, 2=meaningfully present).
Results
26 of 47 office visit transcriptions contained relationships significant events (RSEs) and the others were purely biomedical encounters. Fourteen of the 26 provider-dictated notes contained higher RSE scores compared to AI generated notes. The reverse was true in only two cases. X2 (1, N=52) = 13, p<.001. This can be interpreted to mean that the clinician was more likely to record important relationship data in the progress note when compared to the AI generated note. An unexpected finding was that six charts categorized as purely “biomedical” were simultaneously identified as Balint appropriate cases. This significant emotional component of patient care could never be detected by a Large Language Model AI.
Conclusions
The core values of Family Medicine emphasize Relationship Centered Care whereas AI technologies maximize billing documentation and convenience. Based on the findings, the senior faculty member continued dual entries where he would use AI and dictate his note to preserve relationship significant events not seen by the AI. Our project was also educational for our residents who had largely adopted the AI program, showing that they may need to manually add on documentation for the relationship aspects of visits to ensure that such aspects remain part of the medical record.