Large language models in clinical use
The following interview on the topic of “Language models in clinical use” was conducted with Dr. Michael von Wagner, MD, Specialist in Internal Medicine, Chief Medical Informatics Officer, Medical Director of the Medical Information Services and Digitization Unit, Staff Unit of the Medical Director and Chairman of the Executive Board, Managing Director of the University Center for Digital Healthcare (UCDHC), Senior Physician Medical Clinic 1 University Hospital Frankfurt.
The interviewer was Dr. Michael Rammensee, Managing Director of the AI Quality & Testing Hub.

Dr. Michael Rammensee: Mr. Wagner, could you start by giving us an overview of how AI and large language models work in everyday clinical practice and what benefits they offer?
Dr. med. Michael von Wagner: The use of powerful search engines developed specifically for medical professionals is widespread. They provide access to a wide range of medical information and resources that are crucial for clinical practice and research. The University Hospital Frankfurt has launched the SATURN project, which aims to support doctors in private practice in the diagnosis of unclear and rare diseases: A smart doctor portal and the use of artificial intelligence are intended to provide patients in rural areas, among others, with better and faster diagnostic options.
Dr. Michael Rammensee: Can you tell us something about the application of such information systems and similar systems like ChatGPT in medical care?
Dr. med. Michael von Wagner: The systems work in a similar way to ChatGPT in that it provides precise answers to general questions and acts as a virtual conversation partner. Although some clinics do not use LLMs due to data protection concerns, I see no reason to ban them in general for diagnostic support. We should not do without such systems.
Dr. Michael Rammensee: How do you see the integration of such systems into hospital information systems (HIS)?
Dr. med. Michael von Wagner: The integration of systems such as ChatGPT into HIS is an important step. In 9 out of 10 cases, such a co-pilot helps with questions that I might not think of. For example, it could point out a congenital disposition and suggest that the family be examined. Unfortunately, the cost of an interface is often very high, which often justifies its implementation only if the benefits are great.
Dr. Michael Rammensee: What challenges do you see in integrating these voice-supported technologies into everyday clinical practice?

Dr. med. Michael von Wagner: A major challenge is to ensure that treatment recommendations comply with the guidelines and are not based solely on the doctor’s experience. The doctor should not be disempowered, but should be able to make informed decisions together with the patient. In the meantime, digitalization is increasingly accepted by doctors and the debate about it is waning.
Dr. Michael Rammensee: In your opinion, what still needs to happen to make optimum use of these technologies?
Dr. med. Michael von Wagner: More needs to be invested in the preparation of decision documents. Doctors should be able to make informed decisions based on a wide range of information. A paradigm shift towards “The patient has the right to his file” is necessary. The idea of E-PA (“Talk to your patient record”) could offer solutions. However, there is a dilemma when data is automatically transmitted to the patient without explanation, e.g. tumor markers.
Dr. Michael Rammensee: How could these technologies improve patient care in concrete terms?
Dr. med. Michael von Wagner: In patient care, virtual medical assistants could be used, for example, to record patient consultations, take family histories and store information with metadata in a structured way. Replacing them with chat bots certainly doesn’t make sense, but automating smaller tasks such as recording conversations would be a big step forward.
Dr. Michael Rammensee: Finally, how do you see the future of AI governance in the healthcare sector?
Dr. med. Michael von Wagner: AI governance is an exciting topic and should be considered from the outset. It is about eliminating redundancies and bringing information into a chronology. This will not only improve efficiency, but also significantly increase the quality of patient care.
Dr. Michael Rammensee: Can you explain the importance of interoperability in this context?

Dr. med. Michael von Wagner: Interoperability is crucial. There are already certified services in other digital areas, such as a messenger service. Interoperability will also become an important feature of AI applications. These developments are not specific to the University Hospital Frankfurt, but are generic efforts.
Many thanks to Dr. med. Michael von Wagner for the exciting insights into the topic of “Language models in clinical use”.