The release of ChatGPT by OpenAI in 2022 has stimulated interest in large language models (LLMs), leading to research in recent months on various healthcare applications of LLMs, such as for clinical workflows, online psychological services, but also for triage and symptom assessment in emergency rooms.

Inspired by these technical possibilities, the ILLUMINATION research project envisions that instead of filling out traditional forms, patients in the emergency room could share their symptoms with an LLM-based application. Based on the information provided, the LLM-based application assesses the symptoms and suggests a pre-prioritization to the medical staff.

However, the integration of LLMs in such a privacy-sensitive area also poses significant privacy risks. Therefore, the overall goal of ILLUMINATION is to develop a toolbox of technical privacy methods and interdisciplinary recommendations for the privacy-preserving use of LLMs in health care.

The HCC investigates user attitudes towards the use of LLMs in healthcare, i.e., we want to gain a better understanding of people's mental models and preferences regarding LLM-based applications and, based on this, propose user interface designs for meaningful patient-LLM interactions, with a special focus on factors that influence the disclosure of sensitive data. In the emergency room, however, the special situation of people being under pressure needs especially to be considered, thus, people need to experience the interaction to be appropriately trustworthy.

In addition, we want to enable the medical staff to make an informed decision on the recommended pre-prioritization using methods of human-centered explainable AI (HC-XAI). Thus, it is necessary to communicate the uncertainty of LLM-based decisions to ensure effective human-AI collaboration.

The project »ILLUMINATION« is funded by the Federal Ministry of Education and Research (BMBF).

Project partners:

• CISPA Helmholtz Center for Information Security (CISPA) | Secure, Private, Robust, INterpretable, and Trustworthy Machine Learning (Sprint) | Dr. Franziska Boenisch/Dr. Adam Dziedzic
• Universität Heidelberg | BioQuant / Faculty of Law | Prof. Dr. iur. Fruzsina Molnár-Gábor
• Charité – Universitätsmedizin Berlin | Notaufnahmen Campus Mitte und Campus Virchow Klinikum, Notfallmedizinische
  Versorgungsforschung | Prof. Dr. med. Martin Möckel/Prof. Dr. rer. medic. Anna Slagman
• Algonaut GmbH | Creating Knowledge with Large Language Models