+++ AI Quality Summit on the 11.12. & 12.12.2023 in Frankfurt. Limited capacity for the conference featuring renowned speakers from the fields of mobility & production, finance, health and politics. More information. +++

Category: Blog

The AI Quality & Testing Hub has the mission to promote the quality of AI systems and to make it verifiable. In the Hub, companies have the opportunity to demonstrate and improve quality properties of artificial intelligence (AI) systems. This should help users gain confidence in AI applications and that novel technologies are in line with the European value system.

Large language models in clinical use

Large language models in clinical use

The following interview on the topic of “Language models in clinical use” was conducted with Dr. Michael von Wagner, MD, Specialist in Internal Medicine, Chief Medical Informatics Officer, Medical Director of the Medical Information Services and Digitization Unit, Staff Unit of the Medical Director and Chairman of the Executive Board, Managing Director of the University Center for Digital Healthcare (UCDHC), Senior Physician Medical Clinic 1 University Hospital Frankfurt.
The interviewer was Dr. Michael Rammensee, Managing Director of the AI Quality & Testing Hub.


Dr. Michael Rammensee: Mr. Wagner, could you start by giving us an overview of how AI and large language models work in everyday clinical practice and what benefits they offer?

Dr. med. Michael von Wagner: The use of powerful search engines developed specifically for medical professionals is widespread. They provide access to a wide range of medical information and resources that are crucial for clinical practice and research. The University Hospital Frankfurt has launched the SATURN project, which aims to support doctors in private practice in the diagnosis of unclear and rare diseases: A smart doctor portal and the use of artificial intelligence are intended to provide patients in rural areas, among others, with better and faster diagnostic options.

Dr. Michael Rammensee: Can you tell us something about the application of such information systems and similar systems like ChatGPT in medical care?

Dr. med. Michael von Wagner: The systems work in a similar way to ChatGPT in that it provides precise answers to general questions and acts as a virtual conversation partner. Although some clinics do not use LLMs due to data protection concerns, I see no reason to ban them in general for diagnostic support. We should not do without such systems.

Dr. Michael Rammensee: How do you see the integration of such systems into hospital information systems (HIS)?

Dr. med. Michael von Wagner: The integration of systems such as ChatGPT into HIS is an important step. In 9 out of 10 cases, such a co-pilot helps with questions that I might not think of. For example, it could point out a congenital disposition and suggest that the family be examined. Unfortunately, the cost of an interface is often very high, which often justifies its implementation only if the benefits are great.

Dr. Michael Rammensee: What challenges do you see in integrating these voice-supported technologies into everyday clinical practice?

Dr. med. Michael von Wagner: A major challenge is to ensure that treatment recommendations comply with the guidelines and are not based solely on the doctor’s experience. The doctor should not be disempowered, but should be able to make informed decisions together with the patient. In the meantime, digitalization is increasingly accepted by doctors and the debate about it is waning.

Dr. Michael Rammensee: In your opinion, what still needs to happen to make optimum use of these technologies?

Dr. med. Michael von Wagner: More needs to be invested in the preparation of decision documents. Doctors should be able to make informed decisions based on a wide range of information. A paradigm shift towards “The patient has the right to his file” is necessary. The idea of E-PA (“Talk to your patient record”) could offer solutions. However, there is a dilemma when data is automatically transmitted to the patient without explanation, e.g. tumor markers.

Dr. Michael Rammensee: How could these technologies improve patient care in concrete terms?

Dr. med. Michael von Wagner: In patient care, virtual medical assistants could be used, for example, to record patient consultations, take family histories and store information with metadata in a structured way. Replacing them with chat bots certainly doesn’t make sense, but automating smaller tasks such as recording conversations would be a big step forward.

Dr. Michael Rammensee: Finally, how do you see the future of AI governance in the healthcare sector?

Dr. med. Michael von Wagner: AI governance is an exciting topic and should be considered from the outset. It is about eliminating redundancies and bringing information into a chronology. This will not only improve efficiency, but also significantly increase the quality of patient care.

Dr. Michael Rammensee: Can you explain the importance of interoperability in this context?

Dr. med. Michael von Wagner: Interoperability is crucial. There are already certified services in other digital areas, such as a messenger service. Interoperability will also become an important feature of AI applications. These developments are not specific to the University Hospital Frankfurt, but are generic efforts.


 

Many thanks to Dr. med. Michael von Wagner for the exciting insights into the topic of “Language models in clinical use”.

Review of 2024 – outlook for a successful 2025

As the year draws to a close, we would like to take a moment to reflect on all that we have achieved together. 2024 has been a year of progress, innovation and meaningful collaboration, and none of it would have been possible without your continued trust and commitment. We would like to thank you from the bottom of our hearts for your expertise, dedication and exceptional teamwork that makes AIQ a true pioneer in the field of AI.


This year we welcomed two new team members:

Dr. Simone Amoroso, Head of Technology and Cyber Security Officer (CSO), who brings a wealth of knowledge with his PhD in particle physics from CERN. Simone’s leadership skills and expertise are vital if we are to further strengthen the security and innovation behind AI solutions.

Paul Luca Palupski, AI Systems Engineer Health, who has a Master’s degree in Health Informatics. Paul’s in-depth expertise in medical technology and AI applications in healthcare adds great value to our work, especially in the field of medical imaging and diagnostics.


We are proud to have hosted the 3rd AI Quality & Safety Summit as part of the Digital Summit 2024. The event was a resounding success and brought together thought leaders and experts to explore the future of AI safety and quality. Read more about the summit in our blog.

This year, we became an official advisor to the National Institute of Standards and Technology (NIST) and contribute to the AI Safety Institute Consortium (AISIC). We are honored to support NIST in the assessment, validation and verification of AI systems and to guide the development of AI red teaming strategies.

 

We are also very pleased to advise the “MISSION KI” project, a joint initiative of acatech – National Academy of Science and Engineering and the Federal Ministry of Economics and Climate Protection (BMDV). This important initiative aims to strengthen Germany’s digital competitiveness and we are proud to be part of this mission. Learn more about it here.

In the healthcare sector, we are continuing our collaboration with Prof. Valmed® – Validated Medical Information GmbH to develop and test an innovative AI-supported diagnostic tool based on large language models. This partnership underlines our commitment to improving healthcare through AI.

Our collaboration with the Korean Testing Laboratory (KTL), together with the VDE, was another highlight of the year. Together, we offer training and advice on the EU AI Act to ensure that companies are well equipped to comply with the latest regulatory standards.

We have also continued our efforts to raise awareness of AI and its impact through various public relations activities. In particular, we contributed to the HR Info Radio podcast discussing the risks of AI in the modern world. You can listen to the episode here.


Prospects

As we close this chapter, we look forward to continuing our journey of innovation and growth together in 2025. Your partnership is invaluable and we look forward to continuing to work together on solutions that will shape the future of artificial intelligence in a safe, ethical and impactful way.

Before we dive into the new year, we wish you a well-deserved break filled with joy, relaxation and the company of loved ones. May your holidays be joyous and may the New Year continue to bring you success and fulfillment.

AI Quality Summit 2024

The AI Quality Summit enters its third round!

We are pleased to announce the AI Quality Summit 2024, which will take place this year as part of the Digital Summit. Looking back on last year’s successful AI Quality Summit, which was attended by 150 participants from 10 countries, we are continuing the discussion on the quality of Artificial Intelligence this year.

🌐Experience the highlights of the AIQ Summit 2023 with our “AfterMovie“.

📅 Details of the AI Quality Summit 2024:

  • Date: October 22, 2024 as part of the 2nd day of the Digital Summit
  • Venue: Techquartier | Platz d. Einheit 2, 60327 Frankfurt am Main

This year’s AI Quality Summit will focus on the topic of safety. The program includes horizontal and vertical discussions in three key sectors: mobility and production, finance and health. You can look forward to keynotes, concise topic overviews and moderated panel discussions with Q&A’s from the audience. The AI Quality Summit is organized by the AI Quality & Testing Hub, an initiative of the VDE Verband der Elektrotechnik Elektronik Informationstechnik e.V. and the State of Hesse. The Digital Summit is an initiative of the German government.

On August 1st 2024, the EU AI Act officially came into force, a significant milestone for the industry. This legislation creates the framework for the ethical and responsible development and use of artificial intelligence in the European Union. This year’s summit will therefore focus on and discuss the EU AI Act and the topic of AI safety in particular.

💡 What is the EU AI Act?
The EU AI Act is a comprehensive set of rules that ensures that AI systems within the EU are safe, transparent and in line with fundamental rights. It categorizes AI systems into different risk levels and sets out corresponding obligations for providers and users of AI systems.

  • High-risk AI systems: Subject to strict requirements in terms of data management, documentation, transparency, human oversight and robustness.
  • Limited-risk AI systems: Must fulfill transparency obligations.
  • Minimal or no risk AI systems: Are encouraged to follow voluntary codes of conduct.

💡 Why is the EU AI Act important?

  1. Promotes trust and acceptance: By establishing clear standards and accountability mechanisms, trust in AI technologies should be strengthened and their broad acceptance and innovation promoted.
  2. Protects fundamental rights: Ensures that AI systems respect user privacy, non-discrimination and other fundamental rights, thus promoting the ethical development of AI.
  3. Promotes innovation: Creates a clear regulatory environment that supports innovation while mitigating potential risks associated with AI.
  4. Global leadership: Positions the EU as a pioneer in AI regulation and sets a benchmark for other regions.

Our commitment
At AIQ, we strive to adhere to the EU AI Act to ensure that our solutions are not only innovative, but also safe, transparent and ethical. We believe that responsible AI development is crucial to building a better future and we are excited to be part of this transformative journey.

For more details on the event, please visit the Digital Summit website. Places are limited, so make sure you secure your place in good time and request tickets directly from the Digital Summit.

Join the European debate on the quality of artificial intelligence and help shape the future of AI quality. We look forward to many participants and contributions at the AI Quality Summit 2024.

AI Made in Hessen: Where the Future Finds Its Home 

Hessen is not just a hub of culture and history; it’s also a hotbed for cutting-edge technology, particularly in the realm of artificial intelligence (AI). The state thrives innovation and works to actively shaping the future of AI. Hesse’s artificial intelligence ecosystem is both diverse and dynamic. Industries such as pharmaceuticals, chemicals, and automotive are already using a wide range of AI technologies to drive efficiency, innovation, and competitiveness. But the AI revolution in Hessen doesn’t stop there. In fact, it’s just getting started. 

Numerous start-ups dot the landscape, each one pushing the boundaries of what’s possible with AI, exploring new boundaries in mobility, energy, healthcare, and beyond. Their creativity and drive are favouring the region’s reputation as a powerhouse of innovation. 

Universities and universities of applied sciences in the region are conducting groundbreaking research in AI, pushing the boundaries of knowledge and driving practical applications forward. Their contributions are of great value for the advancement of Hessen to the forefront of the global AI landscape. 

Curious to see what AI in Hessen looks like in action? Look no further than the Hessian AI map, where around 150 examples of AI research and application are showcased: https://best-practice.ki-hessen.de/ . From groundbreaking academic research to innovative industry applications, the map offers a comprehensive overview of the variety of AI in Hessen. Take a journey of discovery through the map and witness firsthand the diversity of AI made in Hessen. 

Copyright: HMD

Furthermore, Hessen welcomes innovators to join its AI community. Whether you’re working on an exciting AI project or offering cutting-edge AI technologies, there’s a place for you in Hessen’s AI landscape. Become part of this thriving ecosystem, share your expertise, and collaborate with like-minded individuals to shape the future of AI in Hessen and beyond. 

In Hesse, we are actively shaping the future. Join us on this journey and let’s create a future together in which AI is a positive force that promotes innovation, prosperity and progress for all.

Do you have questions about the AI map?
E-mail: anfrage@ki-hessen.de

MISSION AI initiative sets benchmarks for AI quality standards

Promoting fair market conditions and innovation through responsible regulation


Promoting innovation and building trust in AI is a crucial step towards widespread and responsible use of AI applications. The National Initiative for AI-based Transformation in the Data Economy (NITD) is taking an important step in this direction with the introduction of the MISSION AI initiative. The initiative announced by Federal Minister Volker Wissing at the “Fueling European Innovation with AI” conference in Mainz focuses on cooperation in the development and testing of AI quality and testing standards.

The objectives pursued by MISSION AI are clear: the standards developed must be usable, appropriate and adaptable and promote an AI ecosystem of trust and excellence in Europe. A leadership role, especially in the area of regulation, means seizing opportunities, taking risks and adapting quickly. A fair balance between innovation and regulation is crucial, and MISSION AI brings together top partners such as the AI Quality & Testing Hub, CertifAI, Fraunhofer IAIS, PwC Germany, TÜV AI Lab and the VDE. Together, these organizations contribute their extensive expertise to make MISSION AI a driving force in shaping the future of AI.

Clear standards are the answer to safety issues in connection with AI. MISSION KI bundles expertise from science, consulting, standardization and testing. The focus is on putting artificial intelligence to the test and improving the quality of AI systems. The EU AI Act specifically regulates high-risk AI applications, while there are transparency obligations for non-risk applications and general AI models. Michael Rammensee, Managing Director of the AI Quality & Testing Hub, emphasizes the need to provide users with trustworthy access to intelligent applications.

The partners of the newly founded group – PwC Germany, Fraunhofer IAIS, TÜV AI Lab, CertifAI, the AI Quality & Testing Hub and the VDE – form a strong alliance that aims to drive the success of MISSION AI.

Further information can be found at www.mission-ki.de.

AI Quality Summit 2023

The AI Quality Summit goes into the second round!


We are pleased to announce the upcoming second AI Quality Summit, which will take place on 11. and December 12, 2023 in Frankfurt will take place. Following the success of our previous event in 2022, where we welcomed 150 participants from 10 countries, we look forward to continuing this important debate.

🌐Experience the highlights of the AIQ Summit 2022: Youtube

This year’s Summit promises to be equally impactful as we address critical quality criteria that include robustness, resilience, performance, testability, transparency and security. Our discussions will be embedded in the regulatory context of EU AI law, along with frameworks from organizations such as the OECD and other multinational institutions. We will also address the urgent need for Europe to compete in global markets with AI quality, as well as the multi-faceted technical challenges that underlie AI quality.

Please note that registration is required for the Summit. You can find the link to the registration here: Registration

The program will include both horizontal and vertical discussions covering three key sectors: Mobility and Production, Finance and Health. A dynamic mix of sessions is expected, including short keynote presentations followed by engaging panel discussions with keynote speakers, concise topic overviews preceded by in-depth sessions led by individual speakers, interactive scientific conference-style presentations and discussions, and moderated panel discussions with space for questions and answers from the audience.

📅 Event details:

  • Date: December 11, 2023 (12:00 – 20:00) and 12 December 2023 (09:00 – 14:00)
  • Location: House of Logistics and Mobility (HoLM), conveniently located near Frankfurt Airport.

The AI Quality Summit is organized by the AI Quality & Testing Hub, a joint initiative of the VDE Verband der Elektrotechnik Elektronik Informationstechnik e.V. and the State of Hesse. We are pleased that Prof. Kristina Sinemus, Hessian Minister for Digital Strategy and Development, will give the opening speech. We are also privileged to have Prof. Philipp Slusallek, Managing Director of DFKI Saarbrücken and head of the research area “Agents and Simulated Reality”, as chair of our program committee and one of our esteemed keynote speakers.

Join the European debate on the quality of artificial intelligence and be part of overcoming the myriad challenges it presents. A preview of our preliminary program, including a list of speakers confirmed so far, can be found as a download at the end of this post.

Places are limited, secure your place in time and register at the following link: Registration.

We look forward to many participants and contributions at the second AI Quality Summit. Together, we can shape the future of AI quality.


🔎For more details about the event, please visit the website, where you can also find impressions from last year’s event: Details.

For further questions or if you need more information, please feel free to contact us directly.

Presentation AI Quality & Testing Hub

What does the hub do and what do pre-prompt injections have to do with quality assurance?

AI systems enable technical systems such as machines or robots to independently process and solve abstractly described tasks and problems. A key challenge here is ensuring the quality of AI. The state of Hesse and the VDE have set themselves the goal of promoting the quality of AI and have therefore launched the AI Quality & Testing Hub. The Hub supports manufacturers in increasing and proving the quality of their AI products. This strengthens the international competitiveness of manufacturers and users of AI systems and increases societal trust in such systems.

Interested? There will be a talk at the Gesellschaft für Informatik e.V. on 07/19/2023.

Sign up here: https://rg-rhein-main.gi.de/veranstaltung/ai-quality-testing-hub