AI Safety at Scale: a new frontier for LLM oversight in healthcare

Newton’s Tree has been awarded funding from the AI Security Institute for CLIO: Clinical LLM Intelligent Oversight, to further develop its generative AI monitoring capabilities.

Large language models (LLMs) are becoming deeply embedded in hospitals, assisting with clinical documentation, diagnosis, and patient communication. Despite their promise, mistakes from AI - such as generation of inaccurate and biased outputs - can propagate, amplifying errors across the entire ecosystem. We need to ensure that LLM technologies remain safe, effective, and aligned with clinical needs, and that clinicians also remain engaged, accountable and don’t blindly trust AI insights.

Effective AI adoption depends on tackling risks head-on. At Newton’s Tree, we’ve already been working on these challenges and have been awarded a grant from the AI Security Institute to develop CLIO, a tool designed to bring real-time monitoring and governance to LLMs in healthcare.

The Solution: Real-Time AI Monitoring

CLIO aims to address the challenge of keeping LLM technologies safe by delivering real time tracking of how LLMs are used in clinical settings, identifying risks before they escalate. By developing proactive safety measures such as tools to flag performance drift and automation biases, we can take steps to ensure AI remains aligned with patient care.

Built on a Strong Foundation

FAMOS (Federated AI Monitoring Service) - our monitoring tool for imaging AI solutions -  is already recognised internationally, having been included in the OECD & GPAI Catalogue of Tools & Metrics for Trustworthy AI. With CLIO, we are expanding AI monitoring beyond imaging into generative AI. Our work is also being strengthened by our collaboration with the MHRA AI Airlock, a regulatory initiative to solve regulatory barriers to support safe and effective AI adoption, testing how our third party monitoring can help the hospital and vendor keep AI safe and reliable.

Get in touch to learn more about CLIO and how you can get involved.