AI implementation in regulated TMF environments: what actually works

Organizations trying to scale AI in their TMF operations face a persistent tension. The technology accelerates month to month, yet inspection readiness demands precision and defensibility. The gap between "what's possible" and "what passes inspection" remains the real obstacle.

At TMF Week 2026, I joined a live panel on how organizations are using AI in their TMF today: what's working, what isn't, and what comes next. Moderated by Paul Carter, Montrium's CEO, the panel brought together Traci Wendler, business owner for TMF and CTMS at Genmab; Kevin Richards, who leads global clinical quality at AstraZeneca; and Eleanor Hewes, an associate principal in Syneos Health's R&D advisory group.

Their experiences point to the same conclusion. The teams closing the gap aren't the ones moving fastest. They're the ones treating AI as a tool that scales human judgment, not one that replaces it.

Moving past AI hype in TMF 

For years, organizations approached AI in TMF with a simple question: what can this do? The answer was autonomy. They envisioned end-to-end document classification with minimal human involvement, automated completeness checks, and predictive flagging of inspection risk. The reality proved more complex.

Traci first tried building an AI document classifier about seven years ago, before she joined Genmab. Her team picked what seemed like the perfect use case: the 1572, a standard form with the same layout, checkboxes and metadata everywhere. The technology should have worked. It didn't. As Traci put it, "we weren't ready for it." That early attempt revealed a critical lesson. Capability and readiness are not the same thing.

Her second attempt, about two years ago, was building completeness dashboards. It also fell short. Since then, Traci says, AI has "become a whole different animal." Genmab is now testing vendors for document classification and content review. She doesn't feel they're fully where she'd like to be yet, but she says they are "just about getting there." The difference this time is expectations: AI handles common, structured record types well, and humans stay in the loop for the rest.

The hype hasn't gone away. Kevin notes it gets more pointed the higher you go in an organization, especially now that "everybody can build agents." The challenge, he says, is "how do you get away from just building sexy use cases to building use cases that actually matter." Eleanor puts it more simply: "You don't wanna implement AI just for the sake of implementing AI."

From unreliable to ready: the four-year shift 

Four years ago, AI indexing was, in Paul's word, "terrible." The technology existed but didn't perform reliably.

Over the last six to twelve months, that changed. Capability accelerated significantly, and as Paul noted, AI is now "starting to really perform well." The standards AI depends on are catching up too, with more granular record type definitions and metadata standards on the way.

But acceleration in capability doesn't mean acceleration in adoption. Montrium's State of AI in Clinical Quality report found 47% of organizations still have no AI use in clinical quality today. Traci cautions that the technology is now outpacing the organizations using it: "AI is almost moving a little faster, and we have a little bit of catching up to do." Her advice is to work with colleagues, get aligned, and make sure AI use meets ICH E6(R3).

That's why the conversation is landing in a more grounded place. At Montrium, we're asking ourselves the same practical questions as everyone else. What's the right application for your environment? How do we validate? How do we govern? How do we keep humans in the loop in a way that's meaningful

Where AI actually delivers in regulated environments

Success follows a pattern. AI works best on high-volume, high-structure tasks where consistency matters most.

Document classification performs well when records are frequent, formatted similarly, and follow standard templates. Metadata extraction succeeds when you're pulling standardized data points from structured forms. Chatbots and everyday business uses, like help with emails and presentations, have been among the most reliable wins.

Traci has seen this firsthand: "Classification tools might be great for 60 classifications, but maybe not the rest of them."

The constraint isn't the algorithm, it's the long tail. Phase 3 trials can contain over 100,000 documents, with many variations by functional area: homegrown site-specific forms, documents with manual signatures, study-specific variations that don't match templates, and records in multiple languages. AI handles high-frequency record types reliably. The long tail, though, requires human eyes.

The actual win wasn't automation for its own sake. It was visibility. Kevin's team at AstraZeneca built live inspection readiness dashboards that surface quality signals across hundreds of trials in real time.

The goal, as he put it, was:

"Really trying to get away from this Excel hell that all organizations are feeling."

Kevin Richards, Sr Director, Clinical Quality at AstraZeneca

To build them, his team met with the people doing the tracking: clinical research associates (CRAs), global study associates (GSAs) and global study managers. Those clinical teams had been starving for a central capability, and they ended up helping shape the strategy. They weren't replaced. They were given tools to stop tracking quality by hand.

That shift (from collating spreadsheets to analyzing signals) is where AI delivers value in regulated environments.

When things break: lessons from failure and recovery 

Implementation failures happened too, and they're instructive. At Genmab, just six months ago, someone classified six documents incorrectly and the AI learned from it, treating that as the right place to file them. The team had to correct it. This wasn't a technology failure. It was a training failure. As Traci put it, "AI is great, but it's also only as good as we train it." Her takeaway is that "it's really important when you first start to work with AI that you're not expecting some miracle." You have to put in the work at the beginning.

Early failures also cost trust. As Paul noted, when AI doesn't turn out well, it erodes confidence, and you have to build that trust back up before people will accept it. Eleanor has seen the same pattern across sponsors. The companies that succeed take time upfront to understand their current processes and pain points, bring in end users early, and back the effort with strong change management and active leadership participation. In her words: "Regardless of the use case, I think if you include the people upfront, that's where we're seeing the most success."

Translation: the wins come when you build with people, not for them.

The human element determines sustainability 

This is where the regulatory frame becomes critical. Building AI indexing at Montrium taught us that getting the model to perform well is one thing. Building something that can stand up in a GxP context is totally different.

The concern isn't whether AI works. It's whether inspectors understand why you chose to use it, can verify it was validated, and can confirm humans review the decisions that matter.

The regulatory framework already points the way. It pushes organizations to build AI with three things in place:

1. Validation: You can show what the model does well, where it struggles, and what training it required. This evidence becomes your defence in the inspection room.
2. Documented rationale: Someone can explain why AI was used for a task and what it was trusted to do.
3. Proportionate human review: Humans stay in the loop where judgment matters, and everything AI does is tracked in the audit trail.

This also reshapes roles. The analogy I keep coming back to is electronic data capture (EDC). Before EDC, clinical data managers spent much of their time on data entry and checking transcription. EDC didn't eliminate the role. It changed what they spent their time on. I see the same trajectory for TMF professionals with AI.

Kevin puts it bluntly: his team is "getting quality professionals to do what they should have been doing ten years ago," digesting information rather than collating it at scale.

The question we should be asking isn't "will AI be able to do my job better than me?" It's "will I be able to prompt, govern and review AI better than the next person?" That becomes the career differentiator. Knowing when the model is wrong, why it's wrong, and what the regulatory consequences are doesn't automate. It just becomes more valuable.

5 ways to implement AI defensibly 

If you're beginning now, these principles separate success from waste.

1. Don't chase autonomy, chase validation: Start by asking what's essential to defend, not what's cool. The regulatory framework expects validation, documented rationale and proportionate human review. Let that drive the roadmap.
2. Pick your first win carefully: Target high-volume, high-structure problems. Not the most important problem, but the easiest one to prove works. Build momentum and internal confidence before tackling complexity.
3. Feed the model what it needs to succeed: The CDISC TMF Standard Model v1 gets more granular, with a definition for every record type, and metadata standards with their own definitions are on the way. That gives AI clearer rules to work with, but only if your data matches those definitions. As Paul put it, AI is a bit like a human: "If you feed it well, it's gonna be healthy."
4. Move at the pace your organization can absorb: Kevin admits he always wants to push, but he's learned that stakeholders can only digest one major overhaul every quarter or six months. As he put it, "we gotta also move at the pace that the organization will adopt." Start cautiously, build confidence, prove value on one use case, then expand.
5. Embed AI into your quality strategy, not as a standalone project: Don't bring in a consultant to do AI for one use case. Build AI as a capability layer on top of your existing quality roadmap. AstraZeneca's approach illustrates this: build the reporting and applications first, then embed AI use cases and agents into tools stakeholders already use
   

Beyond indexing: what sits on top

At Montrium, assisted document indexing, classification and metadata extraction is where we started, and we're now in the final stages of release. But for us, indexing is the foundation. What we're more excited about is everything that sits above it.

The first is quality signals. Given the right context and rules, AI can surface anomalies and flag patterns a human should look at, pointing oversight to where it matters. That maps directly to ICH E6(R3), which asks sponsors to show why they focused their attention where they did.

The second is essentiality. E6(R3) asks whether a record is essential: is the record required to evaluate the conduct of the clinical trial and the quality of the data produced? The guideline gives a defined set of criteria that work much like rules. AI can apply those rules to identify primary evidence, what's essential, and what may not need to be in the TMF at all. The result is a leaner, cleaner TMF. As Paul noted, "we collect way too much information  

What changes when AI works right

When the pieces line up, the shift is bigger than efficiency. Quality teams stop collating data and start analyzing risk. Kevin describes his own team becoming "more of an insights group," moving from finding issues after they happen to spotting risk signals before they matter. Clinical teams get live information they used to track by hand. As Paul Carter observed in closing the panel, that changes the relationship between quality and operations: quality teams are "no longer playing that police role."

That's the real lesson from the panel. AI is ready to take on the administrative work of the TMF. Whether it does depends less on the technology than on how prepared your organization is to use it.

Where to start

You don't need a full AI strategy to act on this. Three first steps you can take this quarter:

1. Map your pain points with the people doing the work.

Before you choose a tool, ask your CRAs, study managers and TMF team what they still track by hand. That's how Kevin's team found its roadmap, and it's the pattern Eleanor sees in the most successful implementations.

2. Define the human check before you deploy

Decide up front who reviews AI output, how errors get corrected, and how that's documented. That way a misfiled pattern gets caught before the model learns it, and the long tail always has human eyes on it.

3. Build your own AI skills.

When an attendee asked whether clinical ops teams will need AI training, Kevin's answer was "100% yes," and I agree. Learning to prompt, govern and review AI is becoming part of the job.

The organizations closing the gap between "what's possible" and "what passes inspection" aren't moving fastest. They're moving purposefully.