We recently brought together leaders from across the clinical trial space for our annual TMF Week virtual summit to hear from over 30 experts on the topics of trial master file (TMF) management, industry trends, and all things clinical operations.
Out of the many interesting subjects discussed at TMF Week, one topic took the lead: data. Data capture, data quality, metrics, and so on. This makes a lot of sense to us, since the best trial stories are told with accuracy and data are crucial to obtaining a precise overview of your progress. The conversations we heard around data ultimately echo the great shift from manual TMF management to eTMFs, allowing for more modernized and data-driven clinical trial practices.
We compiled some our favourite data-related quotes and tips from TMF Week just for you.
Types of data in clinical trials
Data collected from the TMF is one of the most useful types of data for assessing the high-level progress of a clinical trial, so it’s crucial to make sure that you’re collecting and measuring it effectively and efficiently. Clinical data is, of course, extremely important and must be held to a certain standard, which is why the International Conference on Harmonisation created the Good Clinical Practice guidelines to ensure the credibility of clinical trial data.
Before we dive into tips for collecting high-quality TMF data, let’s review some of the key metrics to be tracking in your TMF:
- Completeness: Measures whether your TMF contains all of the necessary information by calculating the percentage of the total number of final artifacts out of the total number of expected artifacts.
- Timeliness: Measures the time it takes for information to be filed in the TMF, primarily through measuring the percentage of artifacts filed on time.
- Quality: Measures the accuracy and completeness of information filed in the TMF through tracking the percentage of reviewed artifacts as well as the rejection rate.
- Administration: Shows whether the TMF has planning measures and defined ownership in effect.
- Use: Measures how often an eTMF system is accessed.
- Volume: Measures the number of documents in an eTMF or TMF, as well as the types and sizes of those documents.
TMF data extends beyond just metrics, though, and can include aspects such as metadata describing the properties of documents within the TMF. Now that we’ve had a primer on which types of data to keep in mind, let’s see what our seasoned experts have to say!
TMF management: it’s all about the data
How to capture and collect TMF data
With an increasing number of organizations making the switch from a paper TMF to an eTMF system, we are discovering more and more ways to make data work for us. But in such a multifaceted environment, how can we know where to begin? Well, for starters, Judyth Zahora from Agenus provided some practical advice for prioritizing actionable collection of data in clinical trials:
“Don’t measure something that you’re not going to take action on. If it’s not actionable, don’t measure it. Collecting measurements, collecting data—it's expensive and time-consuming. It ticks people off, frankly, if you’re going to measure them and there is no escalation or plan what to do with that data. So, if you can’t do anything with it, don’t collect it today.” – Judyth Zahora
For example, if you collect data on rejected documents, you should be sure to issue queries and run reports that allow you to capture the problem and identify its source; if it involves numerous documents from a single user, then more training may be necessary. Just because you know which data you would like to capture, however, does not mean that you’re ready to begin; you need to determine which types of data management techniques will best suit the needs of your organization and your study. In terms of which strategies to employ, Katherine Cianciarelli from Montrium had the following to say:
“By applying data management techniques similar to what we see in our electronic data capture (EDC), we can run logical checks on the metadata stored in our eTMF to make sure it is telling an accurate story.” – Katherine Cianciarelli
In order to successfully capture and collect TMF data, you must first prioritize which measurements to monitor and understand why you are doing so. Once you’ve established the “what,” you can then focus on the “how” by selecting the appropriate data management techniques and devising a practical plan of action.
How to ensure TMF data quality
Once the TMF data are captured and collected, it’s important to ensure that they meet the rigorous industry-wide quality standards. With technology, it’s becoming easier than ever to assess the quality of clinical trial documentation as developments in the field of artificial intelligence (AI) and machine learning allow us to better process and understand TMF data. Franciska Darmer from NNIT explains:
“What AI allows you to do is an automated assessment of the data quality before it hits your eTMF. It allows you to go through thousands of line items and really flag if there are any data inconsistencies between what you expected to achieve and what you have achieved. Did you receive too much data? Was there a lack of data? Were there inconsistencies in the use of data?” – Franciska Darmer
While groundbreaking AI tools certainly have the potential to optimize clinical trial data management, organizations can begin getting a better handle on data practices by simply beginning to transition to electronic systems.
How to master TMF metrics for success
One of the most significant applications of data-driven practices in the TMF space is the monitoring of metrics in order to gauge the overall health of the TMF. Metrics allow clinical operations professionals to assess their compliance with regulatory requirements and their readiness for inspection. As Nisi Nazim of Daiichi Sankyo highlights:
“Completeness and accuracy of the TMF is a significant regulatory requirement. Documents must be available in a timely manner for defined milestones with the aim to maintain the TMF ‘inspection ready.’”– Nisi Nazim
Completeness, however, can be difficult to measure and track, especially when it comes to large-scale studies. Luckily, Brenda Brown from Pfizer had a tip for staying on top of TMF completeness:
“A couple important notes to ensure we have completeness is conducting those comprehensive TMF QC or content reviews, because a lot of times the metrics are only indicators and it’s more like you don’t know what you don’t know until you really complete that review to see what may be missing, etcetera.” – Brenda Brown
Completeness isn’t the only metric with which problems can arise, though, as measuring TMF quality and timeliness can come with its own complications. For instance, it can be challenging to set realistic timeliness expectations that are tailored to the bandwidth of a given organization. Jim Markley of Just in Time GCP shared his take on mastering TMF timeliness:
“You have to start somewhere that’s reasonable, something that’s achievable. [...] Find that kind of sweet spot where maybe you start with 45 days, and if your organization shows that they can do 45 days with no problem, then you bump it up. It doesn’t have to be set in stone right off the bat, but you need to identify something that you can measure with some sort of recourse.” – Jim Markley
How to leverage TMF metrics
eTMF systems have simplified and streamlined the process of collecting and tracking metrics. With more clinical trial data than ever at our fingertips, the question then becomes how best to leverage these metrics to monitor and improve the health of the TMF, as well as TMF management processes as a whole. As Karen Freid from Merck notes, system-generated metrics can be an important source of information:
“System-generated metrics are very important because you can get those out of the system pretty easily in most of the eTMFs, if not all of them, and it’s a good way for you to check the timeliness of your documents...And I think the importance of system-generated reports is that it’s something you can then take those reports and file them. For us, we file those in our eTMF as well to demonstrate that we have looked at those things, and if we see trends, we would do a more intensive search and increase our risk-based approaches for that as well.” – Karen Freid
TMF metric reports can then be used to communicate the progress of the study with important stakeholders, allowing everyone to stay informed and to identify potential problem areas. These reports can be particularly helpful for facilitating the exchange of information between sponsors and contract research organizations (CROs), as Tonia Huggins from Syneos Health explains:
“One of the things that we do is create a TMF health report that we provide out to our sponsors on a monthly basis that gives an overview and a snapshot in time of the TMF completeness, and an overview of the timeliness for document filing. Our completeness reviews are done milestone and event-based, so it provides an overview of where we are at with those milestone reviews for completeness. Are they occurring on time? Are they late? Are they pending? You know, where those are at. And we kind of give this holistic overview in this monthly snapshot report.” – Tonia Huggins
With an eTMF system, you can create and export these reports with the click of a button; this helps increase organizational oversight and ensures that everyone is on the same page.
Key Takeaways on Data-Driven TMF Management
- Successful capture and collection of TMF data begins with creating a detailed plan of action; map out which data you wish to collect and how you wish to collect them before you get going.
- Exciting innovations like AI hold great promise for easing the burden of assessing TMF data, but you can begin optimizing this process today by transitioning to electronic systems such as eTMF Connect
- Monitoring key metrics is essential for ensuring that your TMF is sufficiently healthy and compliant, but remember to keep your expectations reasonable!
- Don’t just track TMF metrics—use them! Metrics are extremely valuable for not only keeping yourself informed about the progress of the study, but also for communicating across organizations.
If you enjoyed reading these tips and want to hear more about the use of data across the TMF industry, or just about TMF management in general, you’re in luck! We’ve made all of the recordings from TMF Week 2021 free to access online here.
