Going through the vendor selection process of any new software is an exciting and sometimes daunting task. Once you've got a good idea of what you're looking for and found a few options that fit the bill, you'll likely reach out to vendors to test drive their tools and have them demonstrate the software for your team.
Product demonstrations are often your first real glimpse at a new way of working. You'll be guided through the user interface, exposed to the different areas of the system, all with the ultimate goal of seeing how the tool will work within your organization. However, demonstrations can be more than just a show and tell. These sessions provide your team not only with the opportunity to see how a system works, but to dig deeper into some of the other items related to selecting a vendor. If they are not prepared for or managed properly, these demonstrations can spin off into tangents that are not of interest or related to your organization in any way and you could be left feeling like you've wasted your time.
As you progress through the vendor selection process for your Electronic Trial Master File system, ensure you are asking the right questions. To keep you on track we've listed out 10 essential questions to ask in an eTMF demonstration.
1. Is your system pre-configured based on the latest version of the TMF Reference Model?
If you don't already structure your Trial Master Files (TMFs) in alignment with the DIA Trial Master File Reference Model, now may be a good time to make the switch. The TMF Reference Model is helping us standardize the way we manage our documents and data, grouping content into process zones and sections alongside standardized metadata. When you have a standardized structure for managing TMF content, you have the ability to dynamically create document placeholders, measure completeness, and automatically associate metadata for all of your document types. This not only makes it easier for your teams to find content, but improves search and retrieval. So, don't be too shy to ask if the eTMF system comes pre-configured based on the TMF Reference Model? and If so, which version are they adopting?
2. What does the validation process consist of?
Whether you have your own in-house validation team or would scramble for resources to perform your user acceptance testing, the validation process isn't going anywhere. Being that you're going to have to go down this road regardless of your preparedness, it's important to understand what validation approach your vendor suggests, and what your responsibilities are during this process. Here are some questions we'd suggest to ask surrounding validation:
- What validation approach do you recommend?
- Does the system come pre-qualified (IQ & OQ completed)?
- Do you provide test scripts and templates to speed the validation process?
- Do I need to update any SOPs or documentation?
3. Is training included and if so, what is involved?
As you can imagine, having your users feel confident and competent in the eTMF system will beone of your primary goals. Poor user adoption leads to inconsistencies in your process, potential human errors and a wasted investment. You'll want to look for a vendor that put a focus on intuitive user experiences and comprehensive training to ensure your team go live with confidence. Here are some questions that will expose a non-customer centric eTMF vendor:
- How much training will I receive as part of the onboarding process?
- What resources are provided during the training sessions?
- Are training sessions interactive or user guide based?
- What happens when new team members join, is there on-demand training material?
- Do you have any self-guided certifications or courses we can take?
4. Are there any system limitations in regards to users, documents or studies?
We are here to talk about the scalability of the eTMF and understanding the full capabilities and limits of the solution. It is important if you are a large organization to understand exactly how many documents can be managed in the system and if there are any limits. Of course, having an idea of your current and expected volume is useful to determine if the system will meet your needs.
You should also look into the number of users you'll need and the number of studies you'll be managing, as you should be looking for a system that can be agile and grow with you. You are going to want a system that is flexible and can handle any influx of documents and users at any given point in time in your growth process.
5. How does your pricing model work?
When it comes to SaaS systems, there are a number of different pricing models that exist in the eTMF space. Depending on the number for studies, sites and users you'll be managing, some pricing models can be more practical for you than others. For example, implementing an eTMF with a model that scales by the number of clinical sites you have, may be more costly for larger studies. However, if you have large teams and require many users, pricing models based on users may be less practical than the per site models.
Do your research on how you'll use the system and find a model that works for your organization. Pricing should not be the driving point when selecting an eTMF vendor, but of course budget is always at the forefront of any decision. Again, don't be afraid to ask what does your pricing model look like and does it work for my situation?
6. How do your support your clients?
Making sure that you are supported, especially during your core business hours is definitely a question worth asking. You may want to ask “How will I be supported throughout the lifetime of my software and does this come at an extra cost?”. Most modern SaaS customers now bundle support into your subscription, adding no penalty or added fee. You'll also want to understand how you receive support. Ask these questions to dig deeper:
- Do your clients submit tickets through a web ticketing system?
- What is your support model and response times?
- Is support included in the fees?
- Is support provided only via email, or will phone and web conference be available?
Another point to be mindful of is on-demand or self-help material. If one of your users gets stuck in the system and doesn’t know how to proceed, you will want to understand how that user will resolve the issue without having to call support. Most vendors will provide standard user guides and quick reference guides for specific activities. However, more and more we are seeing more interactive and dynamic learning content to help users navigate and use the system. It’s important to understand what the vendor has in place for this?
7. Think of your use case and ask how this process would work in the eTMF system?
There are different ways to leverage an eTMF solution, and it is important to understand how exactly it will work for you and your organization prior to investment. How do you envision uploading documents? Do you want to give access to consultants, partners, sites and if so, does the system support the robust permission management? Does the system support bulk upload and batch indexing?
Think about the processes you currently manage manually, and ask the eTMF vendor to demonstrate how you'd carry out these steps in a real-life situation in their system. While the steps may vary slightly from your process, you'll need to evaluate how well you're team will adapt and work in the system with the use case in mind.
8. What is your service level agreement?
Service level agreements are contractual documents that dictate the level of service and support you'll receive from your vendor. While you'll likely go over this during the final negotiation phase, it can sometimes be valuable to hit a vendor with some of the important points up front to not waste time.
At this stage you want to understand what will be the standard response times for support events or system downtime and cover key points related to how issues will be escalated based on their severity. You'll also want to identify how many backups your potential vendor provides and what happens if my system crashes/goes down and what up-time do they guarantee? It's important to remember, no software system is flawless (we're talking about technology after all) and you'll never find anyone in IT guarantee 100% uptime. However, you should be looking for a vendor that is honest and transparent about these items, as these are the vendors you'll find it easier to trust.
Every vendor has their own idea of an onboarding process and a go-live strategy. Deployment of a cloud environment might take very little time, but the difficult part is the onboarding. The scope of this could differ case by case but knowing what is included is imperative to a successful implementation. Training, validation, and other activities are defined differently depending on the organization, this is the same with the go-live date. Go-live may be defined as the day you finish training, or the day you have completed user acceptance testing. Make sure you are aware of what go-live means to your vendor and be sure that it aligns with your needs. Preparing this in advance will allow your organization to have a much smoother implementation process. And most importantly, set realistic goals. In a perfect world we would be trained and up and running in a week, but we still have day jobs and other tasks that get in the way. Implementation takes resources on both sides and setting realistic timelines and preparing in advance will go a long way on your path to eTMF success.
10. Is the system compliant with the regulations that are relevant to your organization?
Compliance with 21 CFR Part 11 and Eudralex Vol. 4 Annex 11 is mandatory if you're doing business in either the US or Europe. However, there will likely be other regulatory requirements that you'll need to comply with, so make note and quiz your vendor. You may also find that some eTMF vendors have produced white papers and public documentation that show the regulatory requirements that can be met through using their tool.
Complying with the regulatory requirements is a given, but you'll need to ensure you ask the right questions regarding what is relevant to you during your demonstration.
11. How does the solution provide access to users outside of my organization?
In some cases, you will need to give access to partners, customers, and even the regulators for collaboration or inspection. It’s important to understand how these users will gain access to the system and how permissions are managed so they only see what you need them to see. The more contributors you have in the system, the more granular you want your permissions to be. Make sure this is crystal clear from the get-go or you can be at risk of not being compliant or exposing confidential information to the wrong individuals.
It's also important to confirm whether users from outside of the organization carry different fees with your eTMF subscription. While most may not make the distinction, it's always good to confirm if there are different prices ahead of time.
The Takeaway
Product demonstrations are only as useful as you make them. Don't be afraid to make it your own and ask questions throughout the session, after all this demonstration is for your benefit. Always try to dive into as much of the system functionality as possible, you’ll be using this software every day and you need to make sure that it suits your needs as best as possible.
Good luck on the hunt!
