To quote Heraclitus, “Change is the only constant.” 2500 years later, in an era of technological leaps, this has never been truer.
For validation veterans exploring the idea of moving their GxP operations to the cloud, this thought can be nerve-racking. There are some who would even claim that ever-changing cloud-based systems can never be maintained in a validated state.
In this article, I will explain why we at Montrium believe that this isn’t the case, and present different approaches that would support the use of a cloud-based system for GxP use. Specifically, I will explore how changes can be managed to ensure that the validated state is maintained.
Classic View: System Freeze and Change Control
In the classic validation view of the world, systems were deployed, qualified/validated, and then released into operation. When changes were required, they would be evaluated, implemented, and tested outside of the production environment, then later deployed into production if it turned out they provided the desired benefits without adding any unintended consequences. For this perspective to work, a high level of control over the system is required. In this model, systems are essentially static, meaning that the environment cannot change on its own.
The shift toward cloud-based systems and DevOps principles has challenged this and brought with it a new reality. Cloud-based systems do evolve, and often outside of the cloud-consumer’s control. After all, a key benefit of adopting a cloud offering is to have constant access to the “latest and greatest version”. New releases are automatically pushed to end users without prior authorization; something that doesn’t sit well with the classic system change control principle. Under this new paradigm, we can either stick to the long-standing “static” view, or embrace the development of new policies and procedures that can accommodate an ever changing system without jeopardizing our obligations and responsibilities as GxP organizations.
Private or public?
In general, cloud is offered in two configurations: public or private. Private cloud can mean a number of things, however, it more or less boils down to having the equivalent of a classic on-premises installation, except virtualized on the cloud. This means you—the IaaS, PaaS, or SaaS consumer—are still responsible for any future changes needed. If you plan to move to a private cloud, then you are still able to follow a classic validation model but need to keep in mind that you will also be foregoing many of the benefits of cloud, not the least being the OPEX advantages you would normally receive.
Public cloud, on the other hand, puts the onus on your 3rd party service provider to update the underlying platform, usually without prior explicit consent. They will inform you of changes to come, and depending on the cloud provider, you can also take part in different release cycles (early, normal, late, etc). However, one thing you cannot do is avoid changes. Changes, at some point, will be imposed on you. How then, in an ever changing world, can you remain in control?
Risk management as a guiding principle
GAMP 5, the validation framework most widely recognized and adopted around the word, has been evangelizing about risk management since 2008. Back then, this novel, risk-based approach to computerized system validation (CSV) was a well-needed answer to the issues caused by the indiscriminate application of CSV principles.
Because CSV was finding its way into everything, it had become a burden to a life science industry that was striving for innovation; driving up costs that outweighed value. The same risk management principles identified in 2008 are still very much at play today when it comes to maintaining validation in a public cloud.
GAMP 5 identifies three key risk areas: Data Integrity, Product Quality and Patient Safety. I would advise anyone to stick to those key areas of risks management when it comes to evaluating their cloud use.
However, in my opinion, there is a critical fourth high risk area to consider: observing and upholding regulatory requirements. As an example, if you are operating in the USA and are planning to manage GxP records, you must comply with the electronic records requirements identified in CFR 21 Part 11. Regulatory requirements are key to reaching and maintaining your validated state and will certainly impact risks tied with your system.
Evaluating Cloud Options through a Risk Management Lens
When switching to the cloud, adequate risk mitigation and management start with your own assessment, a process that should occur during the original system validation effort.
Nowadays, cloud service providers (CSP) are providing customers access to a greater number of resources and independent certifications, alleviating concerns surrounding data integrity. After all, it is the CSP’s business to ensure your data is protected. This means providing you with best of breed systems, a robustness that cannot be easily replicated on-premises over an extended period of time.
The inherent business risk of the cloud, which has played against cloud-based systems when evaluated in comparison with on-premises counterparts, is the potential for new updates to break business critical features/functions. We can call this “interruption of service”. If you are leveraging features that are critical to patient safety and product quality, you may assess interruption of services tied with a specific feature as a high-impact risk. If you are counting on certain features to comply with any applicable regulations, you could also rate those potential interruptions of service as high risk impacts.
Business impact is not the only variable when evaluating risk. GAMP also recommends evaluating the likelihood of an issue occurring, and also your ability to detect an issue before the undesired impact is generated.
In a cloud-based environment, the likelihood for such features to be broken by an update is generally low if you are using an out-of-the-box feature set because such updates are tested before release. However, if you built in-house system customizations, those custom features cannot be tested as part of the normal cloud release cycle and, as such, you should build risk mitigation mechanisms for business critical features.
The point here is that understanding your system critically, its key features and what interruptions of service could mean to your organization should reflect your validation strategy past your go-live.
So how do we go about validating a GxP cloud-based system?
Internalizing the continuous change process
Based on personal experience and empirical evidence, the three most frequent reasons for validated states to be compromised are 1) inadequate change control, 2) poorly developed user management and/or 3) insufficient user training processes and documentation. Of those three points, your change control approach will need to be adapted if you are moving from an on-prem to a cloud-based system, while the other two risks areas can be addressed in very similar ways.
In a cloud-based system, change control in the classical sense can be a serious challenge. In fact, if you are moving to a public cloud, you have already accepted that your system will be changed without explicit authorization, but it does not mean that those changes will occur without your knowledge or your ability to mitigate potential risks. In order to maintain your validated state, assuming you successfully validated your cloud-based system, you should embed the vendor change process into your own processes.
I would like to propose two risk-based strategies that build upon this concept in-terms of maintaining validation in this ever changing environment. One is based on an assumption that you are dealing with a low risk system, and the other is based on an assumption that you are dealing with a high risk system (in terms of service interruptions) as explained above.
Options A: Low Risk Tied with Interruption of Services (Example: Content Management Platform)
- Set your system on long release cycles with your vendor;
- Regularly monitor and analyze Cloud Service Provider information (examples: product roadmap, release notes, end user forums, etc.) to determine the risks (if any) tied with upcoming changes;
- Based on your periodic assessments, implement risk mitigations or prepare contingencies when appropriate in order to prepare for upcoming changes ahead of vendor deployment;
Option B: High Risk Tied with Interruption of Services (Example: Pharmacovigilance Platform)
- Set your production (live) system on the longest possible release cycles with your vendor;
- Regularly monitor and analyze release notes from your Cloud Service Provider and determine the risks (if any) tied with upcoming changes. This analysis should identify tests that should be re-executed and/or modified due to the upcoming changes;
- Maintain a separate environment for change evaluation that is part of an earlier release cycle;
- Periodically run automated regression/recovery scripts covering at least the highest risk items;
- Formally investigate and address any issues identified as needed.
Regardless of the approach you take, the following basic rules should be followed:
- Your change/issue management process should integrate changes from cloud service provider (CSP) as an expected and desirable operating event
- The continuous validation strategy you select should be justified in some form of risk assessment document, especially if you are going for a low risk approach for a GxP system
- Your change management process should support a complete feedback loop that keeps you connected to your CSP and integrates their communications
- Your change management process should incorporate a CSP feedback mechanism to quickly communicate any issues identified on your end
When evaluating risks about changes, it is also important to consider the type of cloud services you are using. As an example, changes for IaaS based systems will likely have a much lower impact than those of PaaS or SaaS, since the applications are kept under complete control of the cloud service consumer.
The Takeaway
If you follow this strategy and you document your analysis, your decisions, and your actions appropriately, you should be able to accommodate your cloud-based changes without sacrificing your validated state.
The validated state is not a thing of the past now that we are dealing with cloud based technologies, however, it finally has reason to be a front of mind, evolving, and regularly evaluated item, something that we call can agree is a smart process moving forward.
