Facility monitoring systems validation-A practical approach

09/01/2001

Validation

by Mike Gough

Just having an understanding of hardware and software systems is not enough. It's time to fully understand the process.

Validation is a word that most people within the pharmaceutical industry know and in general have a good understanding of the terminology (IQ, OQ etc.). Computer systems validation, however, is often regarded as a black art to be avoided unless you are well versed in the relevant spell books.

The problem, of course, is that computers are everywhere; formulations, stock control, integrated manufacturing, vision inspection systems, laboratory analysis, environmental monitoring systems and so the list goes on. They are even in places were you don't expect them, chart recorders, temperature controllers, etc.

To validate a system, just having an understanding of computers and software systems is not adequate. It is essential to fully understand the process and the equipment that is being validated. The people who have the best understanding of the equipment are the manufacturers, but often they know little or nothing about validation.

It is not possible in this article to fully detail all aspects of computer systems validation or the facility monitoring systems (FMS) area. For more information on validation please refer to the list at the end of this article, and for FMSs, contact individual manufacturers for additional information.

Figure 1. A typical facility monitoring system.Click here to enlarge image

Facility monitoring systems are normally only used for cleanrooms and the associated areas. Such systems cannot be used to classify an area or facility. They only perform a monitoring function to at least provide evidence that the area's environmental conditions have been maintained within the required condition. The U.S. Food and Drug Administration (FDA) and other regulatory bodies do accept that if you are using a facility monitoring system, the period of reclassification can be extended (ISO 14644-2).

Typically a facility monitoring system comprises several monitoring devices: temperature transducers, humidity transducers, pressure transducers, perhaps velocity transducers and particle counters (see Figure 1). They may also have digital signals to monitor vacuum pumps and machine running states and have digital outputs for alarm lights, sirens and other control functions. Applications and user requirements vary greatly.

Figure 2. The life cycle of a facility monitoring system. This life cycle model is also referred to as a "V"-model, because the specification flow resembles the letter V.Click here to enlarge image

The most important document of any proposed system is a user requirement specification. Without this document it is not possible to validate a system. This may come as a surprise, but validation is the process of generating documented evidence to provide a high degree of assurance that a system will consistently fulfill its stated function. The user requirement specification states the required functions of the system. A user requirement specification does not have to be a lengthy document, but it must state what functions the system is to fulfill. Although the document is called a user requirement specification, the user does not have to generate the document, the supplier can, but the user must authorize it. The purpose of the user requirement specification is to ensure that both the user and the supplier understand what is required. The document should have a list of "must haves", "want to haves" and "nice to haves". The supplier should fulfill all the "musts," but not necessarily the "wants" and "nice to haves."

From the user requirement specification all other validation documents and stages then fall in place and are normally shown in the form of a V-model (see Figure 2).

These documents and processes form part of the life cycle of the system and are also referred to as the life cycle documents.

This is not the normal or even extended V-plan as detailed within Good Automated Manufacturing Practices, but this is a practical guide. This may also look like a lot of documents and a lot of additional work, but how much does a project cost if it goes wrong? How much does an audit failure cost? The cost isn't just one of money, it affects reputation and causes major rescheduling problems.

After the user requirement specification, the next step is for the supplier to generate a functional specification. This document must address all the users' "must haves", "want to haves" and "nice to haves" and should be generated before order placement. If some requirements cannot be met (as inevitably there are), the non-compliances must be listed within the functional specification. Quite often the requirement is fulfilled in a different way, or the requirement is not essential. The functional specification should also have a cross-reference matrix so that the user can easily see what the supplier's proposal is to fulfill each requirement.

In addition to producing the functional specification, a quality plan or master validation plan should be generated at the same time to state how the project is to be controlled, the people responsible and time scales of each of the project stages (this document should be a requirement of the user).

I have shown design qualification linked to the user requirement specification, functional specification and design specifications. It is essential to check that all items listed within the preceding document have been addressed (not fulfilled, but addressed). Design qualification prevents missing a requirement.

Once the system has been fully specified and agreed to in the functional specification, the design of the system must be specified within an overall design specification. This specification should be a top-level document that clearly states what items are required and how mechanical and software items are to be connected together to fulfill the function specification requirements.

Next are module specifications. A module may be a control panel or a software program, it makes no difference, if something has to be built (or programmed) the requirements of the module must be clearly defined.

The final action of the first part of the V-model is to actually build the panels, order the particle counters and associated equipment and write and configure the software. But as the V-model shows, this is only 50 percent of the project. Once all hardware has been built or delivered to the supplier's site with all the software written and configured it has to be connected and tested to ensure it all works on the supplier's site. We have now moved into the testing phase.

Testing
Module tests must be drawn up against the module specifications to ensure each module functions as stated. In the case of a panel, has all the equipment been installed, has it been wired correctly, is it to the relevant standards, etc.? With software, it is common for the module specification to be wider than a user requires. Having software modules fulfilling more than a user requires enables the module to be reused on other projects. Module tests verify all functions have been completed and work and must include stress testing of the software to ensure that normal error conditions have been correctly handled.

After completion of module tests, the system must then be brought together, with another set of tests applied to the completed system to ensure that it functions as detailed within the overall design specification. Some companies do this on site, but if there is a problem they have the added costs of staff working on site, and typically the design engineers are not on site. Another problem with doing this on site is that any problems are very obvious to the user. Validation is about building assurance.

Users may state that they wish to perform a factory acceptance test. This is very common with delivery of machinery, but not so common with facility monitoring systems. The purpose of a factory acceptance test is for the user to be able to see how the system functions before allowing it to be delivered to site (it is also normally a payment stage). The simplest way of performing a factory acceptance test is to use the systems installation qualification (IQ), operational qualification (OQ) and performance qualification (PQ) documents and to state on the tests that simulators have been used as applicable or that the test is not possible. Again if there are any test failures, it is far simpler (and less expensive) to solve problems on the supplier's site than on the user's.

After either SQ or factory acceptance test the system is then shipped to site, installed and commissioned. Once commissioned I strongly recommend that user training is conducted before any final validation tests (IQ, OQ, PQ) for two reasons: (i) Inevitably there will be minor differences between what has been asked for in the user requirement specification and how the operators use the system. Performing training before IQ, OQ, PQ allows these small differences to be addressed; (ii) Within IQ there must be a test to check that users have been trained.

From this moment on any change to the system must be very carefully considered. Change control must be applied.

Formality not required
The supplier's module tests and SQ tests can be quite informal; by this I mean that the tests do not have to be proscriptive. This allows the engineers to really test the system without spending an excessive amount of time generating test documents. However the purpose of the test must be clear and there must be documented evidence that the test has been conducted. During IQ, OQ and PQ the tests must be very proscriptive to enable a user to perform the tests. The test evidence should not be a simple check box; physical evidence of the test should be supplied (print outs, screen shots, photos, etc.) Primarily it is this test evidence that will be presented to a UK Medicines Control Agency (MCA) or US FDA inspector to show that the function performed as required. Suppliers are responsible for ensuring that they provide the user with sufficient documented evidence to pass an inspection.

Tests within IQ vary. IQ tests should test that all the items listed in the functional specification have been delivered and that they are of the correct type. The functional specification may have stated that a differential pressure transducer 0-100 Pa is to be used with 1- percent accuracy. Has it? It should check that all hardware installed on site is as stated (or better). It should check that all support systems are in place (user manuals, technical manuals, system diagrams etc.). It must check that all instruments have been calibrated and the calibration is still valid. It should also check that the software system discs (CDs) are available and have been filed. A footprint of the software should also be taken at this point (system files, dates and sizes). A system must be able to be re-validated at a later date. IQ tests to ensure all inputs/outputs of a data collection unit are operational and all switches, lights and transducers are functional etc. also need to be conducted. These tests can be regarded as operational tests, and some companies include them in the OQ test document.

Operational qualification
Once it has been documented that the system has been supplied as detailed within the functional specification, OQ can commence. The purpose of OQ is to verify that the system functions (operates) as stated within the functional specification. There must be clear tests that show data is collected correctly and manipulations are correctly applied, data is stored correctly and can be retrieved correctly (I will address 21 CRF part 11 issues later).

The final part of testing is PQ. Normally PQ tests are designed to ensure that the machine operates at the correct rates with the user's product. With facility monitoring systems this does not apply, and in many cases PQ is not performed. With sites that previously used manual methods to monitor the facility (portable particle counters and other instruments) a comparison of the portable particle counters and the FMS particle counters is performed. This can quite often cause test failures due to differences between particle counter optics, electronics and testing methods. Contact the manufacturers of the particle counters for guidance.

Other documents should also be generated for a full validation, including a project completion report or validation review report. This document should be a summary of the validation of the project, it should list all the documents that have been generated along with the outcome of the tests and should have a clear statement of the system's suitability for use.

Supplier audits are also common for the user to perform. This should be conducted prior to order placement to ensure the supplier has good quality systems and is capable of supplying the required system.

21 CFR part 11 regulation by the FDA relates to electronic signatures and electronic records. Most facility monitoring software systems do not use electronic signatures to approve batch release information. All facility monitoring software systems hold electronic data; the regulation applies. In general facility monitoring systems are closed systems-access to the system/system data is restricted by a control method. In this case providing electronic signatures is not applicable. Subpart B paragraph 11.10 is applicable and states that the following must be addressed:

• The system must be validated.
• The system must be able to generate accurate and complete copies of data.
• Data records must be protected.
• System access must be restricted.
• Audit trails must be applied to all data.
• Operating system protection methods should be used where possible.
• Authorization checks must be applied.
• Data input/collection verification must be applied.
• Users must be trained.
• Standard Operating Procedures for the use of the system must be in place.
• Documents must be controlled.
• Change control must be applied.

I have deliberately simplified the above list from the regulation. Tests should be included within IQ and OQ to establish that the system does fulfil the above requirements. Many should already be part of any test, but some are outside the scope of a supplier.

There are some simple methods to make the above process easier and to ensure that a system passes a validation.

• Ensure all transducers, particle counters etc. are suitable for the purpose and have valid calibration certificates.
• Don't use complex data collection devices (PLCs) if they are not required. If a PLC is used, it will have a program. More validation is required.
• When designing a system consider how things can be tested and how documented evidence can be generated.
• Consider 21 CFR part 11 requirements and how these will be fulfilled.
• Use qualified people and reputable suppliers.
• Follow the steps listed within the V-model diagram (Figure 2).

Mike Gough is the systems specialist for Pacific Scientific Instruments and is based in Europe. He has run his own software company and developed his own facility monitoring software system with more than 80 installed systems around the world, the majority being in pharmaceutical facilities. He now works for Pacific Scientific Instruments (a particle counter manufacturer), consulting, commissioning and validating Facility Monitoring Systems throughout Europe and the world. Mike can be contacted at [email protected].

____________________________

For further information please refer to:

• GAMP, Good Automated Manufacturing Practices Guide
• PDA Validation guide
• FDA 21 CFR part 11