Implementing an automated system for environmental monitoring
08/01/2001
Management
by Jeanne E. Moldenhauer and Victoria Galliani
THE SHEER VOLUME OF DATA GENERATED AS PART OF AN ENVIRONMENTAL MONITORING PROGRAM FOR PHARMACEUTICAL PRODUCTS REQUIRES AN AUTOMATED SYSTEM FOR DATA STORAGE AND REVIEW.
There is very little written concerning why the regulatory bodies have so dramatically increased their attention in these areas. There has not been a significant increase in products deemed non-sterile due to environmental contamination.
In fact, there is no direct correlation between the level of environmental contamination and the risk to the product. Regardless of why the increased focus, it is important to have a meaningful, manageable and defendable program for environmental monitoring that will withstand the rigors of inspection by regulatory bodies. One significant problem with the design of this type of system is that a large volume of data is generated.
 Survey worksheet. |
Screen representations supplied courtesy of Compliance Software Solutions Corp.
The data is only useful if it can be trended, reviewed, reported and assessed in a timely manner. Finding out during a regulatory inspection that the environmental monitoring data exhibits a trend is not useful. It is important to know that a trend is occurring when it happens.
 Test site maintenance. |
What is an environmental monitoring system?
When people think of an environmental monitoring system, they usually think of the collection of microbiological and particle data from the air, surfaces and personnel in cleanroom environments. In some cases, they may also recognize that the data storage and retrieval is part of the environmental monitoring system. In fact, a comprehensive environmental monitoring system3 comprises many parts:
- Regulations/industry guidelines/compendia requirements
- Policies and standard procedures
- Utilities
- Lab methods and supplies
- Cleaning and disinfection systems
- The systems being monitored, e.g., air, water, compressed gases
- The limits/levels or control parameters set
- Room classifications
- Frequency of testing
- Selection of sample sites
- Personnel and training
- Culture media
- Lab equipment
- Documentation generated and required
- Approach to data management
- Data collection and analysis tools
- Change control systems
- Data review
- Data interpretations
- Investigations/deviations/corrective actions
- Reports generated
In considering automation of an environmental monitoring system, it is important to assess what part(s) of the system will be automated and how automation will affect the other. One of the most commonly found problems in automating a system is that people look at only one component, e.g., data analysis, when they want to automate.
Considering any form of automation necessitates a conscious decision on what parts of the system are to be automated.
Approach used
In considering the switch to an automated system for environmental monitoring, an approach was defined for how the decision to automate and the potential automation would occur.
- Evaluate/assess the existing system
- Evaluate opportunities for automation
- Evaluate vendors
- Select software
- Install the software
- Validate the software
- Implement the system
 Survey data update. |
System assessment
The nature of the company and the areas where trade is conducted impact the regulations, guidelines and/or compendia that must be followed. Because the company in question had products that were sold in Europe and the United States, it was determined that Fed-Std-209E, USP <1116>, certain EU documents and ISO 14644-1 would be followed.
 Room area maintenance. |
The PDA published a draft technical report on Environmental Monitoring. Within this document a list of considerations for an effective environmental monitoring program and what should be documented were presented. They include:
- Date and time of test
- Test method/procedure reference
- Activity level at site during test
- Equipment identification
- Location
- Area classification
- Schematics of area showing sample site locations
- Sample sites (critical and non-critical)
- Test results
- Evaluator of results
- Date the results were read
- Alert and/or action levels
- Temperature and time of incubation
- Control test results
- Certification, validation and expiration date of media
- Characterization of contaminants
- Name of reviewer
- Reporting of data
- Review of historical data
- Change control system
- Calibration cate on instrumentation
- Methodology, analysis used to specify action/alert level
- System for documenting investigative/corrective action
- Description of deficiency
- Possible cause(s) of problem
- Identification of person responsible for relevant corrective action
- Description of action steps and their schedule for implementation
- Evaluation of effectiveness of action steps
Because this list was comprehensive and represented the consensus of a large task force from more than 10 different companies, it was decided that any system should meet all of these criteria.
 Test reading update. |
It was then important to assess whether the cleaning/disinfection program was established, validated and implemented. This is an important consideration in the effectiveness of an environmental monitoring program. One should also consider whether all sample sites have been selected and justified. An adjunct to this is whether there is data available to justify the sampling frequency used and how it was selected.
Alert and action levels were reviewed to ensure that they had been established and were reasonable. The approach used to establish the levels was also reviewed. Because a statistical approach was used it was considered acceptable.
 Test reading update. |
Data management was reviewed to understand the data collection methods, how the data is analyzed, which approach is used to interpret the data and how it is interpreted. This review also entailed understanding whether isolates are characterized and what methods are utilized. Review of the data leads to review of investigations and corrective actions. It is important to know how they are handled and processed.
The bottom line question is does the system work? This system was being installed initially at a start-up facility and all of the systems were established and working. They had recently been validated and were considered acceptable.
 Time temperature incub. |
Evaluate opportunities for automation
In understanding the system and the desired improvements, it is important to consider where there are valid opportunities to automate the system. Examples of this type of consideration are:
- Is the system intended to accommodate going paperless?
- What is the impact of 21 CFR Part 11 on that decision?
- Will the system be integrated into a LIMS system? If so, to what degree will the system be integrated?
- Will the system generate all the necessary sample labels? If so, will they be in a human readable form? Is there an automated system to re-enter the data, e.g., bar codes?
- How will deviations be handled?
- What types of trending capabilities are necessary? What types are desired?
- What opportunities are there for equipment integration, e.g., direct data transfer from particle counters, air samplers, etc.?
- How much or how little will be automated? Are spreadsheets sufficient? Do you intend to design and develop a system from scratch? If there is an off-the-shelf program available, is there sufficient customization available to meet your needs?
Manual data collection and/or utilization of spreadsheets
Although these systems may appear to be the most cost effective, looks may be deceiving. These types of systems require a significant amount of intervention.
 Control test results. |
Frequently data is difficult to manage. In most cases, these types of programs have little ability to detect trends or excursions, other than exceeding a specific limit or level. When this happens, the excursion might not be investigated or reported to the appropriate personnel in a timely fashion. In general, there are time delays encountered in trending the data, e.g., setting up the requirements for the report, making it fit on the page, ensuring that the correct calculations are used, etc. If the report is not generated routinely, but has been requested by FDA during an inspection, it may be a lengthy process.
Most of these systems are not Part 11 compliant and end up with copious amounts of paper records. There are "hidden" expenses associated with the number of person-hours required to collect, record and manage records.
Another inherent problem is typing errors. Who will review and verify that all entries are correct?
 Media. |
LIMS systems
Although there are many great features of LIMS systems, in a variety of price ranges, there are few LIMS systems designed to manage environmental monitoring data. It is not uncommon to take data from a LIMS system and rearrange it using another spreadsheet or database to yield the desired reports. For those systems that do have environmental monitoring modules available, the system integration is often expensive.
If the module for environmental monitoring is purchased concurrently with the LIMS system, the time necessary for validation of the system may be increased.
Designing a custom system
There are several inherent problems in designing a system from scratch. One of the earliest is that the microbiologists who will use the system frequently do not speak the same "language" as the computer expert who is going to write the software. Allowances for other types of software and data may not be applicable to the regulatory rules for environmental monitoring.
One of the biggest problems is writing an effective requirements definition and accurately specifying needs and requirements. Another problem is the fact that the amount of flexibility you build into the system most often correlates to the increased cost/time of validation.
 Survey review. |
Because the system is designed as a custom application, there is very little room to leverage testing, as the system would not meet the definition of widespread use. The obvious advantage to a custom-designed, in-house system is that it should do and be exactly what you want. Key project individuals need to stay apprised of any changes to regulatory requirements and potential changes that may affect the system during the design and development of the software.
Off-the-shelf software, with some customization
There are some software systems that are available for use, which can be purchased commercially. The benefit is that there are typically reports and data management tools designed into the system. It is important to know and understand those features that are hard-programmed and cannot be changed versus the features that are amenable to customization.
It's important to remember when selecting purchased software that it is not the vendor's responsibility to make the software unique to your company's needs.
 Test trend report. |
It becomes very important to identify needs and wants when assessing the suitability of software for your purposes. Software designed for environmental monitoring data usually provides analysis for data excursions, has reports readily available, limits access to records, and provides a data management tool. Some are available with automatic tickler or reminder systems for testing to be performed. An audit trail and system security features are a necessity for compliance with 21 CFR Part 11.
In most cases, purchased software designed for environmental monitoring data was designed knowing that the end user is responsible for some type of validation of the software. Additionally, depending upon the number of users and how customization is achieved, it may be possible to utilize data from other sources as part of the validation.
Customization of off-the-shelf software may have some pitfalls as well. If by customization the changes are within the scope of the vendor-supported validated software these risks are reduced. However, when significant changes are made to support an individual company, the vendor may not be able to easily maintain the customized version of the software. For example, excessive customization may not be compatible with upgrades or patches. Additionally, the testing routinely performed by the vendor may not include testing for the special custom software. This in turn may limit the amount of on-line support available.
Evaluating vendors
After considering the types of automation that were desired and the resources available within the company, it was decided to look for an off-the-shelf package that could be utilized with limited customization. In the case of environmental monitoring software, there are few vendors available and cost may be a concern. Additionally, it is important to know whether the software manufacturer can meet the requirements for a GMP system. Many times, software manufacturers for the pharmaceutical and device industries may not be regulated themselves. As such, it is important to assess whether they have sufficient knowledge and understanding of the applicable regulatory requirements for users of their products. It is also necessary for them to implement GMP compliance systems internally, which might not be necessary if they sold to non-regulated industries. One way to make this assessment is to perform an audit of the software manufacturer.
Software audits
For the purposes of this case study, a decision was made to utilize the audit checklist generated by PDA.4 A committee consisting of equipment and software manufacturers, pharmaceutical company representatives and FDA personnel developed the checklist. The checklist was selected because of the comprehensive level of detail.
 Historical data. |
The audit provides an opportunity to assess the validation and testing data available at the vendor site, the procedures in place, change control, customer complaints and technical support. Another important consideration is the size and stability of the software manufacturer. Selection of a vendor who is not financially stable can have disastrous effects. Contingency plans should be established to provide for the event of company failure, e.g., what happens to the source code and any testing performed by the vendor. This becomes more important as one increases the use of data generated by the vendor to for their validation package.
Technical support available
An important consideration is the level and type of technical support available. Selection of a European vendor for an American company can work, but if the only available support is via telephone, the differences in time can result is at least a one-day delay for every question requiring technical support. The best systems should have a combination of ways to contact the vendor and/or obtain support. Some examples are: written, detailed user manuals, troubleshooting guides, on-line support, website support and troubleshooting guides, phone, fax and e-mail. Another consideration is whether there is on-site support available. This can be crucial during the initial testing and installation of the software.
Meeting of the minds
One must assess whether the software vendor has a good working knowledge of what is technically involved in environmental monitoring. For example, do they understand the tests required, how data is collected, the time delays associated with obtaining results, what types of data are required? It is also necessary that the vendor be able to communicate effectively with the microbiologists and the information services personnel at the company.
 Audit trail. |
Defining and specifying requirements
Once the end user understands what types of systems are available and possible, it is necessary to specify the exact requirements of the system for purchase or development. One thing that was learned during this exercise is that team input works better. All of the individuals affected by the requirements should be participants in the requirements definition. If the items are discussed in a meeting setting, differences should be negotiated prior to trying to validate the software.
It is also important to understand what types of computer systems and software at the plant must interface with the selected system. In this case study, all of the computers at the site were less than a year old. They were Windows NT and were installed on a validated network. The Microsoft Office 97 software package was installed on all workstations.
 Equipment calibration. |
Software selection
When all of the above have been completed, it is necessary to select the software for purchase. In this case, the EMS, environmental monitoring software system from Compliance Software Solutions5 was selected. A number of factors went into the selection process, including the types of equipment and software available, the location and support level of the vendor, the GMP compliance level of the vendor, cost of the system, ability of the system to provide for a paperless system, etc. This package also met all of the specified documentation criteria established in the PDA Technical Report. Although this package was selected for the company, the approach defined in this paper is applicable for other software systems as well.
Another decision, which was included as part of the selection process, is that only one workstation on the validated network would have the software installed until the initial validation was completed. Subsequent workstations would be added following established change control procedures.
Installation of the software
Any hardware on which software is to be installed should be qualified either prior to installation or concurrently with the installation. The degree of qualification/validation for the workstation is dependent upon procedures at the facility. The requirements for this facility included the necessity for the workstation to be connected to a validated network. Many companies do not have current network validations. Lack of appropriate validation can significantly hinder the ability to effectively validate downstream systems. Additionally, it generally causes the opportunity for other software to be added/deleted from the system without adequate change control procedures to assess the potential impact on validated systems.
Debugging, gaining familiarity
Once the software is officially installed on the workstation, some evaluation should be performed to ensure that the installation was properly executed. This phase should include any necessary debugging exercises and developing familiarity with the software. If customization is necessary, e.g., setting up sampling locations, parameter tables, etc., this should be completed prior to validation tasks. It is useful to try out interface connections during this phase. If bar code readers are being used, are they compatible? Does the printer accept data and print? Can other software required on the system work after the initial installation. Failure to do this can significantly delay the validation-testing phase.
During this time, one should also walk-through the system and ensure that all needed and specified features are available and appear to operate as desired. Validation proof of operation will be included in the protocol, preliminary evaluation and testing helps minimize repetition of validation testing.
Considerations for validation testing
Prior to initiating testing under the validation protocol, one might consider how testing data will be documented, e.g., is it possible to set up a facility or line to be used for validation testing data and can the data be stored and used to assess future changes or requalification of the system? This is useful for change control purposes, e.g., a board fails and is replaced in the printer. Having a testing database allows one to repeat tests with minimal additional effort. This selected software had an option for multiple facilities or plants. Accordingly, a separate "test" facility was selected for entry of testing data. Subsequent qualifications and/or equipment changes, operating system updates and the like would be subjected to re-execution and reprinting of the test data reports to show that all the systems previously tested were or were not affected.
Although the installation procedures vary depending upon the software selected, one must consider what other software is installed on the workstation and how it may affect or be affected by the software being installed. Ten to 20 microbiologists used the selected workstation to check e-mail and to use as a back-up site for word processing activities.
Validation
Generally, a system as complex as an environmental monitoring system requires an installation, operational and performance qualification protocol. For some companies, multiple phases of this testing are combined into one document. The general structure of the testing is as follows.
- Installation testing. This phase of testing should include appropriate checks to verify correct and complete installation of the system on the hardware. It is important to verify that any special vendor requirements for installation have been met. A sample template of this type of testing for the selected system is included with the EMSS.
- Operational qualification. This phase of the testing verifies that the system operates as intended. It is critical to verify that "alarm" conditions also respond as expected. Because the software vendor had a complete package of alarm testing, it was decided to retest 10 percent of the test conditions. The acceptance criteria were set such that if the results of the vendor's testing could not be reproduced, additional testing would be performed. Another way to approach this testing is to use failure mode and effects analysis to evaluate the severity of a failure at each step, assign a numerical value and test those features that exceed a specified value. It is important during this phase to test the normal operation of the system. A sample template of this type of testing for the selected system is included with the EMSS.
- Performance qualification. For many software applications, a performance qualification test is not conducted, as testing of the final associated hardware also assesses the performance of the software. For this type of system, testing was deemed necessary, because of the interfaces to the end user network, e-mail system and other applications available for use on the workstation. Additionally, each end user needs to assess the impact of their standard procedures, policies and methods on the environmental monitoring system.
 Test type maintenance. |
Lessons learned—an end user perspective
As with most software applications, the more flexible a program, the longer amount of time required to validate the system. When the system is purchased off-the-shelf, it probably has been designed to accommodate the needs of a large number of users. As an end user, it becomes prudent to assess whether all features of the system are necessary. Because time and cost of implementation are important for most end users, this type of assessment can reduce the final testing time. If most of the reporting features and options are not going to be utilized, one might choose to do less testing on the reports that won't be used or may choose to block them from use, for example.
Off-the-shelf software necessitates compromise from the end user. It becomes critical to assess whether the compromise is just an inconvenience or does it pose a significant risk to the user? For example, failure to meet the requirements for good documentation practices would be an area that the user would not want to compromise, but they may be willing to compromise on the order of required data inside of a report.
 Application settings. |
There needs to be a meeting of the minds between the vendor and the end user. This includes having individuals that communicate requirements and problems in a language that both sides can understand. In this regard, we found that 1) How management thinks a systems works is rarely how the bench level scientist thinks it works; 2) It will probably take longer than you think to come to consensus; 3) Defining requirements should include the bench-level and testing personnel.
Lessons learned—a vendor perspective
It is important for any vendor to understand and be able to support their customer's needs. Additionally, the product must meet all regulatory and/or industry guidelines.
 Create survey. |
Software programs should minimize the possibility of field entry errors (e.g. spelling), as well as improve the overall efficiency of the system being automated. The use of drop-down menus to input information into text fields (e.g. technician names, equipment identifications, media lots, organism morphology) virtually eliminates the potential for spelling errors. When trending microorganisms, selecting genus and species from a predefined list ensures that names are always spelled correctly, and nomenclature is consistent throughout all the records (e.g. Bacillus subtilis or B. subtilis). Further, the ability to automatically generate barcode labels for each test sample taken will greatly reduce the potential for mislabeling, as well as reduce the time spent to manually create labels.
 Initiate test. |
An effective software program should automatically sequence through fields to ensure that all data is recorded and aberrant conditions are not ignored. The user should not be able to close a record until all fields (data) have been completed. When an alert or action level is exceeded, the system should have the capability to immediately notify the appropriate personnel that an excursion has been detected, so that it can be documented and reacted to in a timely manner. A mechanism to automatically "trigger" the investigation and subsequent closure of a deviation notification is also desirable. Facilities concerned with specific organisms would clearly benefit from an ability to automatically be notified when an objectionable organism is identified during routine monitoring.
 Bar code labels. |
Vendors of automated systems should work closely with industry leaders, and be active members of industry associations. They should also be able to upgrade and maintain their systems in order to keep up with the changing regulatory, industry and technology environments. These commitments ensure that a vendor's software programs keep current with its customer's needs and requirements. Customers should be encouraged to provide recommendations for enhancements to the software, and vendors should be equally receptive to evaluating those suggestions for possible inclusion into future versions. Requests for "customized" features are easier suggested, than implemented. Should a customer's request become a new feature of the vendor's program, it may not be valuable to all users of the software. Vendors must therefore be able to implement these upgrades as optional features so that users should only have to use them if they see value in the enhancement.
In conclusion, developing a good working relationship with customers, and ensuring that support is available at the required times will be key to the success of automating your systems, and building a mutually beneficial business relationship.
References
- IAPS (CFPA) cGMP and Quality Issues for Biopharmaceuticals, June 2000, San Mateo, California.
- Personal communication from Anne Marie Dixon, US Representative for this ISO Group.
- PDA Technical Report 13 Rewrite: Environmental Monitoring, Draft 29
- PDA Technical Report 32: Auditing of Computer Software Vendors...
- Information on this system is available at www.csoftsol.com.
Jeanne Moldenhauer is a senior quality assurance
egulatory affairs professional at Vectech Pharmaceutical Consultants (Farmington Hills, MI). She has an extensive background in the development and management of a variety of sterilization and validation processes in the healthcare industry. She has proven track record of successful NDA, sNDA, AADA, and DMF submissions to FDA, with special expertise in the rehabilitation of companies with negative FDA findings, restoring them to compliance. Additionally, she has substantial experience in assessing and validating laboratory and production facilities in need of solutions for regulatory purposes.
Victoria Galliani is vice president of quality systems and regulatory at Compliance Software Solutions Corp. (Vernon Hills, IL). She has over 20 years of experience in the health care industry. She held several supervisory and management positions in the microbiology laboratory during 12 years at Baxter Healthcare. Her experience includes regulatory affairs, quality systems and documentation management. Most recently she was director of technical services for a sterility assurance company. She holds a bachelor's degree in biology from Millikin University.
Background
Since the issuance of the FDA Guideline on Aseptic Processing in 1987 there has been an increased focus on environmental monitoring as it relates to the potential contamination of pharmaceutical products.
In 1994, the FDA issued a guidance document relating to the sterilization validation data that must be submitted with drug product applications. For aseptically filled drugs, this document listed the types of monitoring data that must be submitted for the product to be approved. It also had references to data being collected for terminally sterilized drug products. Regulatory agencies worldwide have increased their attention on environmental monitoring during drug inspections.
This is evident by the ever-increasing number of FD-483 observations and/or warning letters being issued on this topic.
A recent IAPS presentation on cGMP and Quality Issues for Biopharmaceuticals1 reported that environmental monitoring accounted for 21 percent of all 483 observations in 1998. In 1999, monitoring accounted for 14 percent of 483 observations and 34 warning letters. The concern is not restricted to sterile drugs. Firms have been cited for lack of environmental control for non-sterile drugs as well. ISO documents that are in the final approval process have incorporated requirements similar to ISO Class 8 (Class 100,000) for non-sterile drug production (ISO 14644, proposed).2
Manual systems for data tracking and trending have inherent time delays in assessing the data. The time delay increases with the quantity of data generated. Some companies use a spreadsheet or database applications to store this data. Other companies develop internal custom software to meet this need. Still other companies use their LIMS system software to store this data. Few existing systems have been designed to specifically handle and store environmental monitoring data.
Frequently data is difficult to manage. In most cases, these types of programs have little ability to detect trends or excursions, other than exceeding a specific limit or level.
It is important to know and understand those features that are hard-programmed and cannot be changed versus the features that are amenable to customization.