Validation of an Enterprise Resource Planning (ERP) System: An SAP Case Study Jackelyn Rodriguez Medtronic MiniMed Inc.
❖ MiniMed selected the SAP R/3 edtronic MiniMed is the enterprise software package, Version ❝…[Enterprise world’s leading provider 4.6 C, to replace its current legacy of external programmaResource system. In this phase, the R/3 system ble insulin pumps and continuous affected the following functional Planning] ERP glucose monitoring systems. We areas: planning, scheduling, purchasare a multi-billion dollar corporapackages enable ing, manufacturing operations (plantion which employs approximately ning and production management 2,000 people at the Northridge, an organization only), document control (bill of California, facility. There are apto truly function materials and routings only), inventoproximately 1,600 computer users ry control, inventory accounting, cost that were affected by the implemen- as an integrated accounting, general ledger, fixed tation of a new Enterprise Resource organization. ❞ assets, accounts payable, sales, and Planning (ERP) system, which distribution. Selected modules are needed to be blueprinted, configured, installed, tested, and validated in less than 12 included in Figure 1. SAP is the fourth largest software manufacturer months. Needless to say, this was not an easy task. We needed to purchase an ERPsystem software ap- in the world. SAPis modular software, e.g., Sales and Distribution (SD), Materials Management (MM). plication package that is a suite of pre-engineered, ready to implement integrated application modules catering to These modules share common data to reduce data entry all the business functions of an enterprise, (which in- activities, and increase information accuracy for buscluded the flexibility for configuring/customizing the iness functions. Since this new ERPsystem tracks the movement and functionality of the package to suit specific requirements of the enterprise) that would allow our users to inventory of a medical device product from its receipt to electronically unite the MiniMed community by replac- placement in MiniMed’s distribution centers, and could ing the company’s current software system with a new eventually develop the functionality for electronic batch records, the current Good Manufacturing Practice state-of-the-art system known as System Application and Products (SAP). This new ERP system was Regulations (cGMP) require that the R/3 system be valdesigned to deliver state-of-the-art business applica- idated. These functions are regulated by the cGMP tions that would enable well-controlled growth through requirements defined in the Code of Federal Rega consistent, real-time source of integrated information, ulations (CFR), Title 21, Parts 820, 11, 210, and 211. and ensure timely and accurate business decisions A detailed cGMP impact assessment for the enterprise project needed to be completed. scaled to a billion dollar plus company.
M
May 2003 • Volume 9, Number 3
205
Jackelyn Rodriguez
Figure 1
Selection Modules Module Human Resources (HR) Material Management (MM)
Sales and Distribution (SD) Production Planning (PP) Financial Accounting/Asset Accounting (FI)
Types of Quality Records Training records, certifications Bills of materials, materials specifications, production work orders Customer orders, shipping records Forecast plans, purchasing documents Accounts receivable and accounts payable
Project Team Selection and Responsibilities At the beginning of the project, it was quite critical that certain responsibilities and functions be assigned in order to achieve deliverables in a timely fashion. A multi-functional validation team was formed to oversee and approve all software validation activities. Members included representatives from Quality Assurance (QA), Information Technology (IT), process owners, and business users. Information Technology (IT) was responsible for the following areas, including: • Delivery of computerized systems (computer hardware, operating system, and installed standard application software) • Acquiring the required license agreements • Maintaining a validated system for performing daily network back-up and restore activities. Process owners/business users were responsible for the following areas, including: • Defining the requirements for the software and computerized systems for their intended use • Documenting the Software Development Lifecycle (SDVLC) process and activities for the software and computerized system Quality Assurance QA was the project leader for the software validation process. They conducted the following activities: • Identified all activities that required validation • Generated the validation master list 206
Journal of Validation Technology
• Generated the Process Validation Plan (PVP) and Process Validation Report (PVR) for new or existing software systems developed to support the manufacturing of new products with the concurrence of QA • Reviewed and approved all business process procedure, process flow diagrams, test scripts, business process procedures master list (traceability matrix) • Witnessed all test scripts executions • Confirmed that all test script deviations were addressed and closed. • Managed and maintained the change control process Development Methodology Through the use of the SAP product, Accelerated SAP (ASAP) project deliverables were identified, scheduled, and developed. All of the functional areas generated the same type of deliverables. Careful consideration was made to ensure that detailed documentation was maintained for the cGMP governed areas. Through change control, and preventing unauthorized or undocumented modifications to the system, it was maintained in a validated state. This system was selected not only for business reasons, but also because of the ASAP accelerated methodology. Our enterprise validation project plan defined the scope of the enterprise project validation effort. This document was also used to establish an approach to address validation requirements in order to ensure compliance of the completed system with FDAand internal guidelines. The validation project plan defined why the validation occurred, described the validation approach, specified the required steps and associated deliverables, and defined responsibilities for performing the validation steps. The objectives of the validation project plan were to formally define and document the intended operation of the R/3 system – enterprise project by: • Identifying the documents required to support the validation of the R/3 system • Defining the strategy for verifying that the R/3 system – Enterprise Project performed in a reliable and reproducible manner through qualification testing, based on the requirements specification documents, including the documentation and
Jackelyn Rodriguez
testing of system components, subsystems, interfaces, and data conversion programs defined in the design specification documents • Identifying the need for creating/updating Standard Operating Procedures (SOPs), desktop procedures, and manuals to reflect the actual use of the system • Identifying the need for creating and implementing a training program for all users of the R/3 system enterprise project • Defining the strategy for creating a file of evidence that demonstrated managerial control of the R/3 system (through the use of our change control procedure). There are five major phases needed to implement this type of software according to what is called “The SAP methodology.” Phase I: Project Preparation – Project Plan and Scope Definition The primary focus of Phase I was placed on getting the project started, identifying team members, and developing a high-level plan. The kickoff meeting was extremely important – it was then that the project team and process owners got a clear sense of what their responsibilities were to be throughout the project. Phase I included: • Training the ASAP principles • Identifying key deliverables • Defining the project team and roles • Development of a high-level project plan • Technical requirements planning Phase II: Business Blueprint The primary focus of Phase II was placed on understanding the business goals of the company, and determining the business requirements needed to support those goals. Phase II included: • Organizational structure • Signed off business blueprint (requirements document) • Generation of a business process master list • Enterprise scope document • Baseline scope document Phase III: Realization The purpose of Phase III was to develop a “futurestate” model into an integrated and documented solu-
tion that would fulfill our business process requirements. Phase III included: • Baseline configuration • Company structure and business rule definition • Business scenarios • Confirmation and approval • Final configuration The configuration of each core business process was divided into iterations or cycles of related, “business process flows.” The business process flows were configured in parallel with the development of reports, user procedures, testing scenarios, and security profiles. The cycles not only provided milestones for the project team, but also provided checkpoints to test a “playback” of specific parts of the overall business process. This approach provided immediate feedback, and involved the entire organization throughout the project lifecycle. At this point, our team started looking at the process procedures, and began their training. Additional key deliverables for this phase included: • Developing interfaces, reports, and conversions • Business process playback and approval • Rapid configuration and testing of core business processes • Authorizations setup • Establishing the production environment • Establish the design, develop and test interfaces, reports, and conversions • Develop integrated and documented solutions through cycles • Authorization and system administration Phase IV: Final Preparation The primary focus of Phase IV was placed on completing the final system testing, training end-users, and cutting over both the data and the system to a production environment. Final system testing included conversion procedures and programs, testing interface programs, conducting volume and stress testing, and final user acceptance testing. (User acceptance tests were included in the final system testing.) Another focus of this phase was developing a GoLive strategy. This plan specifically identified the data conversion strategy, initial audit procedures, and a project team support structure. The final step was to May 2003 • Volume 9, Number 3
207
Jackelyn Rodriguez
approve the system and the readiness of the company to go live, and to officially turn on the new ERP system, and to fully train end-users. Phase IV included the following items: • Cut-over plan • End-user training • Integration, volume, and stress testing • Go-live strategy (The existing ERP system was maintained “alive” during this period in the event that cut-over activities failed) • Establish internal help desk • Cut-over to production environment • Performance Qualification (PQ) Phase V: Go Live and Support After “go-live,” our team focused on supporting the end-users. Each day, the business results and system performance were measured and reviewed per PQ requirements. (PQ requirements included the verification of performance against intended use), and training continued for all levels. Business process procedures, preformatted generic process word-templates generated via ASAP, were used to provide detailed instructions to the user on how to execute transactions within SAP.
Development, Implementation and System Configuration Requirements A documented software requirements specification provided a baseline for both validation and verification. The software validation process cannot be completed without an established software requirements specification (Ref: 21 CFR 820.3[z] and [aa] and 820.30[f] and [g]). (http://www.fda.gov/cdrh/comp/guidance/938.pdf) Typically, ERP packages enable an organization to truly function as an integrated organization. This includes integration across all functions or segments of the traditional functions, such as sales order, production, inventory, purchasing, quality management, measurements systems, etc. Because these are typically included in the original requirements for the SAP ERP package, the only real software requirements were developed for custom codes. Our new ERP system handles all types of business functions, and links their related business tasks. With its exten208
Journal of Validation Technology
sive functionality and high-level of integration, this system meets a full range of business requirements, including financial accounting and controlling, sales and distribution, materials management, production planning, and human resources. Each type of business requirement was broken into what SAP refers to as a business process module. The system can be implemented with as little as one module, or a combination of modules, and still provide the integrated business functionality. We decided on a phased approach to implementation, implementing several modules in each phase. System Environment The new ERP system is based on client-server architecture. The basic system environment will remain constant over its life span at the company. It consists of three separate physical environments, or in SAP terminology, instances; a development instance, a test instance, and a production instance. Each instance contains at least one client providing a unique access into the ERP application and its corresponding database. Development Instance The development instance contained separate clients providing individual environments for the functional leads, configurators, data conversion team, security team, and the other ERPimplementation consultants. (Participants in the project included consultants, as well as internal staff because of manpower issues.) The purpose of the development instance was to provide a safe and separate environment where development activities (configuration, data conversion, interface programs, and security profiles) could take place. The separate clients on the development instance further ensure that individual development activities affect only the area being developed. Consultants configured a “baseline” R/3 system representing the business processes identified in the business blueprint phase. Then they would playback key transactions to process owners. This playback of business processes to the project team allowed for feedback, as well as further confirmation that the requirements defined in the business blueprint were being met. To ensure all clients are virtually identical (deviations between clients were reviewed and addressed by the project team on a case-by-case basis), the
Jackelyn Rodriguez
clients were refreshed weekly to include the latest system configurations. All development activities and initial unit testing took place in the development instance. Test Instance The test instance contained several clients similar to the development instance. These clients provided separate environments for preliminary integration testing, data conversion and interface testing, and ultimately, the qualification testing required for validation. A client was available for use in the training of end-users. The purpose of the test instance was to provide an environment where only testing and training could occur. This ensured that in the event of an error during testing, the development and production work is never compromised. Production Instance The production instance contained a single client. This client was a replica of the test instance client upon which validation testing is performed. All updates to the production client were handled through change control procedures. This process involved system configuration or application of software patches in the development instance, validation testing in the test instance, and upon successful completion of testing, transporting the updates to the production instance client. (In the event that a test failed, it was permissible to allocate non critical failures to an issue-tracking system for later resolution, the change control process starts again, beginning in the development instance.) The purpose of the production instance was to support day-to-day operations. (The difference between configuration management and change control is that once the system is approved, all changes must go through the approved change control process.
tion plan was generated and approved. As the new ERP system was in the developmental stage until Phase IV, validation testing and end-user training, and some validation deliverables, did not receive final approval signatures until the system reached Phase IV and was considered “frozen.” Each deliverable, however, began development in the designated phase. All documents that were developed carried a computer-related system validation number and a version number. Once approved, all changes to deliverables were completed in accordance with the SOP, maintenance of the validated system, ongoing evaluation, and change control procedures. The originals of all validation deliverables were filed in the validation history file. They included the following items: Validation Deliverables • Project planning • Validation project plan • Vendor audit • High-level requirements specification • System requirements and design specification (blueprint) • Master list of business process procedures, • Business process procedures • Process flow diagrams • Test scripts • SOPs, user, and system manuals • Installation Qualification (IQ) and Operational Qualifications (OQ) – test instance • Installation Qualification (IQ) and PQ – production instance • Data conversion protocol – test and production instances • Training documentation • Summary reports • Certificate of validation • Change control forms and testing documentation
Validation Approach Although FDA’s guidance does not recommend the use of a specific lifecycle model, we established a software lifecycle model that was appropriate for our organization, as well as compliant with FDA’s General Principles of Software Validation Guidance document. Adetailed assessment of the SAPmethodology against the design control regulation, or the software validation guidance, was completed when the master valida-
Validation Activities Project Planning During the project planning phase, the R/3 system project and its boundaries were defined, and an implementation approach was developed. A project plan was created to outline project tasks, duration, and responsibilities. Throughout the project implementation, the project plan was updated to reflect the current state of project tasks. May 2003 • Volume 9, Number 3
209
Jackelyn Rodriguez
The validation project plan was created to outline the approach, deliverables, and responsibilities to complete validation tasks necessary for the implementation of the ERP system.
System Requirements and Design Specification (Blueprint) This phase defined general system needs required of an ERP system intending to cover the functional areas. A high-level requirements specification document was developed to outline the desired system structure, operation, and system interfaces. The high-
Vendor Audit A software quality assurance evaluation was conducted to support ongoing process and quality improvement efforts by SAP AG (i.e., the company) relative ❝Functional requirements specify to the development and support of what the system must do. They its R/3 software product. In addition, it is intended to provide docurelate to the actions that the mented evidence to directly and proactively support selected validasystem must carry out in order tion activities required of those to satisfy the fundamental SAP clients whose R/3 applications are used to perform or support proreasons for its existence… ❞ cess functions subject to regulation by the Food and Drug Administration (FDA). The audit results were satisfactory. level content of this document was intended to provide a “big picture” of what was required of the sysHigh-Level Requirements Specification tem. The development of the high-level requirements Functional requirements specify what the system specification document was the responsibility of the must do. They relate to the actions that the system project managers and the corresponding functional must carry out in order to satisfy the fundamental leads. reasons for its existence. They include: The Design Specifications (DS) were prepared to describe how the requirements specifications were • Specifications of the system’s functionality implemented. Elements addressed in the design spec• Actions the system must take-check, calculate, ification included: record, retrieve • Derived from the fundamental purpose of the • System hardware system. • Application servers • Nonfunctional requirements are the properties • Database servers that the system must have. For example, these • SAP Graphical User Interface (GUI) workstations are the characteristics or qualities that make the • System software system usable, fast, secure, or reliable. • Operating system • Global requirements are broad, encompassing re• Database quirements, or a constraint on the system. Global • Application software requirements may: • System performance • Network connectivity – Be described in a very broad language • Remote access – Describe a characteristic of the entire system • Security – Reflect a general need of all potential users • Physical security – Constrain the design or use of the system • Logical (SAP application security) • Configuration These high-level requirements are detailed fur• Application extensions ther in the system requirements specification. • Business processes 210
Journal of Validation Technology
Jackelyn Rodriguez
System Design This phase included the initial design of the system environment, a more detailed definition of requirements, and the beginning of system configuration. A detailed requirements and design specifications document was created as part of this phase. The functional leads, configurators, basis administration, and QA-Validation personnel together provided the input to complete this document. Detailed system requirements were defined for each of the functional areas. The requirements then, at a minimum, defined process descriptions and the sequence of operations for each of the functional areas, and their corresponding sub-components. The outline for the functional content was based on SAP’s definition and terminology of the make up of each of our major ERPmodules. This approach was taken to facilitate the documentation of future module upgrades within the new ERP system. System design specifications were created to clearly and completely document the new ERP system’s operating environment, interfaces, security, and functionality. Additional configurations were performed as needed to address requirements outside of the “off-theshelf software package.” Any parameters that could have an effect on the system performance were identified, including the definition of acceptable values or characteristics for the parameters. It was extremely important that the design specifications addressed all of the defined requirements. A one-to-one reference was established between the requirements and the design specifications through a traceability matrix. Traceability Matrix Based on the requirements specification, MiniMed compiled a Business Process Master List (BPML), which identified all business activities supported by the system. This list was composed in order to develop Business Process Procedures (BPPs), which described the use and function of the system required to support MiniMed’s business. Controls were established to provide traceability of requirements to system setup and configuration through to testing strategy. This was achieved as follows: • The strategy used for the Quality System Regulation (QSR) (more generally referred to as
Good Manufacturing Practice [GMP]) impacted transaction codes (t-codes) was as follows: – All t-codes to be utilized by MiniMed were reviewed for GMP impact and categorized as high, medium, or none. This document is generated from the BPML. High Impact Transaction codes (T-codes), determined to have high GMP impact, were further reviewed at the individual field level to identify those fields that had GMP impact. The fields to be reviewed were only those that would be utilized per department work instructions for Phase I implementation. The fields of t-codes with high GMP impact had negative testing performed as part of the validation. Medium Impact T-codes with medium GMP impact were tested to verify proper functionality, and the results recorded for validation purposes. No Impact T-codes with no GMPimpact were tested for proper functionality for business purposes, but the results of this testing were not required to be part of the validation package. Testing strategies and scenarios were based on the outcome of this assessment. Test scripts were pre-approved, and information contained in the BPP’s was utilized fully. The validation test effort, including GMP/QSR critical test scripts, any deviations or non-conformances, and test results were controlled by OQ protocols. All t-codes were functionally tested for business purposes. Data Conversion Data conversion programs were verified and validated as data was compared during the data conversion process verification. The verification was performed by teams of two who reviewed the same samples of data records. It is important to remember that data conversions were not part of the functionality intended for everyday use in SAP. Instead, the data conversions were processes performed by expert individuals in a highly-controlled environment. May 2003 • Volume 9, Number 3
211
Jackelyn Rodriguez
Our data conversion process usually involved the following steps:
• Statement of acceptability of the converted data.
Validation Activities Data Mapping During this exercise, the relationship between data in the legacy and SAP systems were defined at the field level. This activity resulted in a data map for each conversion effort. Data Cleansing In order to ensure the quality of the data being transferred, we examined the data undergoing conversion for issues, such as redundant records, duplicate records, referential integrity, and typographical errors. These issues were corrected on a case-by-case basis. This process often involved significant manual effort. Conversion/Migration The conversion process involved the transfer of cleansed data from the legacy system to SAP, according to the rules defined in the data map.
System Configuration The system configuration phase included the configuration of the system as defined in the high-level requirements specification and detailed requirements and design specifications documents. System administration procedures were developed for general system administration, security profiles, data retention, and change control. As individual module configuration was completed, BPPs were developed to define the necessary procedures for each of the processes (i.e., detailed description of how to execute a transaction). Refer to Figure 2. Validation Plan
Inspection/Verification After the conversion process was run, the data was inspected to ensure that it was properly converted (in accordance with business rules and the data map). The inspection process was based on approved methodology, and the results were recorded. Converted data was subjected to inspection and verification. Each record type (in SAP) was considered a separate population for the purpose of sampling plans. A single individual, using printouts from at least the legacy and SAP systems, performed an inspection of the data. The inspections were recorded by the inspector’s signature on each page of these printouts. In addition to inspection of random samples from migrated data, all conversions were subject to verification through record counts. Records were counted in the legacy and SAP systems.
Procedures for ERP System Administration Documents System Environments 3. OQ – Test Instance ERP System – (per module), (Forecast to Finish [FF] Goods, Purchase to Payment [PP], Quality Module [QM], Financial/ Collections Module [FI/CO]) Security, System Administration, and System Interface testing 4. OQ – Test Instance R/3 System Reporting – On screen and hard copy reporting verification 5. IQ – Production Instance Documents system environment 6. PQ – Production Instance Documents system performance 7. Data Conversion Protocol Documentation and verification of automatic and manual data conversions in both the test and production instances
Conversion Summary Report For each data conversion effort, a summary report was produced. The report included the following:
Installation Qualification (IQ) The IQs documented the system environment of the test instance. Hardware and software test scripts were developed. This included:
• The data map • All required printouts for data conversion • Data inspection records 212
Journal of Validation Technology
1. SOPs and System Manuals 2. IQ – Test Instance
• Master file documentation • Procedure verification • Hardware configuration and inventory verification
Jackelyn Rodriguez
• Connections and cabling • Power supply requirements verification • Disk array maintenance • Training and documentation verification Operational Qualification (OQ) The OQ focused on a complete test of the system and any of its system interfaces. This included the unit (individual per function test scripts) and integration (integration testing and functionality combined) test scripts developed by the functional process owners. (Please see attached test script example i.e., Figure 2) The OQ consisted of verifications and tests to ensure that all components of the SAP R/3 system operated properly according to system specifications, change requests (if any), and vendor documentation. The tests to be executed under this protocol were divided into the following categories: • Procedure verification • Business process procedure verification • Process Flow Diagram (PFD) verification • Validation database setup verification • General functions/Transaction verification • Personnel education and training verification All of the personnel involved in the creation of BPPs and test scripts underwent training on how to develop/create such documents. This ensured not only a greater understanding of the systems, but also gave daily users an opportunity to practice the use of the new transactions, as well as provided awareness on how their transactions affected other functional areas, and to verify system configuration. Also included were system administration test scripts written by the basis administration (consultants) team and QA-validation, covering the areas of stress/volume testing, security, audit trails, archiving functionality, and backup and recovery. (The consulting team helped generate 80% of the test scripts). Performance Qualification (PQ) The PQ monitored the new ERP system performance on the production instance, including the ongoing data transfers from existing legacy systems to the ERP system, (This was performed once the system went live).
Data Conversion Protocol A separate protocol was developed to document the data conversion methodologies, and verify data transported from the existing legacy systems to the new ERP system. In order to adequately test the new systems functionality, the data conversions occurred on both the test and production instances. The conversion for the test instance included all of the automatic data transfers, and a limited sample of manual data conversions that mimicked (at a smaller scale) the production environment. For the production instance, full automatic data conversions and full manual data conversions took place. In both instances, a single sampling plan was created for each data conversion to ensure data integrity. Typically, it is QA-Validation that is responsible for ensuring protocols are developed with the functional leads, configurators, basis administration, and basis conversion teams. (Please refer to page 215 for conversion strategy details). At the end of the system configuration phase, a dry run (informal validation testing) was conducted on the test instance using draft qualification protocols. Once the dry run was completed, appropriate changes were made to the validation deliverables, and if necessary, to the system configuration. Validation Deliverables: Validation deliverables included: • SOPs user manuals (SOPs were a validation deliverable because references to the old ERP system transactions were included within the SOPs. In order to ensure that all old references were removed, the new references to the new PFD, which included the new BPP transaction code numbers, had to be included as a deliverable). • Procedures for functional areas • Training documentation showing appropriate training materials and documentation of training for system users • Summary report summarizes the testing results of the qualification protocols • Summary report summarizes entire validation effort • Validation Certificate (a Validation Certificate is required per ASAP methodology upon completion of this phase). The certificate states that the R/3 system consistently performs according May 2003 • Volume 9, Number 3
213
Jackelyn Rodriguez
to the user’s requirements for the system • Change control forms and testing documentation. Change control documents and supporting test documents showing control of the change process, as defined in the SOP, maintenance of the validated system, ongoing evaluation, and change control procedures Validation Testing and End-User Training Validation testing and training took place during this phase. Training materials were finalized, including the development of user manuals and modifications of any SOPs affected by the implementation of the new ERP system. Training of end-users occurred prior to the start of the system implementation and ongoing support phase. The bulk of validation testing took place on a client, (i.e, the computer station, that was set-up for validation testing in the test instance). This validation client was an exact replica, with respect to system configuration, of the production client on the production instance. An IQ and two OQs (of the new system functionality and reporting) were executed on the validation client of the test instance, followed by an IQ on the production client of the production instance. Testing Testing was performed to document and confirm that the implemented system satisfied requirements and design intent. All tests were traced to requirements and business processes. The testing process included the following categories: • SAP environment verification • Workstation verification • Test system verification • Application extension verification • GMP business process verification • Security verification • Operational support verification • Production system verification • Production evaluation verification A release to the validation memorandum was approved before installation and verification were performed in the test environment. Pertinent executed installation tests were approved before operational testing occurred in the test environment. A release to 214
Journal of Validation Technology
production memorandum was also approved before installation and verifications were performed in the production environment. Pertinent, executed installation tests were approved before performance testing occurred in the production environment. Test Plans and Test Cases Our software test plans were test cases developed using the blue printing requirement, which generated what we called the BPML. This list included every single transaction code that was used as a checklist to correlate transaction codes with test scenarios. This also included the cGMP impact assessment. This list was also used for comparison to the process flow diagrams (see Attachment 1), that were also part of the blueprinting process. As a result, our software validation testing appropriately matched the risk associated with the system. “Real World” Testing FDA’s software validation guideline1 document recognizes that “software is designed, developed, validated and regulated in a real world environment.” Because of that, during testing, it was important to consider what level testing was appropriate, and if all of the “high risk” software integration scenarios were considered, tested, and challenged when designing the software validation. Levels of no-impact, medium, and low were assigned according to GMP impact. Single unit testing was performed on transactional code outside the scope of validation testing. GMP/QSR Impact and Test Script Preparation The extent of testing for any business process or transaction should be determined by the degree of GMP/QSR impact for that process. The GMP/QSR impact was recorded in the BPML. The following guidelines can be followed in the determination of extent of validation testing for any business process or transaction: GMP/Q SR impact No Impact Medium High
Validation testing No validation testing performed Verify the transaction performs in normal (ideal) situations Verify the transaction performs in normal (ideal) situations.Also verify proper operation of all system functions and controls, which have GMP/QSR impact through challenge (negative) testing.
Jackelyn Rodriguez
GMP Business Process Verification GMP business process testing was designed to ensure that the integrated business processes work in accordance with the process flows, configuration specifications, and the functional/design specifications. This verification included positive and negative tests. Positive tests challenged the normal business process. Negative tests introduced challenges to the execution of the normal business process. These challenges included, but were not limited to, entering invalid data and out-of-sequence transactions. Based on the requirements specification, we compiled a BPML that identified all business activities to be supported by the system. This list was composed in order to develop BPP’s, which described the use and function of the system required to support our business. The strategy that was used for QSR (generally referred to as GMP) impacted transaction codes (tcodes) was as follows:
and software configurators to address deviations and closure in order to complete the test script process. GMP Business Process Verification Our GMP business process testing (see Attachment 3 for an example of a test script) was designed to ensure that the integrated business processes worked in accordance with the process flows (see Attachment 1) configuration specifications, and the functional/design specifications. This verification included positive and negative tests. Positive tests challenged the normal business process. Negative tests introduced challenges to the execution of the normal business process. These challenges included, but were not limited to, entering invalid data and out-of-sequence transactions Security Verification Security testing was designed to verify that the implemented SAP security architecture functioned in accordance with security specifications. This testing was executed for the following two areas:
Business Process Master List Area
Business Process Procedure Title
Number Deficiencies FX Convent Planned Order to Production Order – Individual FX Collective Conversion of Planned Orders FX Conv.plan.ord.to prod.ord.part.redct FX Display Material Bills of Materials (BOMs) FX Display Allocation to Plant FX Multilevel Billing of Materials FX Summarized Billing of Materials
Transaction GMP Code ABCXX1
Impact Med
ABCXX2
Med
ABCXX3
Hi
ABCXX4
Med
ABCXX5
Med
ABCXX6
Hi
ABCXX7
Med
This example was created using the BPML generated as part of the ASAP implementation methodology. Each business process listed is a process that was implemented during our ERP implementation Because our software validation activities and tasks were dispersed, occurring at different locations, and were conducted by different organizations, we used QA teams to oversee all testing activities, including recording deviations, working with testers,
❶ Transaction access ❷ Integrated business scenarios Transaction access demonstrated the proper operation of the core transaction protection object, and configuration of activity groups. In this testing, the following was demonstrated for each tested activity group (roles): • Transactions required for the activity group (roles) were accessible • Transactions not required for the activity group (roles) were not accessible During security validation scenarios, the business processes were executed with users assigned to the designed profile in the SAP user master record. This process verified that the security model was correctly configured to permit the execution of SAP tasks using their assigned role. Data Conversions The data conversion protocol was executed on both the test and production instances. This was typically the responsibility of the functional leads, configurators, and basis administration and conversion May 2003 • Volume 9, Number 3
215
Jackelyn Rodriguez
teams, as well as functional area super-users to assist and execute these qualification protocols. (This included data maps, data conversion verifications, and record counts). In the event a discrepancy occurred during testing, an evaluation of the discrepancy was made as to the level of severity, and a decision was made to continue testing using change control procedures, or to halt testing until a resolution was found. The discrepancy, evaluation, and decision to proceed or halt testing was documented in the testing log of the associated qualification protocol. Upon completion of the qualification protocols, a summary report for each protocol was developed. The reports detailed the results of the qualification testing. Business Process Procedures and Test Script Training The purpose of this was to demonstrate that BPPs and test script writing was sufficiently understood, and training was performed with the end users. Training documentation for all personnel that were tasked with writing test scripts attended the briefings and workshops was required. The following table included some of the minimum training requirements: Description Overview • BPPs and Test Scripts • Testing and Validation Test Script Test Script Control Log BPPs and Test Script Workshop • Business Process Procedures (See Attachment 2) • Test Script Example (See Attachment 3) • Template • Sign In Sheets • Deviation Logs
Upon successful completion of the above training activities, test script writing commenced. System Implementation and On-Going Support During this phase, the project team needed to maintain control of the system, while allowing for a continuous improvement process. In our case, the new ERP system was maintained in a validated state through our change control process, as well as configuration management. Required revalidation was evaluated, and occurred for any new implementation 216
Journal of Validation Technology
phase, development of an interface from an existing system to the new ERP system, or validation for a new software release. In each case, the existing validation deliverables were referenced with additional deliverables created to document new features (and corrections) and their interrelationships. Processes for the development, use, operation, and maintenance of the ERPcomputer system also included: • Maintenance • Backup • Record retention • Virus protection • Startup and shutdown • Problem reporting and tracking • Performance monitoring • Transports (changes to the current system) Once the system was approved and validated, all changes underwent a review and approval process to ensure that all testing was completed. Additional functionality testing was performed whenever changes, updates, and/or upgrades needed to be made. Examples included: ❶ Basic functional testing of PROD (the production environment) after a Hotpack installation is complete ❷ A change to a transactional code configuration Specific testing requirements needed to be identified in order to install the latest kernel update. These types of tests must be conducted to ensure the system is performing as expected and data integrity is not compromised. All original documents should be filed in the Validation History File as they are completed. Once the system was online in the production environment, the PQ was executed, and the system performance was monitored for a period of no less than three weeks. Any disruptions to the system performance were documented in the testing log of the PQ. Following the specified time minimum duration, the Summary Report for the PQ was completed, which summarized the results of the initial monitoring period. A final Summary Report was then generated to summarize the results of all validation activities for the new ERP system. Upon approval of the final Summary Report, a validation certificate was issued. ❏
Jackelyn Rodriguez
➲
About the Author Jackelyn Rodriguez is Director, Quality Systems and Regulatory Compliance for Medtronic MiniMed. She has 19 years experience in all facets of quality assurance and regulatory compliance. She specializes in International and U.S. regulations, which define quality systems, design control, CE-marking, risk management, medical device reporting, post-market surveillance and vigilance. She also has extensive knowledge of 21 CFR Part 11, as well as HIPAA requirements. Ms. Rodriguez holds a BS in Business Management, and is a certified member of the Board of Examiners for the Malcolm Baldrige National Quality Award program, and an ASQ Certified Quality Auditor. She has written articles for the Journal of CGMP Compliance, and has been quoted in the several Medical Device related journals, such as the Gold Sheet, and the Gray/Silver Sheet. She can be reached by phone at 818-576-5624, by fax at 818-576-6266, and by e-mail at
[email protected].
Reference 1.
FDA. Software Validation. http://www.fda.gov/cdrh/comp/ guidance/938.html.
•
FDA. General Principles of Software Validation; Final Guidance for Industry and FDA Staff. January 11, 2002.
Suggested Reading
Attachments are presented on the following pages
Article Acronym Listing ASAP: BOM: BPML: BPP: CFR: cGMP: DS; ERP: FDA: FF: FI: FI/CO: GMP: GUI: HR: IQ: IT: MM: OQ: PFD: PO: PP: PP: PQ: PVP: PVR: QA: QSR: RFQ: SAP: SD: SDVLC: SOP: T-Code:
Accelerated System Application and Products Bill of Materials Business Process Master List Business Process Procedures Code of Federal Regulations Current Good Manufacturing Practice Design Specification Enterprise Resource Planning Food and Drug Administration Forecast to Finish Financial Accounting/Asset Accounting Financial/Collections Module Good Manufacturing Practice Graphical User Interface Human Resources Installation Qualification Information Technology Material Management Operational Qualification Process Flow Diagram Purchase Order Production Planning Purchase to Payment Performance Qualification Process Validation Plan Process Validation Report Quality Assurance Quality System Regulation Request for Quotation System Application and Products Sales and Distribution Software Development Lifecycle Standard Operating Procedure Transaction code May 2003 • Volume 9, Number 3
217
Jackelyn Rodriguez
218
Journal of Validation Technology
Jackelyn Rodriguez
Attachment 2
Display Allocation to Plant Business Process Procedure Revision
Description
ECO Number
Prepared by
Released By
Date Released
New Release This document contains information, which is the property of MiniMed Inc..This document may not, in whole or in part, be duplicated, disclosed, or used for design or manufacturing purposes without the prior written permission of MiniMed Inc.
Reviewer:
Date:
Requester/Author
Date:
Process Owner:
Date:
Quality Assurance:
Date:
Overview: Display Allocation to Plant 1. Purpose and Scope Business Process Procedures Overview To display specific material BOM based on plant allocation.
Input – Required Fields
Field Value/Comments
Material Plant BOM Usage
Input requested BOM Part Number Input Primary Plant Location
Output – Results
Comments
Displays primary plant location for specified BOM 2.Tips and Tricks Not Applicable.
Attachment 2 (Continued) May 2003 • Volume 9, Number 3
219
Jackelyn Rodriguez
Attachment 2 (Continued)
Overview: Display Allocation to Plant 2.1.
Step 1.1: Access transaction by:
Via Menus
Logistics>Production>Master Data>Bill of Material>Material BOM>Plant Assignment CS09
Via Transaction Code 2.2.
Step 1.2: On screen “Display Plant Assignment: Initial Screen,” enter information in the fields as specified in the table below:
Field Name
Description
R/O/C
User Action and Values
Material Plant BOM Usage
Part Number Applicable Plant Indicates which applicable BOM
R R R
Input Part Number Input Applicable Use Drill Down to Determine Application
Required/Optional/Conditional (ROC) Required (R) Optional (O) Note: All remaining fields need to be left as is. 2.3. Step 1.3: Click on the Enter Button ✓ 2.4.
Comments
Conditional (C)
Step 1.4: Results of the Display
CS09 Display Allocation to Plant _ Microsoft Word Plant assignment
Edit
Undo Extras System
Help
SAP
DISPLAY PLANT ASSIGNMENT: CURRENT ALLOCATIONS All allocs to BOM BOM
000000016
BOM Usage
1 Production
Material allocations – BOM 1
07004110-001 1001
CS09
2.5.
220
Step 1.5: After viewing necessary information, click on the screen to main menu
Journal of Validation Technology
sapdev01 INS
to return through the previous
Jackelyn Rodriguez
Attachment 3
Unit Test Script Example Opportunity to Cash Monthly Finance Test Script Plan Approval By
Name (Please Print)
Purchase to Payment Forecast to Finished Goods
Signature
Date
Reviewed By: Quality Assurance Approved By: BPP Reference BPP Step Step Description
Required Field
Expected Results
BPP1ME21N.1
1.6
Adopt purchase requisition to purchase order
Purchase req:
Information from purchase requisition copied to purchase order being created
Y
BPP1ME21N.1
1.7
Add vendor information
Vendor Number:
Document type defaults as Standard PO. Document Date defaults to today’s date
N
BPP1ME21N.1
1.8
Enter organizational data
Enter Purchasing Data entered Group: 001 Purchasing Organization: 1000 Company Code: 0010
N
BPP1ME21N.1
1.9
Enter Item Data
Net Price: $10.00 Currency: USD Plant: 1001
Other required data defaults
Y
BPP1ME21N.1
1.12
Enter Item Details Tax Code: PH
Data entered
Y
BPP1ME21N.1
1.13
Save Purchase Order
Purchase Order created
Purchase Order Number:
Y
Passed
Failed
Corrected, Retested, and Passed
Test Completion Results
Test Script Results Approval By
Name (Please Print)
Actual Screen Tester OK Results Print Date Required?
Signature
Date
Reviewed By: Process Owner Approved By: Quality Assurance Approved By:
May 2003 • Volume 9, Number 3
221
