Ups and Downs of Taking the Bus—Part II

An Early Adopter of Fieldbus Technologies Overcame a Few Start-up and Commissioning Issues and Qualified a Control System

In this two-part series, Wyeth's Gary Forrest discusses the challenges and benefits of implementing fieldbus technologies at a new bioprocess development facility. The first part can be read in the Q2 issue of Industrial Networking or by visiting IndustrialNetworking.net/wyeth1 right now.

By Gary Forrest, PhD, Wyeth Research

Wyeth’s new bioprocess development facility in Pearl River, N.Y. was planned to include process development and manufacturing of high-value products from microbial fermentation. New equipment and full automation, through the use of a modern, integrated process control system, would permit batch automation in accordance with ISA’s S88 standard. And the fermenters would be equipped with instrumentation and controls that connected to the process control system via fieldbus technologies.

Wyeth elected to move the license for its existing Emerson Process Management DeltaV process control system and some of the process control system elements to the new building. A previous system upgrade had added fieldbus instrumentation and controls, and Wyeth specified fieldbus designs for the new plant, intended not only to replace existing facilities, but also to add new functionality.

Commissioning

System commissioning for this system was a multi-step process beginning with factory acceptance testing that took place from September through December 2003.

During this period, there was limited coordination between the fermentation equipment manufacturer and the automation supplier, and some configuration and wiring issues were identified. Resolution of these issues was deferred until installation and commissioning. The fermentation skids were shipped at the end of December and placed in storage awaiting the completion of building construction. Site installation, startup and commissioning of the fermentation skids began in August 2004.

Problems identified during fermentation-skid factory-acceptance testing needed to be resolved on-site. Although there were a few issues with FF devices, the fieldbus provided a solid communications platform with sophisticated interaction between the DeltaV process control system and the field devices. FF provided full digital communications and diagnostics for all devices without complex configuration or interpretation of signal bits (Figure 1).

Figure 1
Bus Transfer
The temperature and pressure transmitters on biowaste vessels are FF devices. The digital I/O is based on a Festo Profibus-DP solution. But liquid level is measured by a 4-20 mA HART device unavailable in a fieldbus implementation.
Source: WYETH
There were, however, problems with our specific implementation of Profibus-DP instrumentation and controls. Personnel experienced with the startup and commissioning of Profibus-DP fieldbus systems were not available. Eventually, the problems were traced to wiring problems such as improper connector and wire combinations. Specific wiring requirements were not well understood during system installation, and diagnostic tools for the analysis were not readily available.

In addition, the software configuration of each Profibus-DP device, through the use of a GSD configuration file provided by the instrument and controls vendor, required the complex specification of slots and signals that needed interpretation by software on a bit, byte or word basis.

The DeltaV process control system provided interaction with the Profibus-DP devices in a manner that generally was limited to setpoints, process variables and digital states. Despite specific individual difficulties, the Profibus-DP instrumentation has provided reliable, robust communications with the DeltaV process control system since completion of commissioning.

Software Synergies

Emerson, through its local representative, Control Associates, based the DeltaV software configuration on a vendor functional requirement specification (FRS) supplied by Sartorius BBI. The fermenters were placed into service for R&D applications in Q4 2004 (Figure 2). Because the facility was up and operating for R&D applications, Wyeth had the luxury of entering a software development prequalification period for the DeltaV process control system.

During this period, Wyeth created its own comprehensive FRS for all systems not included in the Sartorius vendor documentation. At the same time, these documents went through several modifications as the software was modified to conform to user requirements. The DeltaV process control system was ready for qualification in August 2005.

Figure 2
Controlled Skid
Sartorius chose instrumentation and controls with Profibus-DP connectivity for its 400-liter working volume fermenter.
Source: WYETH
System Qualification—Fieldbus I&C

Wyeth carried out the control system validation with the help of Valspec, a validation contractor in Royerston, Pa., that specializes in qualification of DeltaV systems. A validation plan was developed in accordance with the International Society for Pharmaceutical Engineering‘s Good Automated Manufacturing Practice (GAMP 4), which established the scope, strategy, deliverables and responsibilities for system qualification.

Wyeth and Valspec addressed a key question in the validation plan: “How do we qualify fieldbus I/O?” The strategy involved the preparation of three documents. All field instrumentation and controls were to be qualified through an enhanced commissioning loop-check protocol prepared and executed by Wyeth personnel. The DeltaV process control system and the configuration software not involved with I/O configuration were to be qualified using an installation and operational qualification (IOQ) protocol developed by Valspec and executed by Valspec and Wyeth personnel. A third performance qualification (PQ) protocol was to be prepared by Wyeth to address specific user requirements not covered in the IOQ.

Enhanced commissioning differs from normal commissioning. It is carried out in accordance with a protocol that has been preapproved by the company’s quality organization, using the same quality standards that are applied to the qualification of pharmaceutical manufacturing equipment.

All of the instrumentation and controls installed on the DeltaV system were tested in accordance with this pre-approved document. The loop-check protocol verified:

  • I/O model and serial number
  • I/O presence in the DeltaV configuration database
  • configuration software date and audit trail version number
  • correct addressing
  • graphic links
  • correct functioning of the I/O point
  • presence of the I/O in the OSI PI historical database.

A page was provided in the loop-check protocol for each I/O point, the contents of which were customized for each type of I/O point. Conventional, Foundation fieldbus, Profibus-DP and AS-i bus I/O points were verified with information appropriate to that point type. The IOQ protocol was developed in accordance with GAMP 4 principles to provide verification that the control system software and hardware met design specifications.

A traceability matrix demonstrated correspondence between system-design documentation and system testing.

The IOQ protocol verified:

  • hardware and software installation
  • system graphics
  • control loop operation
  • phase and procedure operations

Additional Wyeth requirements for validated computer control systems were addressed in a PQ protocol. In accordance with GAMP 4, the PQ protocol verified and documented user requirements recorded in Wyeth policies and procedures for validated computer systems.

The PQ protocol included documentation of:

  • regulatory analysis, risk analysis and electronic records/electronic signature assessment
  • supplier assessment
  • operation and administration procedures assessment
  • personnel training assessment
  • security and access control assessment
  • system performance and capacity assessment
  • database backup testing
  • disaster recovery plan assessment
  • preventive maintenance procedures verification.

A qualification summary report for the process control system was issued and approved in November 2006. Life cycle management design, commissioning and qualification are elements of the system life cycle for qualified systems in pharmaceutical manufacturing.

Qualification testing establishes a qualified state, but the qualified state must be maintained. Preventive maintenance, change control and period system reviews are practiced for the life of the system.

In summary, Wyeth, Pearl River, N.Y., has been an early adopter of fieldbus technologies. The use of fieldbus instrumentation and controls resulted in some startup and commissioning issues, but these issues were resolved after site installation. The new bioprocess development facility has a fully validated process control system for the support of R&D, clinical production and commercial manufacturing. The qualified system incorporates all of the advantages of the new digital fieldbus technologies. The functionality and reliability of the fieldbus instrumentation and controls has been outstanding.

Gary Forrest, Ph.D., is associate director—bioprocess development at Wyeth Research, Pearl River, N.Y.

In this two-part series, Wyeth's Gary Forrest discusses the challenges and benefits of implementing fieldbus technologies at a new bioprocess development facility. The first part can be read in the Q2 issue of Industrial Networking or by visiting IndustrialNetworking.net/wyeth1 right now.

Bus Basics

Conventional instrumentation and controls use a pair of wires for each device to carry analog signals for communication. In contrast, fieldbus instrumentation and controls use a digital network that permits the instrumentation to communicate with the other instruments and the process controller. With a fieldbus, a network of field devices is connected to the process controller via a pair of wires.

These wires provide bidirectional communications from multiple transmitters and control devices to the process controller. Fieldbus instrumentation for process applications comes in a variety of forms, including AS-i bus, DeviceNet, Ethernet/IP, Foundation fieldbus and Profibus. Open system standards and vendor product certification are available from independent foundations and associations of vendors for each fieldbus type.