Users have been applying “distributed control” for decades. The process control network (PCN) has extended beyond the control room for nearly as long. And a question we’ve always faced is: “If I need and value full network redundancy, how much is that redundancy compromised if the physical layer follows the same geographic path?”
In the large process industries, huge quantities of highly flammable, explosive or toxic substances are boiled, reacted, transported and stored over large geographic areas. The risks associated with running such processes—with no clue what the process is doing, no alarms and no ability for the humans to interact with the automation system—cause nearly everyone to install redundant networks over distinctly separate geographic paths. But even in these circumstances, the physical network more than likely travels the same path for some distance. How much is too much?
About 15 years ago, I was on a team that did a large instrument upgrade in a refinery. In addition to the issues of redundant paths for the process control network, the team had to rationalize using overhead runs in cable trays versus underground cables for multi-pair, home-run instrument wiring. All our refineries at that time used underground conduits for the runs between field junction boxes and the control house, mostly due to concerns about fires. After 20 years, the underground runs had become as much of a liability as a safeguard because many were likely lying in water for most of that period. Issues with poor signal integrity were common. Plus, the cost of installing new underground runs—allowing for a hot cutover of the plant to new instruments—would price the project out of contention for then-scarce capital. So, we thought a bit about overhead runs and the liability posed by having the integrity of a large refinery’s control system hinging on them.
Any network routing scheme benefits from such an analysis. What bad things are likely to happen, and how likely are they? Redundancy addresses potential failures of media (copper and fiber) and interface hardware (switches, hubs and NICs), but not common mode failures inflicted by the environment they are run in. How should your enterprise assess its physical network vulnerabilities?
Physical harm to network media is a main concern. In a refinery, one worries about fires. Localized fires, like pump seal blowouts on auto-igniting, hot hydrocarbons, used to be relatively common. But these hazards are localized, so it’s usually possible to route cable trays, at least those carrying critical home runs and network media, safely clear of high fire hazard areas. However, a few other potential threats exist, and even a remote risk of total loss of view may be intolerable when these conduits carry the backbone of the PCN.
Trays are usually sized and supported for a full snow load in addition to the cable fill, but what other flotsam can land in there with your network media? Even in newer plants, shards of insulation and tin from process lines, towers or tall reactors can land in them. If you live in a coastal area, strong storms might fling some even bigger objects around. While it’s officially “verboten,” you can probably expect that an operator or craftsperson may find the tray is a handy place for a foothold, if not to walk on occasionally. There are some safety valves that relieve to atmosphere or potential line leaks whose discharge may be unfriendly to cable insulation.
Trays are designed to accommodate tray covers, but I’ve yet to be in a plant that actually used them. Even if you install them when the system goes in, they may come off and stay off indefinitely the first time a plant project needs to pull a new cable in (think Panduit). So, banking on the physical protection of a tray cover may not be infallible insurance against threats from above.
Given the vagaries of physical media integrity, it may be easy to convince your team that the money and effort for geographically separate routing is justified. But the other side of the risk equation is likely consequences. There are processes where a total loss of view is no big deal. A brewery, a wastewater plant, even a water treatment plant, may be able to tolerate a span of time with no connection to a host or central control room. I’ve been in batch pharmaceutical plants where the process was still documented by hand, on paper. And, even in more hazardous processes, the loss of view might be limited to a small subset of loops.
While we’re fortunate to have a wealth of increasingly reliable networks, physical media and interface hardware, users will still benefit from weighing the benefits of separate network routes for critical PCN infrastructure.
John Rezabek is a process control specialist for ISP Corp., Lima, Ohio. E-mail him at [email protected]
