By John Rezabek
A few weeks ago, our instrument specialist made the rounds of remote multiplexor panels to check for power-supply failures. In recent years, we've been a little alarmed to find one of the two redundant dc power supplies failed. We unknowingly were limping along on a single power supply, at risk of losing an entire node of multiplexed I/O.
By definition, these measurements are "indicate only," mostly thermocouples and RTDs, and not essential to the functioning of the plant. Nonetheless, losing an entire bank would be more than a little unnerving for operators.
What's a little ironic is that this power-supply health information is available through the multiplexor systems' standard diagnostics, over the same Modbus 485 link through which we get the process data. If we just set up the serial interface to query a few more registers, we'd have access to a wealth of diagnostics—not only power-supply status, but individual I/O card statuses, communication status and more.
The same is true of a number of other devices connected through serial interfaces, including analyzers, flow meters and small PLCs. Why aren't we making full use of the diagnostics we have?
One of the challenges is the time it takes to decode and test the diagnostics available. This data is often a bit-string, so the developer or end user has to figure out which bits mean what, and whether a zero is good or vice versa. Testing sometimes can be a challenge. If you have to fail something, you risk a potentially wider malfunction. Usually when I tell an operations supervisor, "Nothing bad should happen, but there's a chance you'll lose all 120 temperature indicators in Area K," the response is: "Can you do this on someone else's shift?”
Especially when hardware is reliable, implementing diagnostics like these, that require a little effort, becomes a task akin to reading the manual—that is, no one bothers until something's broken. But it only takes a few small victories to convince your boss that the effort needn't repeatedly return to the back burner.
One of the common pieces of hardware that fall in the category of "reliable and therefore forgotten" is the Ethernet switch. I have a pair of 3Comm Superstack switches—my system supplier doesn't even recommend them anymore—that have been cruising along bumplessly for years. But a sudden failure of one of these switches, or even a node becoming isolated, would be pretty troubling. I'm not even sure if I can get spare parts. Why aren't I pulling in some of its diagnostics, or at least viewing them directly on occasion? More modern industrial switches have support for Modbus and OPC, which might make it simpler for an end user to configure, display and trend some of the switches' more interesting statistics.
A very rudimentary diagnostic is to try to capture or derive a status for the points coming in over a serial link. Our implementation has the unlucky feature of holding last value—flat-lining—on communications loss, like one would experience if both redundant power supplies failed. To implement status, we first have to choose a variable type that supports status and then either do an "AND" of some selected diagnostic bits and set the point's status accordingly or test if at least the "comm. Fail" state is flagged correctly by the serial interface device. If you already are living with a configuration that includes hundreds of such points, this could be a long effort. It's much easier to implement when you're deriving your templates early in the design phase. Some systems support classes of modules, which can be a path for mass-editing of a class of module, such as serial Modbus data points.
Another interesting diagnostic that end users could exploit more is the "AO Readback" variable of HART valve positioners. This represents the actual valve position as seen by the positioner. To get it, you need a HART-capable I/O card. Patching it back from an asset-management system—using OPC, for example—could be a little dicey, as the possibly long and variable latencies make the potential for stale data large. Some HART polling applications can take a long time—hours—to cycle through all the HART devices.
By making full use of the diagnostics we already have, we not only help our enterprise and the people that come after us, we encourage and push our counterparts on the supplier side to provide us with diagnostics that are timely, meaningful and easily accessible.
John Rezabek is a process control specialist at ISP in Lima, Ohio.
