Monitor the Networks

Software Tools that Help When Things Go Wrong At 2:00 A.M.

By Paul Miller, Contributing Editor

The proliferation of standard Ethernet on the plant and factory floor, makes it clear that many of these industrial networks are not being monitored adequately. That’s because the people on the floor don’t always have the right tools and because corporate network support people have different priorities.

Without effective monitoring capabilities in place, plant or factory personnel don’t learn about impending problems or network events until after they occur. All too often, this is right in the middle of the night when outside support is not available. The right network monitoring tools can help plant-level people to determine quickly whether a problem is in the network or in a machine, controller or other network-connect device. With this information in hand, they can identify, locate and resolve many problems before the corporate network support people finish their first morning lattés.

Help! I Need Somebody

Paul Sperling, technical systems manager for the food products manufacturing division of Kroger (thekrogerco.com) in Cincinnati, has worked on both sides of the IT fence. “When you have enterprise tools, there generally are politics involved with who can do what with the software,” says Sperling. “It’s easier to provide the people on the plant floor with intuitive tools that don’t get in the way of what needs to be done.”

In many industrial plants, the people most responsible for quickly fixing the automation network if something goes wrong in the middle of the night are not all that well-versed in IT. “They need tools that will help them quickly find the problem using a familiar industrial software interface,” comments Troy Watts, automation technology manager at the Encompass Group (encompassgroup.ca), a Vancouver, B.C.-based system integrator. “Enterprise-level packages work fine in larger plants where there is enough resident talent, but it’s likely that these will be beyond the scope of the average plant.”

Without some type of network management, it can be difficult to diagnose automation system performance issues. “For example, there could be a periodic high traffic load, due to something like a data backup for servers, that impacts system performance,” states Dan Knight, marketing manager at Cisco Systems (cisco.com). “With no way to monitor the network and correlate trends and events, it could be very difficult to understand the causes of these types of issues. Spending time and resources trying to diagnose problems without the proper tools can be very expensive in an environment with high cost of downtime or performance issues.”

Products Galore

Clearly, there’s no shortage of products being offered. IBM, Cisco, Computer Associates, Hewlett-Packard, Microsoft, Sun Microsystems, Juniper Networks and other well-known companies offer network management software and solutions. Most of these are enterprise solutions designed for large, complex corporate networks. As a result, they’re typically very expensive and require a high degree of IT expertise to use effectively. These solutions are not well-accepted by overburdened engineering, maintenance and operations staffs at the factory level.

“The IT tools that are out there are too complicated for use at the plant floor level by plant engineers and operators,” says Mark Fondl, president and CEO of Network Vision (intravue.net), Newburyport, Mass. (Figure 1) “They need tools that enable them to solve problems at the plant floor level quickly and easily. This helps them understand the network better so that they can communicate more effectively with the IT guys.”

Effective Detective
Figure 1
This software automatically discovers all the devices on the network and then maps their connections.
Photo by Network Vision
Encompass Group’s Watts has used N-Tron switches and software for about four years in 20 automation projects, including bulk material handling plants, oil and gas production fields, pulp and paper mills and a plywood manufacturing plant. N-Tron (n-tron.com) offers ruggedized, DIN-rail-mounted industrial switches plus network monitoring software intended for use by plant-level people, rather than IT professionals (Figure 2).

“We embedded its N-View into existing RSView and Wonderware HMIs so the plant guys are using their existing screens,” says Watts. “We’ve animated the screens to show the state of the network and put in alarms to indicate hardware, traffic or media issues. This provides a point of reference so that the plant people can start fixing things before they get out of hand and determine the root cause of recurring problems. When we started using this product, it was the only solution that offered OPC to let us plug into HMIs without having to expose users to yet another piece of software.”

While it’s often assumed that lower-cost, unmanaged switches can’t be remotely monitored or diagnosed, N-View software claims a unique solution for that. Switches equipped with the appropriate firmware periodically autocast a small Ethernet packet containing port-by-port switch status. An OPC server captures the data, and the variables can be displayed as string variables in most HMI applications with OPC client capability. This provides remote network traffic, status and error monitoring for N-Tron unmanaged switches with real-time data visibility within familiar plant applications. Both the software and OPC server are included at no charge with the switches.

String Conversion
Figure 2
Figure 2: OPC variables are presented to the OPC client as string variables, which most HMI software packages can convert into the appropriate data type required for display alarming and trending.
Photo by N-Tron
The company argues that this approach is more efficient, more secure and less intrusive on the network than the simple network management protocol (SNMP) approach used for fully managed switches. “SNMP uses the entire TCP/IP stack to send and get messages, creating lots of network traffic and overhead,” says Warren Nicholson, N-Tron’s president and CEO. “This verbose approach is very intrusive on the network. N-View messaging is more efficient and less intrusive, and since it uses a read-only approach, it’s also perfectly bulletproof and secure.”

Less Fuss?

Since they don’t assume a high level of IT expertise, network packages designed for industrial environments enable plant-level people to quickly identify, isolate and respond to specific issues on the production network without having to call in IT people who, at best, often just don’t understand the urgency.

We’ve seen that some of these industrial network monitoring packages are designed to be used exclusively with that vendor’s switches. Others are based on industry standards such as SNMP and work with most vendors’ fully managed switches (see sidebar Sometimes It’s a SNMP at the end of this article).

Most industrial packages also provide web interfaces. Many use OPC to display the network monitoring data in HMI applications from familiar companies such as Wonderware, Rockwell Software and GE Fanuc. ODVA’s Common Industrial Protocol (CIP) also is beginning to make its appearance in a handful of vendors’ industrial switches.

Capabilities to Spare
Whether intended for enterprise environments, industrial environments or both, most current network monitoring packages provide the following capabilities.
  • User interface
  • Real-time reports and graphics
  • Ability to monitor network devices and services
  • Auto-discovery of devices and services for monitoring
  • Ability to set alert and alarm thresholds
  • Network management packages add the capability to configure or reconfigure the network, individual devices on the network and network services.
You’ll find an overview of some of the software packages available for monitoring or managing industrial networks in an extended version of this article at IndustrialNetworking.net/monitoring.
Plant and Enterprise: Convergence, Not Collision

Not all the experts polled agreed that industrial networks even require special tools. But almost everyone agreed that the industrial and enterprise worlds are converging rapidly, and their Ethernet networks represent the epicenter of this convergence. As a result, unless the network in question is either very small or stands alone, companies will need to adopt a more collaborative approach for managing their industrial networks within an enterprise context.

“Current information technology is plenty robust and can be effectively managed in a plant environment,” believes Ernie Rakaczky, networking consultant at Invensys (ips.invensys.com). “In an industrial environment, however, the difference is not so much about how you manage the network itself as how you go about managing the devices on the network. The world needs to start coming together through cross-collaboration between plant engineers and enterprise IT people.”

So, if the manufacturing and IT networks are converging, are organizations and skills starting to converge as well? “The longer-term direction is toward more convergence and use of common tools, or at least protocols usable in multiple tools, depending on function and responsibilities,” comments Cisco’s Knight. “Rather than have two groups managing different parts of the network using different tools and technologies, a converged organization with a common set of tools to manage the overall manufacturing environment has a number of advantages.”

Which Package Is Best For You?

Things to consider before committing yourself to any specific package for your plant, factory or other industrial site will depend on a number of different factors.

  1. Size and complexity of your network
  2. Criticality of your applications
  3. Type(s) of switches used in your industrial network (unmanaged, partially managed or fully managed)
  4. Specific switch vendor or vendors
  5. Your network topology
  6. Network media used (copper, fiber and/or wireless)
  7. Degree of IT expertise available at the plant or factory level
  8. Availability of enterprise-level IT support and software
  9. Corporate standards, if any
  10. Cost constraints

The main point is that you’re using Ethernet networks in your plant or factory, so you should have an appropriate solution in place to effectively monitor, diagnose and manage your network or networks. This solution might or might not come from the same vendor that supplies your switches and other network hardware and might or might not be additive or complementary to commercial network management software that is running at the enterprise level.

What’s important is that the people on the spot who are responsible for keeping your plant or factory operating when something goes wrong at 2:00 a.m. have simple and intuitive tools available to them to help them figure out what went wrong, pinpoint the location and resolve the situation before it starts costing your company a lot of money in lost production.


Sometimes It’s a SNMP

Simple Network Management Protocol (SNMP) is an open standard within the application layer of the Internet Service Protocol Suite. It is used in all commercial enterprise management software and many industrial network management packages. SNMP is designed to allow Ethernet devices on a network to be remotely monitored and managed in a uniform manner, regardless of vendor. SNMP agents in managed network devices perform diagnostics and communicate back to the network manager stations (NMS) that host and execute the management applications. A Management Information Base (MIB) database enables each unique object identifier to be placed within a hierarchy and accessed in a common manner.

“As Ethernet devices grow in number and importance on the plant floor, SNMP’s importance to manufacturing operations will grow also,” believes Harry Forbes at the ARC Advisory Group (arcweb.com). “This stems from the opportunity to leverage SNMP to derive greater value from manufacturing operations. Increasingly, SNMP will serve as a foundation for visualization ‘dashboards’ monitoring production assets for real-time performance management (RPM) applied to those assets.”