Jim Montague is the executive editor for Control, Control Design and Industrial Networking. Email him at [email protected].
Just as the Florida Keys now are linked to the mainland by a causeway, formerly isolated islands of machines and production lines are being drawn into widespread automation and enterprise networks. This archipelago helps coordinate operations, optimize performance, improve efficiency and aids profitable decision-making. However, as users of IT, business and process control networks have learned, more-inclusive access creates more risks and requires more and better security.
Up to now, machine security has sought mostly to protect remote, dial-in monitoring, diagnostics and troubleshooting applications, but these solutions are rapidly becoming more complex and closely integrated, especially as end-user networks expand.
Secure the Support
SHL Automation in Bottingen, Germany, has been building polishing, grinding, deburring and brushing machines since 1989, and integrating them with robots and other production systems to produce turnkey solutions that include training and service (Figure 1).
Figure 1: SHL Automation's robotic polishing and super-finishing machine uses PLCs, PC-based controls and robots that it can access and support over secure, permission-based links to users' VPNs and via TeamViewer remote-support software.
"For 25 years, we've taken our customers' parts and developed plans to perform what they need, and this often means using secure, remote access," says Thomas Koch, SHL's executive sales vice president. "We typically use TeamViewer IP-based support software for remote access, but we can also work via customers' virtual private networks (VPNs) that are segmented and firewalled. This gives us a direct connection from SHL to their sites and equipment over a separate network, which is more secure because our users decide when to allow us to access their noncorporate networks. For example, a user will establish a TeamViewer account and username, and then give us a password and permission when needed."
To address internal as well as external security, CNC machine builder Okuma America in Charlotte, North Carolina, reports it recently developed and launched its OSP Virus Protection System (OSP-VPS) as a factory or field option on its OSP machine controllers. OSP-VPS embeds a virus-scanning application programming interface (VSAPI) from Trend Micro on Okuma's controls to prevent infection by viruses from networks or USB flash drives.
Figure 2: The U.S. Dept. of Homeland Security's (DHS) U.S. Computer Emergency Readiness Team recommends a network architecture that compartmentalizes communications and uses a defense-in-depth strategy with intrusion detection and a security incident and event manager (SIEM). This diagram is part of "Improving Industrial Control Systems Cybersecurity With Defense-In-Depth Strategies," which is available at ics-cert.us-cert.gov.
Okuma reports that OSP-VPS provides reliability, maintainability, safety and usability. First, before a virus pattern file is distributed and search engines are updated, operations are verified in advance by Okuma to prevent CNC software-detection errors. Next, the system issues an alarm when a virus is detected, and operators can check the corrective actions taken on a display screen. Third, virus scans are performed after completion of automatic and MDI operations to prevent slowing of screen responses due to scheduled virus scans. Finally, pattern files and search engine updates can be done either manually via the operator's PC or automatically via the Internet. Also, an option to update through the operator's PC or USB memory can be used for machines that aren't connected to a LAN or those on a closed LAN.
Basic Protections
Also Read: Industrial IT Takes Some Trust When The Internet is Not a Guaranteed Safe Place
Naturally, unauthorized probes, intrusions and attacks get all the attention, but network misconfigurations and other mistakes cause far more unplanned downtime and damage. Either way, these new industrial network archipelagos need serious security evaluations and upgrades. This means: use complex passwords; enable network-protection software and encrypted communications; identify all network-access points; close all unneeded ports and services; and segment and divide networks.
Simplify and Secure
Logically, one of the best ways to improve machine security is to turn off or remove unneeded software, hardware, services or access points, as well as simplify associated network designs, components and potential failure points. For example, Frimo Group in Wixom, Michigan, and Lotte, Germany, builds machines and production systems for automotive and other plastic-parts manufacturers at 15 locations worldwide, and its remote service technicians have supported its clients' operators for about 20 years, initially accessing its machines' PLCs via 56-kbps analog modems and telephone lines (Figure 3). Over the years, broadband Internet connections replaced analog links, which established manipulation-proof connections through VPN tunnels, while faster data connections between users' plants and Frimo's technicians allow the machines' industrial PCs to be operated easily. In fact, using virtual network computing (VNC) software, the entire screen content of a remote computer is transmitted and can be used by service technicians just like a local PC. Frimo mostly uses its remote capabilities for rapid fault clearance, but expanded services are available, and security is essential.
Figure 3: Frimo's plastic pressing, forming and thermoforming machines use security routers to simplify networking, configuration, remote services and security functions.
"Increasingly, powerful industrial PCs have taken over more functions in our machines. Analog connections are no longer sufficient to ensure remote maintenance for these computers," says Axel Starflinger, IT administrator at Frimo. "We adapt our machines to the specific requirements of our customers. With fast and secure VPN connections, we have access to all the devices in the machine. For example, our service allows us to remotely set up an extra checkbox in the PC's visualization system or adjust the parameters of a frequency converter."
To keep its remote services and clients secure, Frimo initially used a large supplier's broadband, remote-maintenance solution, but found it too complex and costly. It decided to implement 80 mGuard industrial security routers from Innominate Security Technologies, a division of Phoenix Contact, because the configuration templates with required parameters could be read into mGuard on an SD card. This reduced configuration time from several hours to a few minutes because the only items that need to be added are customer-specific entries for the VPN connection, the customer's default router network and the machine's IP addresses. Address conflicts are avoided by mapping the machine network's real addresses onto virtual IP addresses through the 1:1 network address translation (NAT) function of the VPN router. That means added adjustments to the machine's internal address space no longer are needed.
In addition, Frimo deploys an mGuard bladeBase compatible with up to 12 mGuards in a standard, 19-in. rack at its headquarters, which connects to all its locations via an internal multiprotocol label switching (MPLS)network. All technical parameters and authorizations are already set up in the rack, so new machines can be added and connected without making added entries. Consequently, when servicing is required, technicians from any location obtain remote access to the client's machine via the blade unit over the VPN connection, depending on their authorization levels.
"With our centralized solution, we created a uniform and standardized access solution for our subsidiaries. This simplifies operation, and administration costs are considerably less," Starflinger says. "Security concerns are initially high, especially in the automotive sector. However, the benefits of rapid troubleshooting and the security features of our solution are convincing." Starflinger says that mGuards integrate three coordinated security components based on hardened, embedded Linux programming, including a bidirectional, stateful firewall, flexible NAT router and a secure VPN router with IP security protocol (IPsec) encryption. He adds that operators' IT teams especially appreciate that mGuard prevents external access to the machine by default. This means a secure data connection can be established only with the explicit authorization of the operator via a VPN hardware switch, so access to the machine is only initiated by an outgoing connection controlled by the customer.
Lessons from IT
Because machine control and automation networks increasingly overlap onto enterprise and IT-level networks, it shouldn't be surprising that they can use their security methods.
For instance, Wintersteiger in Ried, Austria, deploys more than 40,000 automatic ski and snowboard servicing machines worldwide, so its engineers must make quick, remote diagnoses to provide support, especially because many of its end users have limited technical expertise. Consequently, Wintersteiger uses Talk2M Internet service portal and software from eWon, which allows its engineers access from anywhere at any time, even using mobile devices (Figure 4).
"Once we have the customer's agreement, our engineers create a connection to the machine with just a few mouse clicks and get a 1:1 image of the relevant operator guidance shown on the screen via VNC," says Reinhard Pfeffer, technical customer service director at Wintersteiger. "A camera integrated into our remote system also is employed more frequently. The customer simply points the camera at the problem area, and I can see what the matter is straightaway. In most cases, no further explanations are required."
Figure 4: Service engineers at Wintersteiger use an Internet service portal and software—with the permission of their users—to support their fully automatic ski and snowboard servicing machines from anywhere, even via mobile devices.
In addition, Talk2M enables several engineers to securely access many ski and snowboard servicing machines simultaneously. "In principle, all our service engineers have access to the system, and appropriate access rights can be specified to clearly regulate who can do what. We also use some freelance engineers, and we can connect them to our account management system at any time without any added license payments. The user interfaces generally are set up in a very transparent way, and so we can see at a glance who's logged into the Talk2M server. On average, we have about 100,000 VPN connections running on our Talk2M server at the same time, and that says everything about the performance our system is achieving."
Daymon Thompson, TwinCAT product specialist at Beckhoff Automation, adds, "Machine security can be tricky, but it's manageable. Use methods similar to enterprise security, such as software and hardware-based firewalls and segregated networks. You need a good communications protocol to log data onto the enterprise network for calculations such as overall equipment effectiveness (OEE) or to manage recipes."
Beckhoff recommends the OPC UA protocol from the OPC Foundation because encryption is built in, and it uses an X.509 certificate, username/password combination and Kerberos authentication. This encryption eliminates man-in-the-middle (MITM) attacks and other intrusions that seek to inject or pull data. Also, all ports in the firewall can be blocked, allowing only approved traffic. All of Beckhoff's PC-based controllers can serve as OPC UA clients or servers, so they can take PLC data, OPC UA data and field-sensor data, log it onto an SQL database running on Microsoft Azure or other cloud service, provide secure encryption from the machine to the cloud, and display it on a tablet PC or other web-enabled device.
"This is where the Internet of Things (IoT) and Industry 4.0 are really emerging, but these implementations must be secure," Thompson adds. "Fortunately, setting up security with an OPC UA client is simple, and communication can be as easy as adding a function block to a PLC. Also, and by default at the machine's fieldbus level, slave devices on Beckhoff's EtherCAT network will destroy any non-EtherCAT frames, reducing the risk of denial-of-service attacks and rogue network frames."
People Issues and Cloud Cover
Despite all the terrific security tools and software available, experienced control engineers and IT professionals warn that each can be circumvented by unintentional mistakes or premeditated human actions. One infected USB stick plugged into a secure area can render many defense levels useless.
Luckily, the most psychologically beneficial way to establish the eternally vigilant mindset required to meet machine security's ongoing needs is to simply view it under the better-known, longer-established machine safety umbrella, which also demands continual and longstanding attention. Just as control engineers and operators woke up to the fact that machine safety is just another way to be smart and run efficiently, many are beginning to appreciate that security can provide the same benefits of reducing downtime and other setbacks.
"Firewalls can be set to accept data only from trusted nodes, but a machine builder's staffers or contractors can get compromised, just like Target's well-known HVAC contractor," says Tom VanNorman, cybersecurity services manager at Phoenix Contact. "So builders also need a security policy for their supply chain." To organize and establish secure firewalls, zones, conduits and other protective layers, VanNorman recommends that machine builders learn about and follow the ISA 99 and IEC 6443 standards, which recently merged into the ISA/IEC 62443 standard. It shows how to conduct risk assessments (RAs) for security, identify threats, adopt password policies, and segment and firewall networks. In addition, VanNorman reports that IT-level tools such as Snort, Industrial Defender and Sourcefire can examine network traffic and find problems.
"I also recommend using virtual machines because they can be accessed and updated faster, which makes many security tasks easier," VanNorman adds. "This allows a builder to have a different virtual machine for each client, and, because a virtual machine is really just a software file that does data processing, nothing needs to be bolted on. What's also needed are policies and procedures to govern these devices, such as which passwords to use, when and how to apply software patches, prohibiting web browsing on production's PCs, how to find anomalous network traffic, and what to do when equipment breaks down."
