Remote Machine Support Must Be Everywhere

Oct. 24, 2014
Remote monitoring, analysis, support and control is getting more routine and secure, more comprehensive in its scope and capabilities, and may even gather enough data to improve machine designs
About the Author

Jim Montague is the executive editor for Control, Control Design and Industrial Networking. Email him at [email protected].

Can't work everyplace at once? Better find a way.Manufacturers demand 24/7 uptime and system availability, but that means machine builders and support technicians must be ready at a moment's notice to maintain, troubleshoot and repair practically every machine they've ever sold in hundreds if not thousands of locations scattered worldwide.While human cloning isn't practical yet for machine builders and other stressed-out professionals to make all the copies they need of themselves, there are other helpful tools they can employ to get in front of all the equipment they must maintain. Remote monitoring, support and even control are offering more varied and sophisticated capabilities for accessing machines and production lines from a distance and then examining, troubleshooting, repairing and upgrading them so users need not go to each physical location.More recently, rather than communicating via old-fashioned modems or asking for permission to access virtual private networks (VPNs), the newest remote-support methods enable builders and technicians to service protected versions of a machine’s operating data and software, which are delivered to virtual, cloud-based services that don’t require users and their IT staffs to permit access to their internal networks.Also Read: Everything That Rises: The Convergence Between Control Systems and Asset-Management Systems Is Inevitable
Watching Watch Parts Production

Figure 1: To monitor and service it Swiss-type lathes, Tornos uses its Tisis machine communication system, which integrates a password-protected Web server into its machines, communicates via VPN, Internet protocol (IP) and Wi-Fi and delivers HTML-based displays to PCs and tablets. The builder also uses TeamServer software to access VPNs with permission, take control of machines remotely and access CNCs to solve problems.

[PHOTO CREDIT]Jim Montague

For instance, Tornos in Moutier, Switzerland, builds Swiss-type lathes and other equipment, such as its six-axis SwissNano CNC machine. It uses bar stock to manufacture wristwatch, medical and dental device parts up to just 4 mm wide and 28 mm long with a precision of ±1 micron. Where watchmakers traditionally used several cam-driven lathes to make their tiny parts, two-year-old SwissNano can make 85% of typical watch parts in a footprint that's only 600 mm wide but contains a multi-spindle lathe with two rotating tool positions and 12 total positions (Figure 1). Tornos also has a subsidiary company, Almac, which builds milling machines based on SwissNano's frame."Companies were not making the old cam-operated machines anymore because they were so costly to build, and so the watch industry wanted a new, different type of machine with counter-spindles mounted on three axes, so they could get better centering and alignments. Watch makers also wanted more tools, easier access, and simple presets to switch out worn tools," says Paul Cassella, applied technology manager for Tornos Technologies US in Lombard, Illinois. "The watch industry previously used some CNCs, so we developed SwissNano with Fanuc Oi-TD CNC controls, chucks from Rohm, vacuum parts recovery and six pneumatic lines for grippers and other functions."However, because Tornos deployments to users like Rolex and other watchmakers are growing along with the recent resurgence of the industry, it needed a better way to keep tabs on its machines in the field. So, about a year ago, Tornos launched its Tisis machine communication system, which transfers 2D and 3D CAD/CAM designs to its machines, selects tools and checks on their programs and operating status. Tisis uses a small, password-protected Web server integrated into its machines, communicates via VPN, Internet protocol (IP) and even Wi-Fi, and delivers HTML-based displays to PCs and tablets."Users can remotely organize parts databases, set production rates to run machines overnight, monitor their machines from home and receive preset alerts and alarms," says Cassella. "Besides using passwords, we also rely on Wi-Fi's native encryption for added security. Though it's not part of Tisis, we also use TeamServer software to access users' VPNs with their permission, take control of our remote machines and go into their CNC controls to solve most problems."A few years ago, remote access was just about monitoring machines, says Sal Conti, remote monitoring product manager in Rockwell Automation's Remote Support Services division. "Now it's allowing machine builders and users to deploy their best engineers anywhere at any time, doing it a lot less expensively, gaining data for much better proactive maintenance and letting technicians know what they'll be facing and what they need before going into the field," he explains. "We do more remote monitoring and troubleshooting and less remote control. However, we tell users how to fix their machines, or we can fix them by putting machines into program mode, and making needed changes."One of Rockwell Automation's primary remote monitoring and assistance tools is its two-year-old Virtual Support Engineer (VSE), which can be scaled to fit the size and risk threshold of each user's application, use store-and-forward functions to preserve data, display operating trends following alerts and alarms and maintain access via cellular links if an IP connection isn't available.Simpler Security? Sweet!
Always Available—Securely

Figure 2: Big Drum's ice cream filling machines, such as its Choice Filler, can be continually monitored and serviced by technicians and maintain 100% availability via VPN network connections enabled by security routers, which serve as secure, encrypted gateways and firewalls.

[PHOTO CREDIT]Big Drum

Of course, the dark undercurrent and persistent question that goes with all this remote access is how to maintain its security. One instructive solution comes from Big Drum Engineering in Edertal-Giflitz, Germany, which designs, builds, installs and supports filling machines for the ice cream and other food industries worldwide. Its machines fill liquid products in containers from 50 ml to 5 l with an output of up to 50,000 units/hr (Figure 2). The machine builder reports its end users are increasingly demanding continuous system availability because any downtime can quickly lead to production and financial losses. Consequently, Big Drum views "demand-oriented maintenance" and fast troubleshooting via safe, secure remote services as critical for keeping its users up and running."With the exception of the Sahara Desert, our systems are used in almost every region of the world," says Andreas Itter, Big Drum's sales and marketing manager. "To ensure 100% availability, we've developed an effective and secure remote service infrastructure. Large customers such as Nestlé and Unilever expect us to deliver services in accordance with total productive maintenance (TPM), and similar demands from other companies have also increased strongly in this area."Big Drum has provided remote machine support for about 10 years, constantly monitors its machines and transmits key parameters to its headquarters, which allows its technicians to maintain ongoing data control. Its service department is staffed 24/7 and can immediately react to any machine disturbances. This always-on availability typically means users need and expect a permanent, online connection to Big Drum as a prerequisite for better service, preventive maintenance and higher security. All its filling machines with PLCs from Rockwell Automation are routinely equipped with remote service modules. The builder reports it presently operates more than 100 of these remote systems with different end users.To achieve secure, online connectivity, Big Drum also employs mGuard security routers from Innominate Security Technologies, a division of Phoenix Contact. These switches protect IP data connections with a VPN-enabled Ethernet router and a configurable firewall with dynamic packet filtering. Technicians connect to users' plant operators via a VPN, and mGuard serves as VPN gateway, connecting the technicians to the plant network via the Internet. These secure, broadband IP and VPN connections for online monitoring are more reliable and stable than former modem connections, and they can handle increasing data volumes and other services. In practice, Internet connections to Big Drum only materialize when a VPN key is manually switched on.To safeguard confidentiality and authenticity, mGuard uses cryptographic protocols and hardware-accelerated encryption with 3DES (168 bits) or AES (128, 192, 256 bits) and the IP Security Protocol (IPsec). An integrated firewall also helps seal off Big Drum's system from users’ production networks, and a configurable, stateful, packet-inspecting firewall protects against unauthorized access. Also, a dynamic packet filter scans for new connection attempts based on their addresses, ports of origin and destination, and it blocks any unwanted traffic.These security measures are needed because Big Drum increasingly performs needs-based maintenance of devices that can wear out faster and require closer monitoring, such as servo motors and drives. Increased temperatures in these components can indicate problems due to wear. If predefined tolerance values are exceeded, an automatic email warning can be sent to the service team, which conducts indicated inspections and can prevent previously unavoidable interruptions. Likewise, its continuous, online connection also allows Big Drum's technicians to assess and verify new equipment performance during startups, introduce optimizations during the warranty period and monitor deviations in filling volume or compliance with predefined opening and closing times of the valves or machine-specific cycle times during the final service phase."Our remote services increase users' system availability, and we've reduced fault-clearance times by 70%," says Itter. "With access to remote data, not only can we troubleshoot faster, but we're less expensive for our customers due to the elimination of travel costs."Unusual UsersLogically, once remote support shows it can be used securely and gains wider acceptance, all kinds of new, nontraditional and unexpected applications want to try it, too.For example, VRTX Technologies in San Antonio, Texas, uses a high-pressure flow technique called "dynamic cavitation technology" to clean cooling water in its users' HVAC, refrigeration and process cooling systems. However, as its skid-based systems multiplied worldwide, VRTX's staff realized they needed access to their PLCs on the skids, even though accessing them through most customers' IT infrastructures was often difficult. In fact, only 5% of VRTX’s users allow them to access data about their systems. After talking to several users, VRTX decided it needed a cellular monitoring system for its water treatment skids, which would enable it to monitor the equipment and give users access to their data without having to pass through IT systems."With direct access to our PLC, we felt we could monitor the operation of our systems," says Carl Steffen, VRTX's engineering services manager. "We'd be able to get alarms, see detrimental system operation and offer better and more informed technical support." Following a lengthy investigation, VRTX selected Netbiter EC220 gateways and Netbiter Argos Web-based, remote monitoring systems from HMS Industrial Networks."The initial reason we chose the Netbiter solution was the Netbiter EC220 gateway offered RS-485 interfaces in a small, well-built device," says Steffen. "It also offered preprogrammed GSM cards that would immediately work in many of our desired countries without the need to negotiate with local cellular carriers. Although the EC220 offered many standard I/O connections, we initially only interfaced with our PLC via the Modbus interface."Steffen explains that, at first, VRTX wanted to capture system information only once or twice per day. However, when its staff learned they could get information more frequently, they started finding benefits for its customers that they hadn't seen before. The ability to get more frequent information also led VRTX to change some of the treatment system’s operations, which also led to better treatment and a more consistent product."Remote monitoring has increased the value of our treatment skids," adds Steffen. "It enabled us to change system settings without needing to call the customer or send out a technician. It's also enabled us to troubleshoot failures and have the appropriate repair parts on site as our technicians arrive. It also gives us insight into the operations of our systems on a minute-by-minute basis, whereas, in the past, we only saw changes over long periods."Remote support benefits upper levels and end-user sites, and it inspires machine builders because it affects how they do their jobs says Kamalina Srikant, product manager for condition monitoring solutions at National Instruments, agrees. "Many engineers and technicians don't have to make as many trips to users' locations, which is changing their thinking and showing up on their bottom lines," says Srikant. "At the same time, traditional power-generation applications like coal are being joined by more natural gas, wind and solar, and all of these have a lot more equipment in critical paths and need much more remote monitoring."Also, based on its long experience in helping users gain insight into the health of rotating machinery and making business decisions to successfully implement predictive maintenance, National Instruments recently released its NI InsightCM Enterprise ready-to-run software with tightly integrated and flexible hardware options for online condition monitoring applications. It's used to acquire and analyze measurements, generate alarms, visualize and manage data and results and simplify remote management for large deployments of monitoring systems.Really Far Out SupportNaturally, once remote machine support is established between builders and users, it's tempting to see just how remote that support can be. Strong IP connections, VPNs and communication signals can go anywhere, way beyond land-based shops and factories, but they'd better have stellar support.For instance, Aquatic Engineering & Construction in Aberdeen, Scotland, U.K., recently worked with system integrator MAC Solutions in Redditch, Worcestershire, U.K., to improve monitoring of its ship-based tensioning machines, which lay down, install, transpool, recover and decommission flexible and semi-rigid cables and other products on the seabed for clients in the oil and gas, telecommunications and energy industries. Though it strives to make its equipment reliable and can resolve many issues with a phone call, Aquatic wanted to quiz its tensioners remotely and reduce service engineer call-outs to vessels."Aquatic provides equipment for the installation and spooling of flexible flowlines, umbilicals, cables, wire ropes and coil tubing," says Brian McRitchie, electrical manager at Aquatic’s marine workshop in Peterhead, Scotland. "Installed on marine vessels, this equipment can include anything from small tensioners to dual-tensioner systems, powered reel-drive systems or a fully modular carousel system with built-in tensioner," he explains. "Our customers hire this service, and expect the equipment and the personnel that operate it to perform reliably with minimal downtime. These ships can be located anywhere in the world, so if there's an uncommon problem with our machine, we need to resolve it quickly."McRitchie researched a suitable VPN router that Aquatic could integrate with its 85Te dual-tensioner system, and found MAC Solutions and eWon's 2005CD VPN routers and Talk2M Internet-based, remote-access, support and diagnostics software and service. Aquatic installed three 2005CD routers with dual local area network (LAN) and modem connectivity on the 85Te, and it monitors the tensioner via VSAT global, satellite-based Internet links or 3G cellular Internet connections. Besides having an Ethernet port, two of the three routers also have SIM cards, so, if a VSAT link is unavailable, Aquatic's engineers can access their equipment via 3G."To test and fully understand the VPN router and Talk2M, we trialed the system on one of our 50Te tensioners in the workshop at Peterhead," explains McRitchie. "Everything worked well, and, if we didn’t understand something, the team at MAC Solutions quickly provided us with the necessary technical support. With Talk2M, it’s as if the service engineer is physically onboard the ship, next to the machine or control cabinet, accessing the HMI display or PLC with a laptop."McRitchie adds that eWon and Talk2M give Aquatic a differentiator in the dual-tensioners it rents to marine vessel operators. "Some of our tensioner systems also have cameras installed, so local operators can see what's happening from their control rooms or booths," says McRitchie. "Talk2M also enables me to access this same Web-camera view. I can also screen-share with a user to remotely instruct them on the operational idiosyncrasies of the systems or how to navigate through the more isolated control and monitoring screens."
Offshore Performance Onshore

Figure 3: Statoil is implementing a $2.7-million, condition-based, performance-monitoring and maintenance system on its Gina Krog oil and gas platform in the North Sea, so onshore personnel can monitor its pumps, compressors and other mechanical equipment to maximize their efficiency and identify problems before they affect production.

[PHOTO CREDIT]Statoil and Emerson

Likewise, Statoil in Stavanger, Norway, is implementing a $2.7-million condition and performance monitoring system on its Gina Krog oil and gas platform in the North Sea (Figure 3), which will allow its onshore personnel to continuously monitor the new fixed platform’s critical and essential pumps, compressors and other mechanical equipment to maximize their efficiency and identify any potential problems before they affect production. This integrated, condition-based maintenance system provides machinery protection, prediction and performance monitoring of all critical and essential assets, and includes Emerson Process Management's CSI 6500 machinery health monitor and AMS Suite predictive-maintenance software, which is built on Meridium’s Asset Performance Management (APM) software.[pullquote]These software and hardware components will allow Gina Krog to deliver equipment health alerts and predictive diagnostics, which will enable the platform's staff to perform corrective maintenance actions to avoid unplanned shutdowns and maintain production while reducing maintenance costs. In addition, AMS Suite will aggregate all asset data to present a clear picture of overall asset health and performance, so work notifications can be created and fed back to Statoil's SAP enterprise asset management system for immediate attention. The platform also includes a wireless communication infrastructure based on WirelessHART (IEC 62591), which will support future enhancements to capture added equipment data on the platform at less cost than wired communications.Forecast: Mostly CloudyBeyond routine monitoring and maintenance, many remote support tools are gathering and archiving large enough amounts of information to undertake big-data efforts, such as advanced analyses of operations and improved machine designs.
Remote Support To-Do List

Every application has its own unique characteristics and requirements, but there are some common requirements, methods and tools that machine builders, integrators and others can use to establish remote support, monitoring, troubleshooting and even control.

  • Reexamine end users' business performance goals; evaluate machines, production lines and other equipment for achieving them; and evaluate how and where report monitoring and support could help.
  • Analyze existing network infrastructure, including ports, other physical and wireless connections, communication protocols, hardware components and software used.
  • Determine if network upgrade from dial-up modems to VPN and IP-based communications would improve remote support to end user and establish permissions and policies for granting access.
  • Coordinate deployment of remote monitoring components with existing machines and equipment, especially to make sure remote support devices don't affect production operations.
  • Implement appropriate level of security functions in remote support devices and networks, including secure IP and VPN links, configurable firewalls, packet inspections and dynamic data filtering.
"Just three or four years ago, remote support via VPNs was new for many machine builders and unknown to many others," says Mariam Gallegos, product marketing specialists for network security at Phoenix Contact. "Even those who were connected probably had remote desktops or public IP links that weren't very secure," says Gallegos. "Now, VPNs are tied to hubs and servers and then go straight to cloud-based infrastructures, such as our free mGuard Secure Cloud (mSC) service, which can also host clients' virtual machines and VPN tasks as needed, and maintain 24/7 security. Tying these functions together saves costs to our customers and also means they no longer need to maintain as much IT knowledge and can instead go to our website and connect to their operations data."Launched in February 2014, mSC is presently supported on PCs and Android devices, and will be available on Apple iOS in 2015. It uses IPsec security protocol and performs high-level AES 256 encryption with hashing algorithm, says Gallegos. "Remote support used to require dial-up or old-style VPNs that users had to maintain," explains Gallegos. "Now, we host sophisticated, simple and flexible VPNs for all customers using our devices. Their engineers just sign in, click to see the data for whichever of their machines they want and get back to making and shipping their products. This isn't machine-to-machine; it's more like technician-to-machine."
About the Author

Jim Montague | Executive Editor, Control

Jim Montague is executive editor of Control. He can be contacted at [email protected].