Remote Diagnostics: Machine Support Takes to the Internet

June 6, 2012
Machine Builders Have Difficulty Finding Competent Automation Support
As Control Design celebrates its 15th anniversary, we bring you a look back at some of the topics we have covered and that have generated the most discussion among our machine builder and integrator audience. This article is part of our June 2012 cover story, "We Celebrate 15 Years."
About the Author
Joe Feeley is Editor in Chief of Control Design and Industrial Networkingmagazines. He joined Putman Media in 1997 to help start up Control Design. He has more than 20 years of engineering and management experience in the U.S. and Europe in industries that include high-purity semiconductor products and other specialty materials that required direct involvement with the associated machine designers.For years, after-sales support posed unique problems for machine builders. It could be a lucrative gross revenue source, and it could be enormously expensive to execute.

As builders commissioned machines all over the country and all over the world, finding competent automation support could be a challenge. It often meant lots of frequent flier miles and a short-staffed home office with all the tech support out on the road. Machine users had trimmed technical support and relied more on the builders for help.

In addition, machine builders had been yanked from the security and knowledge about their own, often proprietary controllers into the Wild West of open, standards-based architectures that required new skill sets and difficult growing pains. There didn't seem to be anything in it for them.

We reported that it all started to change toward the end of our first 10 years in 2007 when the ability to remotely monitor, configure or even control machines moved from minimally helpful phone modem connection speed to the web as digital buses, industrial Ethernet, Internet protocols, fat pipes and VPN tunneling turbocharged new possibilities.

The results of an audience poll we presented early in 2009 confirmed both progress and a lot of work yet to be done. While 44% of the respondents said they provided some degree of web-based monitoring on machines, another 29% said customers wouldn't let them through the firewall, and 17% said they couldn't make the value-proposition argument fly.

Benefits Outweigh Risks
In an April 2009 OEM Insight column, "It's VPN for 24/7," Chris Roach, vice president of customer service and support at FKI Logistex (since bought by Intelligrated) in St. Louis, made his support of remote access pretty clear. "When there is a problem—or, preferably, when a problem can be addressed before it matures into failure of a mission-critical system—the best solution is often to allow individual OEMs access to their installations through secure, remote-access VPNs," he wrote.

He also noted, "Despite the encrypted protection that VPNs can offer, the question remains: 'Who gets the keys to the castle?' The integrity of the global supply chain rests on the integrity of everyone who interfaces with it; so who do you trust?"

Roach argued, "It's become clear that, with the acceptance of Cisco-encrypted VPNs, there is another solution. There is an increased trust in software and networks that hasn't existed before. Instead of hiding behind firewalls and login procedures, we should take advantage of the constant stream of encrypted data. A partnership with a trusted, qualified and authorized OEM has the benefits of security, trust and 24/7 monitoring services. The high level of complexity in today's systems and the need for constant uptime now can be matched with secure access through VPNs. The pipeline of information can be kept wide open, with constant access to data."

Later, in our June 2009 issue, we learned how a few builders approached the security issue in "PC Access Could Invite Hackers."

MoCo Engineering & Fabrication in Spokane Valley, Wash., builds lumber-handling equipment, and it also implements remote-access security via manual customer interaction. "We offer a remote access option to our customers," noted Loren Wernecke, electrical and hydraulic manager. "It's a web port device that uses open VPN tunnel technology, and the customer has to provide Internet access. Once the tunnel is created, MoCo can connect to any Ethernet device on the private side of the web port as if they were onsite."

Fewer Resources, Better Service
In our November 2010 issue, "Machine Tools on the Internet," by Markus Schmolz, manager at Schwäbische Werkzeugmaschinen (SW), machine tool manufacturer in Schramberg-Waldmössingen, Germany, detailed his company's comprehensive monitoring services that gather and report machine lifecycle cost data for its customers.

"What is new is that this global data acquisition allows us not only to determine the ideal intervals for use-dependent maintenance, but also to provide customer-specific, matched solutions with very useful background information for the production," said Peter Siegel, responsible for the development of online services at SW. "This gives the management an operational transparency that it doesn't have otherwise."

More than 65% of the SW machines sold (in 2010) connect to the Internet. "However, by no means do all customers use the resulting information fully," Siegel said at the time. "Many still envision reactive maintenance service as the only possible use, and only when a fault has occurred."

Several years ago, SW equipped its spindles with sensors that could check status without removing them from the machining head. "The maintenance staff no longer needs to wait for our alerts and updates. They can work with the data themselves," Siegel reported.

The data that SW provided gave the machine operator and its customers security. "All we need is Internet access," Siegel explained, "and our customer receives the complete services."

Although this might appear simple, in practice, Schmolz wrote, the execution frequently encountered hurdles. There is general mistrust of Internet connections in production. This also means that the IT department is involved. "In the past, IT departments were not much concerned with numerical controllers and other onboard controls," Siegel said. "Today, controllers are part of the global network, so it is necessary to involve the IT department and to open the required communications paths and establish some mutual trust within the company."

In an August 2011 case history, "Remote Service via VPN," Stefan Winzinger of machine builder groninger USA told us how his company realized that modern communication technologies could help it to provide better support to more customers without overextending its internal technical resources.

"Our company developed Remote Video Service (RVS), which provides a customer-initiated secure virtual private network (VPN) connection between a machine at the customer's facility and groninger's secure internal service network," wrote Winzinger, an electrical and programming engineer. "We use a VPN network address translation (NAT) router on both the customer and groninger service network sides to initiate and maintain a secure encrypted VPN connection. Remote Video Service is offered as an option when purchasing a new machine, or an upgrade for existing machines with Ethernet capability. The remote service function allows groninger to provide better support to more of our customers."

If customers have a technical problem with a machine, they can call the regional groninger service office in Charlotte, explained Uwe Klaus, service team leader. "The groninger engineer on duty will ask the customer to initiate a VPN connection by operating a key switch. A VPN tunnel then connects the customer to that regional groninger service office through the Internet. The machine network—which can include PLCs, HMI, servo controllers, and other Ethernet devices—is at the customer's end of the VPN tunnel."

Service engineers and specialists at groninger can access these devices to see live program status, make changes if needed, back up or restore programs, create new recipes, and provide machine or software updates or revisions. "The remote service function also provides the capability for the customer to connect a remote-control camera to the machine network," Klaus added. "In addition to actually seeing live PLC and I/O status, groninger engineers can see the machine from an operator's perspective by panning, tilting and zooming the remote-controllable camera to the problem area."

Typically, it's the customers who control their VPN key switch and initiate the connection to the groninger service network, Klaus explained. "However, this isn't the only scenario. In some cases, customers could choose an always-on VPN connection. Machine access for ongoing remote preventive maintenance is a good reason to have an always-on connection. In this mode, basically all analyzing, recording and monitoring items can be done automatically if the customer chooses."

Cloudy, Not Stormy
In February and October 2011, we ran responses to reader questions about web-based diagnostics, one of which asked for experience-based opinion about third-party, cloud-based services for remote troubleshooting.

We heard from Keith Jones, PE, president of system integrator Prism Systems, who wrote, "We recently had an OEM approach us with this very need. The OEM has machines distributed across the U.S. and in a couple of other countries. They needed production and status information transmitted to a database from each machine each hour to display on a website for their users. They wanted alarm information transmitted immediately, and they also wanted the ability to remotely modify parameters and set points."

Jones said the OEM would sell a number of these machines to each of its customers, which would then place the machines in remote locations and maintain them. "Our customer, the machine builder, wanted to collect the information and sell a subscription to the data to its customer," Jones explained. "Having real-time data and the ability to remotely interact with the machines provided the OEM with a value proposition that would increase demand for its machines and create a new revenue stream."

The technology to do this is readily available, but it is probably over the heads of most machine builders, Jones indicated. "Unless you have software people on staff, I suggest getting a software company involved for the truly technical portions. Building a robust system of this nature is complicated and almost always will require custom software. For our customer, we developed the software for them so that they own it. The revenue they get from selling subscriptions to the data is paying for the development cost, and their sales growth as a result of the new features is an added bonus. They have taken a balanced approach split between a cloud deployment and in-house development. They are paying for external hosting of the system, but maintain full control of the development by partnering with a software company. For them, this has worked out very well."

Progress in use and and acceptance of these technologies marchs ever-forward. Our November 2011 cover story, "Connect Four...," showed how far in some unique cases, now that cellular 4G bandwidth and apps appear to have a place at the remote monitoring table. "I do design work for several machine builders," said Matt Youney, owner of Youney Instrumentation and Control Systems Engineering in Lake Worth, Fla. "I have projects deployed all over the world. This morning, I was making updates to a semiconductor die-handler machine in Switzerland while drinking my coffee in Florida. With remote access, my customers can't tell if I am in my office, or on my sailboat in the Florida Keys. As a contractor without remote access, I wouldn't be able to do my job efficiently. My customer base would be limited to local customers, most of which moved abroad in the past 10 years. I need these tools to compete in the global economy, and hopping on a plane to fight a fire is really a last resort."