Tending the flock

Bob Waterbury, Senior Editor
 
Ask an exhibitor at PackExpo, IMTS, or the Converting Show, among others, what sets his machine apart from the competitor across the aisle. Chances are that exhibitor will mention after sales service and support—including product and applications knowledge, training, documentation, spare parts, software and hardware upgrades, and diagnostics and troubleshooting assistance—as major difference.

In fact, suppliers and OEMs are beginning to recognize that service is an increasing contributor to the corporate bottom line. Downsizing and outsourcing are enabling manufacturing companies to concentrate on their core competencies. This is thinning the engineering ranks at customer manufacturing sites, however, and creating a growing need for outside diagnostic and troubleshooting expertise that can improve machine performance and maintainability—and overall customer profitability. Enter the OEM in an expanded role.
 
Opening Up to Remote Diagnostics
"If we provide a PC as part of our machine solution, we automatically include an industrially hardened modem and remote-access software to allow us to troubleshoot their equipment," says Ken Grimes, vice president of the manufacturing division, Carotek, Matthews, N.C., (www.carotek.com). "Likewise, if we provide a PLC, we'll include an industrially hardened modem that can be tied directly into the programming port of their PLC. Using a dedicated line, we can then call in and diagnose their system inexpensively.” The alternative may be $800-1,000 per day plus travel expenses to put an engineer on site. Furthermore, that remote diagnostic capability is not expensive. “We're talking about less than $300 total for hardware and software plus a few dollars for air time—for either a PC or PLC solution," says Grimes.

As a major machine buidler, rebuilder, and systems integrator to both discrete and process industries, Carotek even provides e-diagnostic, troubleshooting, and monitoring services to customers using machines from different builders. "If customers provide us access to the code, we can diagnose, troubleshoot, and even change machine operating parameters from our own offices—either via the web or dial-up line," says Grimes. "We also have a special support services group that operates under contract to perform calibration, repair, and troubleshooting of all types of industrial controls such as smart valve actuators and instruments beyond the brand labels that we commonly support. This is done both remotely and on-site per customer agreement."

One of Carotek's programmers was on a job recently when a call came in from a large distribution warehouse about four hours away. The distribution center had experienced a lightning strike that shut down nearly their entire operation, and wanted to know if the programmer could quickly get them running again. Because Carotek had installed a modem on their PC and PLC, the programmer was able to plug his laptop computer into a phone line and reload two essential drivers that put them back into business without further delay—saving them many thousands of dollars. 

"One of the things we've noticed is that software support is becoming so complex that end users are no longer able to do that themselves," says Grimes. “As a result, we sell remote diagnostic services on a scheduled basis to many different customers. It is a growing part of our business operations—and our bottom line."

Great Expectations
Today's machine shop can’t be satisfied with a fabricating center that simply punches holes in 15 mm metal or that cuts complex shapes in 18 mm plate. It needs a machine with a flexible controller that can interface with sophisticated CAD/CAM computers, run the latest software revision, and feed production data to plant-wide ERP or MES systems. And when trouble occurs, it needs a machine with built-in diagnostic tools to help resolve problems and minimize downtime.

Machine builder W.A. Whitney Co., (www.wawhitney.com), Rockford, Ill., known for its machine tool plate technology, is helping its customers achieve these benefits with an open CNC solution.

“When we set out to improve our 42-tool 3700 ATC punch/plasma fabricating center, we knew we needed to incorporate open architecture in the control scheme,” says Al Julian, marketing manager for punch/plasma fabricating machines at Whitney. “Open-architecture CNCs give us long-term flexibility for designing interfaces, for integration, and for remote diagnostics, which is exactly what fabrication shops demand.”

While CNCs have a reputation for long-term stability and reliability, even the best machines need service to maintain 95% or better uptime. In the past, that often meant a site visit for a service call. Now, with a modem card in the re-designed 3700 ATC and remote-access software installed, OEM technicians can access the fabricating centers’ data from across town or around the world at any end user site.

Whitney customers can equip the Open Factory CNC solution from GE Fanuc Automation (www.gefanuc.com) with remote-access software such as Symantec’s PCAnywhere. This software allows a technician in Rockford, Ill., or anywhere else with security clearance to dial in to the machine. Because the system is echoed onto the remote computer, the technician has access to all of the troubleshooting functionality built into the CNC, as well as tools normally available only at the OEM’s site. 

Once the diagnosis is complete, on-site maintenance personnel from the plant can generally complete repairs. The open architecture and remote-access software allow the CNC owner to take advantage of the OEM’s expertise without incurring transportation and lodging charges normally associated with a service call. More important, the fault can often be diagnosed and repaired, and the machine returned to service in less time than it might have taken the technician to pack and head out to the airport.

“Downtime can cost CNC users thousands of dollars per hour,” Julian says. “Our customers need and expect immediate response. With remote access, we can provide that. No field service person could possibly arrive as quickly as we can dial into a machine.”

Whitney also relies on the CNC’s screen display function to help with remote diagnostics. Since a service engineer might not know precisely what software applications are installed on an open CNC, the engineer can call up the CNC display for a common interface to all controls–independent of any additional software. And, even though some custom interfaces don’t provide direct access to certain CNC functions, the screen display allows access to all setup and maintenance functions. “A tool like this helps speed diagnostics when customer-installed applications are added to machines,” Julian says. 

In addition to raw numbers and positioning data, operators have access to screens developed in Visual Basic to show diagnostic data on the punch and ram as well as information on active tools, work clamp positioning, and performance parameters from the plasma cutter. Furthermore, shop floor control data can also be fed into the user’s plant-wide MES or ERP system, or stored for later retrieval in standard database formats.

Alarms, such as work clamp failures or tool-wear cautions, can trigger local signals, such as a blinking display on the CNC or other on-machine status indicators. On a networked machine, the potential exists for alarms to activate pagers, e-mail notification systems, or automated voice mail messages. The CNC might even call Whitney directly to “ask for help.”

“If there’s a machine fault, we can find the reason quickly,” Julian says. “Even non-CNC control problems can be diagnosed, though, thanks to the logging features that allow us to view input and output data.”

Grab the Data; Save the Day
Ariel Corp., Mt. Vernon, Ohio, (www.arielcorp.com), builds both standard and custom compressors for the natural gas industry, and offers its customers both web and dial-up diagnostic access. In addition to the usual product and applications information, its web site contains useful data on how to select between rotary and reciprocating engine solutions, and even offers a calculation feature to help determine horsepower requirements for particular applications.

"There's a lot of initial customer resistance to allowing remote access to their operation and maintenance data," says Michael Porter, engineering programmer for Ariel. "However, once they understand what it can do for them, they become quite enthusiastic about e-diagnostics and troubleshooting."

Porter explains that there are no real technical obstacles to remote third-party diagnostics, only political and organizational impediments. For those end users that appreciate the benefits, Ariel even offers online monitoring. This service provides a complete periodic snapshot or data set of their operation. This gives them a running picture of both operation and maintenance parameters that can be used to smooth production, and identify and correct potential problems before they arise.

"Industrial modems, PCAnyWhere, and LapLink provide the necessary tools," says Porter. "And we can get all the data we need. The only problem is that data collection technology has made quantum leaps forward in the last several decades, while the ability to handle, manage, analyze, and apply it has not kept pace. Fortunately, our customers are beginning to recognize the importance of maintaining a comprehensive production data base, and are looking to us to help extract the e-diagnostic and troubleshooting information they need to make their operations more cost-efficient and effective."
  
Dare to Be Different
Hooper Engineering, Sarasota, Fla., (www.hooper-eng.com) builds horizontal form, fill, and seal packaging machines for the food and medical industries. Its flagship is the N-2500 machine that has been adapted to many different applications. "We were looking for a management tool that we could add to our machine and set it apart from the competition," says Hooper Engineering president John Ford. "And, through information contained in Control Design magazine, we found a PLC-based embedded web server that fit our needs. Not only does embedding this device in our machines put us above the competition functionally and diagnostically, but it allows remote access to machine functions and data—without any special programming."

Hooper has more than a year of experience now with the embedded web server that’s part of Schneider Electric’s Transparent Factory program, and has had several opportunities to test the effectiveness in locating and resolving problems. "Using any web browser, we simply type in an IP address and it allows us to access information," says Ford. "This information may come via a web camera, manuals, direct production and maintenance data, safety circuits, or I/O signals. Using the information, we can diagnose conditions and change parameters, or even change operating procedures.” Because the modular web server resides directly on the machine, the customer doesn't even need a computer sitting beside the machine. “Just several days ago one of our machines began indexing incorrectly,” adds Ford. “Alerted to the fact, we accessed the machine through the web server and found the module that had to be fixed or replaced." 

Advanced Engine Technologies (www.aetco.com), San Leandro Calif., builds engines and machines used in the transmission and storage of natural gas products. There was some initial resistance, but AET’s customers are now beginning to appreciate the advantages of e-diagnostics. 

"With any of our fully integrated systems designed for the natural gas industry, we now offer primarily dial-up remote diagnostic capabilities," says Alden Cramer, AET’s director of software engineering. "Most of our clients are in Southern California, and they are required to collect and report environmental compliance data to the Air Quality Management District.” This would be difficult to capture and report economically without the use of remote monitoring and diagnostic capabilities. “In fact, we now offer remote diagnostic services to customers independent of product sales—even to those who don't buy our integrated systems," adds Cramer.
 
Semiconductor Fabs Lead the Diagnostics Charge
The semiconductor industry is perhaps the overall leader in use of third-party remote diagnostics at this point. Industrial OEMs have been providing semiconductor fabrication equipment with this type of capability for at least 10 years now.

Sputtered Films Inc., (www.sputtered-films.com) in Santa Barbara, Calif., a machine builder for the semiconductor industry, says it began to use a modem line to remotely access and talk to its equipment almost a decade ago. Today, SFI uses an ISDN line to bring up a customer's user interface in real time and then access error and alarm logs. In so doing, it can walk the customer through the corrective actions in real time.

Tom Plotkin, SFI director of automation and software and originator of the application, initially used QNX as the operating system software. All commands were menu-driven. Communications through a graphical user interface now require greater bandwidth and an ISDN line, but even back then, he had almost complete access to the machine and could analyze, diagnose, troubleshoot, and even collaborate with others on the proper corrective action. The remote diagnostic system also allows software upgrades or modifications to be downloaded as needed. 

Semiconductor solutions providers such as Berkeley Process Control, (www.berkeleyprocess.com) are using a server within the fabrication facility to connect to their tools and collect performance data. "Instead of using separate phone or ISDN lines, Internet technology let us implement seamless, inexpensive network connectivity," says Berkeley president Paul Sagues. "This really defines the future of e-diagnostics, and can be expanded to cover the varied facets of e-manufacturing as well."

KLA-Tencor, (www.kla-tencor.com), San Jose, Calif., is also using remote diagnostics technology as the foundation of its iSupport program that enables off-site technical experts to access its tools at customer sites, operate them in real time, and diagnose and resolve problems as they occur. This is accomplished through a secure, online connection that is controlled by the customer at all times. iSupport increases tool uptime, enabling chipmakers to achieve significantly lower cost of tool ownership, and improves device yields.

Data collected from iSupport is kept in the integrated diagnostic monitoring server (iDM), which KLA-Tencor installs directly in the customer's fab. This intelligent server continuously monitors tools that are connected to iSupport and alerts the customer and KLA-Tencor when a problem is detected. The server acts as a database by storing and analyzing tool data for reporting and trending. And because this is controlled by a web browser, customers can access any information stored in the server.

"Since its introduction last year, our iSupport program has been adopted by 13 of the top chipmakers worldwide," says Mike Allison, general manager of KLA-Tencor's worldwide support operations. "Customer data shows that more than 85% of unscheduled service calls can be assisted online, and 33% of unscheduled calls can be completely resolved online with iSupport.” By providing e-diagnostics technology to other leading equipment suppliers as well, the company is making even greater progress in helping mutual customers minimize their manufacturing costs.

Even Sematech, (www.sematech.org), Austin, Texas, the semiconductor industry research and development arm, is spearheading a special program to develop standardized remote diagnostics technology to be used to achieve increased production efficiencies throughout the industry. When the semiconductor industry signals a general turnaround in the national economy, it plans to be well positioned to take advantage of e-diagnostics as a tool to ensure market dominance.

Remote Diagnostics Changes Relationships
As we have seen, e-diagnostics and troubleshooting is becoming a major support service after the sale. In fact, a number of companies are organizing to provide this service as an independent profit center—even to companies that are not customers for their hardware and software. Suppliers are gearing up to provide this support as well, both directly to end-users and to the OEMs and their customers. 

"The web, Ethernet, and TCP/IP have opened up a lot of new challenges and opportunities for us and our OEMs," says Jeff Meyer, technical services manager for Omron, Schaumburg, Ill., (www.omron.com). "We have the tools in place to give OEMs and systems integrators the ability to remotely access, diagnose, troubleshoot, and program systems remotely just about anywhere in the world using web or dial-up capabilities. They, in turn, are offering that capability in support of their machines operating in customer locations around the globe. From thousands of miles away, an OEM can check the number of hours on a machine, see if problems exist, and offer maintenance solutions without ever stepping outside the office or delaying to make an on-site inspection. That adds value to the end product and increases OEM service revenues at very little expense."

Rockwell Automation also sees light. "Rockwell is offering its OEMs the tools and technology to support customer needs for remote diagnostics either directly or through Rockwell resources," says Pat Babbington, vice president of customer support, Rockwell Automation (www.rockwellautomation.com). "A remote diagnostics modem kit allows OEMs to build e-diagnostics support capabilities into the machines they supply to their customers. Or, if they wish, they can upload customer programs to us for review and diagnosis of a problem. Some of our products even allow us to diagnose faulty boards in drives and other equipment—whether our equipment or someone else's. It's evident that OEMs are being challenged to supply higher and higher levels of diagnostic and maintenance expertise, and we want to support that any way we can."

End users are concentrating more on core competencies. Thus, they are getting out of supply chain management, logistics, service, maintenance, etc., as much as possible and concentrating on manufacturing their primary product(s). Because many of the support functions are being outsourced, industrial machine builders are challenged to supply increasingly higher levels of diagnostics and maintenance expertise.

"The end user skill set is changing," says Babbington. "Operators have been upgraded in many cases to assume the role of applications and product specialists. Technicians are trained to operate machines and perform basic maintenance. But there is a growing gap when it comes to engineers who understand and can relate to the overall role of machine operation and maintenance, and who are prepared to diagnose and resolve problems as they arise.” This expertise is increasingly outsourced, and the responsibility often falls upon the OEMs and integrators to identify and resolve production problems."

In addition, e-diagnostic and troubleshooting services are becoming powerful weapons in the increasingly competitive global marketplace. There’s a growing cadre of companies that use their machines as marketing tools to ensure continued customer loyalty and long-term business relations.

"Some of our customers for packaged process control solutions offer our equipment free of charge to their end users in exchange for exclusive long-term product usage," says Carotek’s Grimes. "For example, we fabricate a process control skid for one large customer that sells polymers to the pulp and paper industry. The skid includes pumps, valves, HMI, instrumentation, and e-diagnostics valued at $135,000. He gives that package free of charge to his paper customers so they will use and track consumption of his polymer chemicals.” The process package is used to accurately inject the polymers, monitor the process remotely, and trace the polymer usage to improved quality and quantity of paper product. “We then collect the SCADA information that shows polymer usage, and track it against various production parameters,” says Grimes. “This provides clear evidence that the end user can present to show that his paper product is superior in quality to that of his competitors. It's a win-win for everybody."

AET thinks along similar lines. "We also have major energy customers that may buy an engine generating set and then rent it out to the end user," says AET's Cramer. "That way, their capital investment is kept low, and they can concentrate on producing power. And with e-diagnostics, the operation and maintenance can be monitored and corrected efficiently and economically."

Remote Diagnostics: Support's Leading Edge
After-sales service and support encompasses a number of remote or web-based facets including software updates and patches, technical notes, application notes, CAD drawings, and manuals. However, the ability of a machine to automatically dial up an OEM, either routinely or in emergency, and report machine operation and maintenance information is a major advance in machine control and design. And, Omron's Meyer observes, "It is coming at a time when that type of service is greatly needed by end users who are constantly challenged to reduce all support activities in order to concentrate on their core production competencies."

As our examples indicate, OEMs that embrace this concept enthusiastically are finding that it pays not only in terms of increased customer loyalty, but in terms of new and/or increased revenues. And their end user customers are reaping greater production efficiencies and reduced maintenance worries in a global market where even a small advantage can mean the difference between profitability and financial ruin.