Maximum machine uptime is the ultimate objective of modern manufacturing. That’s why we cover topics such as simplified mechanicals, standardized controls, vibration analysis, and remote diagnostics. These are thoroughbred technology applications that industrial machine builders everywhere rely on to help improve their machines.
Diagnostic sensors and predictive software are a growing part of these approaches to increasing uptime. After all, it’s far better to fix problems when they’re small and easily serviced than it is to fix a machine in the field after it begins acting up or breaks, and then holds up a customer’s production line.
Diagnostics at Work
FLSmidth, a builder of cement machinery in Bethlehem, Pa., had been plagued by users calling its support technicians for troubleshooting advice. These customers didn’t have staff capable of troubleshooting electrical and electronic problems, so FLSmidth’s technicians were spending two to four hours per call helping customers identify their problems.
As a result, the company partnered with Rockwell Automation to develop a new pneumatic transport system with more built-in diagnostic capabilities. The HMI terminal (See Figure 1 below) has diagnostic screens and alarms that describe exactly what’s wrong when a problem arises. Typical problems the system finds are motor-trip conditions, valve-coil failures, blown fuses, power-supply failures, and so on. Customers can fix most of these problems themselves.
Each machine’s normal operating conditions can be analyzed by SPC or similar software to look for trends. Jeff Cawley, engineer at Northwest Analytical, an SPC vendor, says SPC can detect problems before they become problems. “SPC is applicable to monitoring any parameter, such as measuring shaft load and bearing wear,” he explains. “SPC differentiates signals from noise, is more sensitive to equipment and process deterioration than simple inspection systems, and enables a dynamic, intelligent system response.”
Consequently, specific problems can be solved, such as a suspected vibration. The more advanced software at the central server can fully analyze spectral data coming from the remote machine and render a diagnosis. Your service engineers can also access a machine’s complete history, and run specific analyses to help troubleshoot a user’s machine.
Problems in multiple machines also can be analyzed. For example, let’s say that motors are failing on a regular basis, but only in certain machines. An SPC analysis can find the problem, adds Steve Wise, director of statistical methods at InfinityQS. “The analysis is trivial,” he claims. “The challenge is gathering the right kind of data from the field. If each failure is recorded, the system, using Pareto charts and simple run charts, is able to establish a personality for each machine. To see if any one machine was acting differently than similar machines, we can add multiple machines to the study, and organize the graphical representation to display machine-to-machine differences. Meaningful correlation studies require tracking specific suspect parameters with specific failure modes.”
Wonderware’s product manager, Nancy Venable, adds it’s possible to define a variety of downtime conditions based on triggers coming from one or more machine inputs. “As machines operate, all this information is collected and stored,” she says. “In time, trends from the database appear that determine the leading causes of downtime, when they occurred, why they occurred, what shifts have the highest downtimes, which machines have the lowest downtimes, and why. This information can then be used to determine patterns of failure and appropriate preventive maintenance procedures.”
Some devices can even troubleshoot themselves. Cummins, a builder of generator sets and transfer-switch controls in Columbus, Ind., employs controls that self-diagnose and diagnose problems remotely. “We check failure modes of engines, circuits on the engine, failure of certain control functions, excitation system failures, output power failure modes, output breaker failures and many more variables,” says Gary Olson, technical counsel at Cummins. “We sell about 20,000 systems a year with this capability. We’re about to start doing this on consumer products, which will make this number jump to more than 100,000 machines per year.”
Combining a modern fieldbus-type network with smart sensors makes it easier to implement these on-line diagnostics. Ron Newlun, controls manager for AEC, a plastics manufacturing equipment builder in Wood Dale, Ill., combines DeviceNet with smart I/O. “The intelligent network and I/O provide diagnostics that allow us to turn our HMI touchscreen into a genuine maintenance tool for our customers at no extra charge,” he says. “The I/O’s built-in diagnostics let maintenance personnel know what to bring to any given location, whether it’s a screwdriver or a replacement part. These diagnostics also help improve production scheduling, plan preventive maintenance of mechanical components, and a host of other things.”
Every Rose Has Its Thorns
As you might expect with a still-emerging technology, not everything is perfect in diagnostics land. “On some of our higher-end equipment, it works out fairly well,” reports Dan Stirpe, control engineer at Dauphin Graphic Machines, a builder of machinery for commercial printing and newspapers, located in Elizabethville, Pa. “In other cases, it was the equipment we added—to read the diagnostics and relay the information to operators—that caused the failure. We have far more failures in status reporting than we ever had in the devices themselves.”
Stirpe adds his company doesn’t access diagnostics over the Internet. “Most of our customers aren’t willing to risk unwanted access into their networks,” he explains. “We generally have a dedicated phone line run into our control consoles, and access via a remote-access server (RAS). Remote diagnostics work fine for some things, but often it’s not that useful. If you know there is a hardware failure and which piece of equipment on the system is causing the problem, then you can access it remotely, and determine why it’s not working. If the problem is communications or a high-level control issue, then it’s nearly impossible to diagnose remotely.”
Nonetheless, diagnostics seem to help sell equipment. “I find it amusing,” he adds wryly, “that the systems that use ‘remote diagnostics,’ ‘self diagnostic,’ and ‘easy to operate’ as part of the selling features are the same ones that have the most failures and the longest learning curves.”
Operators at this HMI terminal in a cement plant see diagnostic screens that point out problems. If they need help, an engineer from the machine builder can dial-in remotely and view diagnostic information to help with troubleshooting.
Overcome Resistance
Forward-looking companies like Dauphin Graphics often discover pitfalls and problems long before anyone else. Still, Stirpe’s observations are right on the mark. Any builders embedding on-board diagnostics must be careful that they don’t create more problems than they solve.
However, once somebody proves that a technology works, vendors discover that new market niche, and products emerge to fill its void. For example, the International Sematech (ISMT) consortium of semiconductor IC makers, equipment suppliers and software vendors, has climbed aboard the diagnostics bandwagon. ISMT defined e-Diagnostics as a model for predictive maintenance, which should help drive down equipment support costs, reduce mean time to repair, and cut machine downtime.
Likewise, diagnostics are being built into motors, drives, controllers, sensors, and other components as a matter of course these days. A study by Venture Development Corp. reports that 89% of all pressure sensors are now “smart,” and VDC forecasts that total to pass 91% by 2010. A smart sensor allows remote calibration and ranging, self-calibration and digital communications, and, most importantly, it performs self-diagnostics. If pressure sensors are following this path, it stands to reason that most other sensors also are getting smart. Most drives and MCCs (See Figure 2 below) already contain all the diagnostic information you can possibly use, and a variety of inexpensive vibration sensors are readily available.
For instance, GE Fanuc builds diagnostics into its CNC machine-tool controls. “Our CNCs now have a multitude of built-in functions to aid the maintenance team,” says Mark Brownhill, GE Fanuc’s machine tool product manager. “Operation history records every button pushed and screen visited. Alarm history records every message from the CNC or the machine tool interface. The machine ladder and the status of every input and output can be viewed on the CNC screen to help troubleshoot problems. The servo system’s tuning performance can be adjusted automatically or manually using a virtual oscilloscope. With these integrated ‘black-box flight recorders’ and diagnostic tools, a maintenance team can understand all the conditions leading up to a problem, as well as the final condition. This allows the effects of machine crashes or mis-operations to be considered along with the resultant product failure. All this diagnostic information can be viewed at the machine or across the web, allowing service engineers to arrive at the site with relevant parts and knowledge for the fastest mean time to repair.”
The Alvey Systems unit of material handling equipment builder FKI Logistex in St. Louis, Mo., uses smart I/O and DeviceNet. It recently installed an automated material handling system at News International Ltd, London, U.K. Ed Zahler, Alvey’s projects director, says the bus provides a high level of diagnostics. “Since DeviceNet is wired directly into a controller, the PLC can monitor everything that goes on,” adds Zahler. “If a photo eye goes out, the PLC’s diagnostics pick up that it’s not functioning right away.”
Because the PLC is tied into the main control system, information about system status, including maintenance issues, is graphically displayed to an operator in real time, so problems can be addressed immediately.
Many components and systems, such as the motor control center, come equipped with condition monitoring and diagnostics built in.
Software Assists Uptime
One big difference today is in the software adding uptime in these applications. Last year, we saw that a machine builder or SI usually would have to develop most of the software itself to obtain data, analyze it, and present it to customers and maintenance and support staffs. Today, you can buy most of the necessary software.
For example, GE Fanuc Automation has added diagnostics to its Proficy automation software to track downtime, setup time, speed losses, minor stoppages, part-yield issues, and their root causes. “Many end users implement the system to drive their own efficiency,” says Brownhill, “but it’s also being used to monitor performance-based contracts, allowing users and machine builders to have a common understanding of machine availability and the true reasons for any losses.”
In addition, performing vibration and spectral analyses “on the spot” is getting easier all the time. National Instruments (NI) says you can do it yourself. “The assumption that machine builders can’t or won’t do analysis and diagnostics in the machine doesn’t completely reflect the state of condition-monitoring equipment today,” says Chris DeFilippo, NI’s sound and vibration product manager. “Today, the ability to do condition monitoring is being integrated with the same equipment that does control. Our LabView software has statistical tools such as wavelet analysis that machine builders can use to track and identify problems such as bearing impacts.”
However, this means you have spend plenty on expensive software, right? Apparently not. “LabView starts at $999 for a base development platform,” he says, “but most developers add functionality for a development system, which costs $3,500 to $6,500. However, there’s no cost to deploy a LabView executable that simply does data acquisition and transfer for an unlimited number of systems.”
On the other hand, if you don’t want to develop vibration analysis software yourself, you can easily buy it. NexJen Systems, a supplier of test and automation systems in Charlotte, N.C., developed its VTS condition monitoring system, which is suitable for machine builders. Based on LabView software and NI’s PXI data acquisition system, VTS acquires real-time waveform data from a machine, analyzes it for problems, and predicts them.
VTS performs analyses such as waterfall plots, spectral maps, shaft centerline, and orbit plots. These essentially examine the spectral “signature” (See Figure 3) of the machine, and look for pending problems in motors, bearings, and other rotating components.
Similarly, Emerson Process Management got into the “on-the-spot” machinery analyzer business last year with its Machinery Health Analyzer, a system that uses vibration analyzers on motor-driven pumps. The vibration analyzers send data to a central processor (See Figure 4 below) for further analysis. This system is designed to support multiple machines from a central location in a plant.
Emerson’s software runs with its asset manager package, typically used in process control applications. However, for machine builders that build big iron such as entire assembly and production lines or systems with multiple motors and pumps, the Emerson system might be practical. This system also might be practical for machine builders that make more modest machines, but sell them by the dozens and hundreds. Emerson doesn’t advertise or promote that type of use, but there seems no reason why its asset manager and diagnostic software can’t be located in your home office to analyze vibration data sent from individual machines worldwide.
Hey, Machine: Phone Home
Many diagnostic systems allow machine builders to communicate via the Internet to machines located anywhere. However, allowing technicians to analyze a situation and help a customer troubleshoot problems is only half the solution. The other side is enabling the machine to phone home, and send regular reports and diagnostic data for further analysis.
One way to diagnose machine problems is to take a laptop into the field, connect to machine sensors, and run the analysis on the spot.
There’s no reason why advanced analyses can’t be done in a central location, where the cost of analysis software can be spread over the installed base of machines. A cheap data-gathering device in the machine could send data via phone lines, the Internet, or a cell-phone connection to the central office. Emerson’s system allows remote devices to communicate to the central processor via phone lines. They could be calling a processor in your home office just as easily. The LabView system can have executables installed in machines at no extra licensing cost.
Cummins has remote capability, for example, but doesn’t use it often. “We can do remote diagnostics, but not many customers or service technicians are fluent in the practice,” notes Olson. “It requires an infrastructure at the home plant that’s set up to immediately respond to the request because, once you connect, you can't wait around for days to do the service or adjustment.”
Olson adds that Cummins has the necessary infrastructure. “We use it occasionally when there’s a big problem and no other way to fix it. Typically, we get the customer to set up a phone link to the control system, and then we set up a parallel phone line to talk to the customer or technician while we’re looking at the control remotely.”
A recent CONTROL DESIGN survey found that 26% of readers can’t consider using the Internet for remote diagnostics because your customers won’t let your machines through their firewalls. With modern communications technology, that isn’t a problem anymore; just let your machine phone home.
“A cell phone network will suffice,” says Somnath Deb, chief technology officer for Qualtech Systems, a conditioning monitoring company. “A ‘smart’ machine now only needs to send a message, rather than tons of data. Besides, a ‘hidden’ network connection is undesirable because it bypasses the safeguards a customer has in place to protect its system.”
Machine vibration data can also be gathered remotely and processed at a central location, such as your home office. This engineer is looking at a motor-driven pump analysis.
In addition, as I pointed out in “Handheld HMI: The Wait Continues,” CONTROL DESIGN, April ’06, handheld PDAs and cell phones have come a long way in the past few years. Today, you can buy a $500 PDA, cell phone, or half-size “mini PC” with 128 MB of memory and a 300 MHz processor, which is enough to run HMI/SCADA software. HMI/SCADA software is another $300-500 or so. A shrewd negotiator could drive the OEM cost down below $500 per machine, which is about the cost of one service call.
Consequently, you probably could set up an independent, remote, data-gathering system in your machine for about $500, including hardware and HMI/SCADA software. The system wouldn’t interfere with your normal machine controls. It would “mine” whatever condition monitoring data was needed from the PLC, MCC, network and I/O, package it up, and ship it all back to your home office once or twice per day. If it detected an alarm condition, it could notify the local operator and the home office.
Sounds far fetched? Hardly. Collecting condition data remotely and processing it in a central location is being done every day by machine builders and other OEMs. For example, InfinityQS International, a vendor of SPC software in Chantilly, Va., reports that its clients include Ball Corp., a manufacturer of packaging machines; Raytheon, a maker of silicon wafers; FMC, a manufacturer of food processing equipment; and Phoenix Automation, an on-line inspection company for the automotive industries. All of these connect multiple machines—sometimes hundreds in a single plant—to servers that perform statistical analyses on parts, production, and machine performance.
Steve Wise, director of statistical methods at InfinityQS, says the data-gathering requirement at each machine is minimal. “There needs to be some form of software that will write the desired information to a flat ASCII file,” he explains. “Our software can be set up to automatically strip data from that file on a regular basis. Alternatively, an HMI software developer can write data directly to our ActiveX subgroup component.”
Also, HMI/SCADA vendor Wonderware has a software suite with asset management, machine monitoring, and statistical analysis programs. Nancy Venable, product manager, explains that machines can be connected to the software suite in several ways. “All the machines in a customer’s plant could be connected to one server through a local plant network, or the machine builder might opt to embed a small standalone single-node solution for monitoring and control,” she explains. “A hybrid solution contains both elements, where the standalone nodes are embedded into the machines, and a central server collects information from the nodes for processing.”
