Most machine builders and integrators get to experience the joy of supporting obsolete machines and control hardware. What was easy to support and understand in the past is often difficult and somewhat annoying, if not impossible, to support today—10 years later. Much time is spent searching spare part shelves, old PCs and the Internet for the hardware and software needed.
Starting now, gather all the machine and program history, both on paper and digitally, and keep it in a locked, controlled file cabinet and server. Keeping good documentation such as the schematic and program descriptors, spares on the shelf and those old software floppies and CDs is good practice. While the internet and online depositories are helpful, detailed plans for the replacing, designing, programming and upgrading of obsolete components are a must.
Proactive strategies for obsolescence
"Hearing that your components are obsolete is never good news in any type of manufacturing industry," says Jerry Flynn, principal engineer at Concept Systems, a member of the Control System Integrators Association (CSIA), headquartered in Albany, Oregon. "For most, the first thing that comes to mind is always, 'How much is this going to cost me?' Depending on the situation, it can be a lot. It isn’t just the cost of new components, but the downtime associated with replacing them. Fortunately in today’s world most manufacturers are rather proactive about informing their customers when components will be going obsolete."
This is all well and good, but what if you have a breakdown and only then learn that your failed component went end-of-life five years ago? asked Flynn. "Fortunately there are choices out there,” says Flynn. “A simple Google search will reveal a large number of companies that make their living off of refurbishing and remanufacturing old and obsolete parts. Sometimes you can even find parts on sites like eBay. The downside to this is that the parts rarely come with any type of warranty; they can be extremely expensive; and at times they are even broken upon arrival (Figure 1)."
Figure 1: Those ugly, obsolete components are not good—unless you have one that is needed.
(Source: Concept Systems)
A common strategy is a manufacturer stockpiling parts they know are obsolete. "This can work for a time, but, in most cases, it is more of a delay tactic," says Flynn. "This strategy makes sense if the end product is going to be obsolete itself soon and the machinery mothballed. However, in most cases the parts and those with the ability to install them will eventually be gone."
There are those who have held on to old equipment and want to keep it running until they have no choice but to replace the entire system. "I have personally witnessed some scenarios along these lines, usually related to older PLCs and programming languages that are no longer in use," says Flynn. "The big manufacturers don’t want to support these systems, but often times they do still have the ability to do so—for a steep price. Software from the 1980s with an original price tag of $300 can cost you upwards of $20,000 today. These costs have forced many companies to re-engineer their systems from the ground up. They could pay the outrageous software fees to recoup their original engineering efforts, but for another $10,000 they can have a brand new system put in. In the end, waiting this long to replace your system doesn’t make fiscal sense."
Plan for obsolescence
"Many control system suppliers drive change and, subsequently, obsolescence without a good plan," says Craig Souser, president/CEO at JLS Automation in York, Pennsylvania. "However, the big challenge is when the form factor that you used on one system changes. For example, maybe you bought a drive with an integrated power supply, and now it’s a modular configuration with separate components that have a completely different orientation, interconnects, connectivity or all the above."
The new replacement product may even function differently. "A loss of features or functions is not unheard of," says Souser. "What happens when a certain level fieldbus goes obsolete for a drive offering? You can’t change the entire control scheme to replace one device, but you might have no choice."
Most parts, or only the most important ones, will become obsolete, and that includes the part number itself. "Every purchased part we buy gets our unique part number,” says Gary H. Lucas, director of innovation at Innovative Treatment Products in Owings Mills, Maryland. "We never use the manufacturer or vendor part number as our own; they too frequently change. When we are alerted that a part is no longer available it is immediately marked as inactive in our MRP system. This means it can’t be put on purchase or sales orders but is still in our system for reference when a customer calls. The new and different part gets a new part number in our system, along with the new manufacturer’s part number (Figure 2)."
Figure 2: Components and part numbers come and go, and the configuration changes, but what doesn't change is the engineering approved OEM part number created.
(Source: Innovative Treatment Products)
Engineering should be involved with part number changes. "Nearly every obsolete part change elevates the process from purchasing up to engineering for review, says Lucas. "Our system tracks every part in every job so we know exactly which customers have the obsolete part."
It is inevitable in most manufacturing facilities to find that a number of components running the machinery are reaching their end-of-life phases. "The best starting place is to reach out to the equipment manufacturer or a local integrator to find out what options you have,” says Flynn at Concept Systems. "If components you have are going obsolete, it’s likely that the manufacturer already has a planned approach for replacing those components. At times you may even receive credit for your existing components, or there may be a way to put new components in with minimal downtime, if that is your priority. This is always a good time to explore upgrade options, as well, and perhaps get an attractive ROI on any necessary replacements."
Dealing with obsolete components in a plant has several risks and associated costs, says Swapnil Adkar, global product manager at Honeywell, a member of CSIA. "Plant operations and maintenance teams need to carefully plan their activities and approach toward managing control systems," he says.
Examples include ensuring the right inventory of obsolete parts is maintained until migration of these systems is completed. It also requires carefully planning any modification to hardware and software with the right mitigation plans. The plans should include ensuring expertise and knowledge of obsolete systems at the site is retained. If not, a migration to new, state-of-the-art PLC technology, not only will ensure system operation, but improve plant availability and reliability.
Hardware and software
It is not uncommon to search the Internet for the obsolete components, especially to support existing equipment. "We are forced to do this all the time as the plants we build are frequently quite old and operate 24/7. Moving to a new PLC would be an enormous task," says Lucas at Innovative Treatment Products. "Unfortunately customers don’t seem to understand that PLC parts should have critical spares just like pumps. If the PLC doesn’t work, it is likely none of the pumps will either."
There will often be times when you will need obsolete components for jobs you are currently working on, says Mark Teed, controls program manager at Applied Manufacturing Technologies, a CSIA member, headquartered in Orion, Michigan. "This often happens when a customer, typically a second- or third-tier supplier, wants a duplicate machine, and a period of time has gone by,” he explains. “In this case during the quoting process, you will need to inform the customer of a potential problem in this area. If the customer still wants identical components, then the search is on."
Options mainly fall into two categories: warranty and nonwarranty. "Most manufactures don’t simply stop making the components, but they will economically obsolete them by raising the price,” says Teed. "Sometimes the price will triple overnight. However, when this happens they will give you a pretty lengthy warning and often have an alternative component available."
If a warranty isn’t a concern, you can go to the gray market and see what you can find, Ebay is a great source for this. Another option would be to go to the original supplier of this product and see if the supplier has any used components, maybe from a demo unit.
It is not easy to get rid of obsolete components such as spares on the shelf. "This will often be the hardest decisions to make: Should I just take the loss and put them on eBay, or should I hold on to them because I might need them again?" asks Teed. "The problem with holding onto them is that the part is likely out of warranty. I once held onto a brand-new, in-the-original-box Rockwell Automation PLC-5/11 for 15 years before I found an opportunity to use it in an emergency—for a dead-in-the-water machine service call."
Software should be part of the obsolescence discussion, as well. "When I think of programming, I think of PLCs and HMIs," says Teed. "It seems like every month or two manufacturers come out with new ‘rev’ levels (revisions) of software. Some of these new rev levels are so specific that even the reps for the product can’t keep up with them."
We have a rep for one of the major PLC/HMI manufacturers come to our office quarterly just to try to keep us up to date on these changes, explains Teed. "So, remember, if you choose to use an obsolete PLC/HMI, the software you currently have in your computer may not program it," he says. "I have in the past needed to find and purchase an old laptop with a Windows XP operating system, a PCMCIA (Personal Computer Memory Card International Association) slot and an RS-232 port to have the ability to program old equipment, and I still have it. So, when thinking about using an obsolete PLC/HMI, take into consideration the amount of time and effort it will take just to have the capability to communicate with it."
Why not just upgrade?
It is not easy to just upgrade the design before obsolescence occurs. "The saying, 'If it’s not broke, don’t fix it' applies here," says Russ Bullmer, director of technical product support at Allied Electronics & Automation. "If the customer is an OEM machine builder that designed in various control components, they invested a lot of time and money and resources to get the equipment running and approved. When looking to upgrade a machine, changing a component may require equipment reapproval, so there is a cost tradeoff between how long to maintain the current machine design and investing engineering resources in an upgrade (Figure 3)."
Figure 3: Sometimes it pays off having those old parts sitting on a pallet somewhere, but maybe it's time to upgrade.
(Source: Allied Electronics)
If the customer is a user of machinery that contains control components, oftentimes, upgrading equipment means downtime, so why change if the current machinery is running? "There is also an inherent risk if a user changes parts that are not the same," says Bullmer. "If the component change results in a piece of equipment not functioning properly, who is responsible for the lost production time? Our experience is that most maintenance people will not change a component until it is their only option and they have exhausted all other options for an 'exact cross.’"
Ease of design and programming needed
There are ways to minimize design and programming time needed when replacing obsolete components. "Cost, risk and time for migration can be minimized by selecting a control system with sufficient tools and documents to simplify the process," says Swapnil Adkar at Honeywell. "Replacing obsolete components involves careful planning and execution. Reverse engineering of design and application programs requires significant engineering effort. Planning for replacement starts as soon as the decision for replacement is made, and every step in the process needs to be supported with documentation. Leveraging a migration tool with the ability to document information right from the site survey to conversion of the design and application simplifies reverse-engineering efforts."
Select hardware and software platforms with a proven track record of long-term availability, support and service. "PC-based control technology is inherently well-suited to combating obsolescence for machine builders because of platform openness and the use of standard hardware built from the deepest pool of commercially available components on the planet," says Eric Reiner, industrial PC product specialist at Beckhoff Automation. "By strategically sourcing components from companies like Intel and ARM, PC-based control manufacturers can ensure lengthy product lifecycles and extended service that can go for decades."
If a machine builder has the viable option to stick with what it has for a longer period of time, it naturally means less engineering costs in the long run, continues Reiner. "This is another point where a wise choice of suppliers is critical—you can lower your cost of engineering by working with the suppliers that help you handle migration easily, and only when necessary," he says. "Leverage standard software platforms for all controllers and direct functional hardware replacements that anticipate and properly address your future needs. Obsolete components or not, if a machine builder must address new required modifications or enhancements to an existing application, it might be time to upgrade to a new controller. However, the timing and the engineering approach still has to make long-term financial sense for your company."
Parts vs. components
"There are definite differences regarding obsolescence strategies between component parts used onboard and parts used as control system components such as PLCs, HMI and drives," says Allied’s Bullmer. "Interconnect, passive and electromechanical (IP&E) and semiconductor component manufacturers are good at communicating component lifecycles, and oftentimes work with companies such as SiliconExpert who provide a central database engineers can use to determine part obsolescence and replacements."
On a more local level, some distributors are creating programs that help their customers with obsolescence for lines they are authorized to sell. "However, when researching for control-system component obsolescence, the task is more difficult, as each manufacture has its own way to communicate product obsolescence and replacement," says Bullmer. "The typical method I’ve seen is a general email sent out to distributors, announcing the product obsolescence, which then becomes the distributor’s responsibility to notify customers of the upcoming obsolescence and possible last-time buy."
Deal with the challenge
"The focus here should be on avoiding the challenge of obsolescence altogether," says John Fryer, senior director, industry solutions, Stratus Technologies a member of CSIA. "Companies should retrofit the infrastructure with longer shelf-life technology from vendors that have longevity and set-and-forget solutions with automated diagnostics and component failure mitigation."
Some vendors would rather claim non-obsolescence. "One of the key factors contributing to industrial-control-system obsolescence is the fact that most manufacturers are dependent on third-party semiconductor suppliers," says Albert Rooyakkers, founder and CEO of Bedrock Automation. "Because chips tend to have shorter life spans than industrial control systems, vendors are likely to obsolete the whole system based on its components. A variety of embedded components may become obsolete sooner and cause the vendor to obsolete a system sooner (Figure 4)."
Figure 4: Some embedded components that might become obsolete sooner and cause a vendor to obsolete a system sooner.
(Source: Bedrock Automation)
Bedrock Automation is in a somewhat unique situation, in that, being owned by a large semiconductor company, Maxim Integrated, it manufactures almost all of its own chips and thus has total control of when to obsolete a component, if ever. "Bedrock Automation can offer a non-obsolescence policy knowing that we can manage semiconductor obsolescence issues."
Perhaps Flynn at Concept Systems sums it up best. "In short, the best way to deal with obsolete components is to be proactive about it—examine what lifecycle phase your components are in," says Flynn. "If they are approaching end of life, contact the manufacturer or integrator about a plan to bring things up to date. There are many options for doing so, and going forward can give you many advantages you hadn’t initially anticipated, such as increased production, digital copies of your electrical schematics and less maintenance requirements. The best strategy for dealing with obsolete components is replacing them before they become obsolete components.”