By Bob Waterbury, Senior Editor
Proprietary systems no longer rule. OEMs are using off-the-shelf personal computers, commercial technology, and standards-based software to devise best-in-class solutions to automation problems. They are using open systems to maximize functionality, achieve cost-effective control, simplify integration, and reduce operating and maintenance costs.
Success stories about PC-based open systems are becoming common throughout the machining, metalworking, packaging, and factory automation industries. Recognizing both the immediate operational and future migration benefits, end users are specifying more PC-based open systems in OEM and system integrator contracts.
Anything seemingly that good must also have drawbacks. Indeed, when selecting hardware from one vendor, software from a second, and cabling and networking devices from a third, there are bound to be incompatibilities or inconsistencies.
Sometimes these problems are not evident until it is too latewhen a major production problem arises. "The beauty of open systems is that they drive costs down while they increase functionality," says Dave Riling, advanced technologies manager, Wright Industries, Nashville, Tenn., a builder of automated assembly and robotic systems for industries that include semiconductors, electronics, and chemicals. "But when things go wrong with multi-vendor system solutions, support can be a nightmare. Judging from the amount of finger pointing that can occur when problems arise, you might think you're dealing with Curly, Larry, and Moe."
Open-System Standards Abound
An open-system application should be able to communicate and interoperate freely in a heterogeneous environment. In addition, an application and its databases should be capable of transport and migration from one version to another without losing data or functionality.
Unfortunately, there is no universally accepted definition of an open system. "An open system may be defined from the standpoint of what is commercially accepted, something based on a national or international organization standard, a consortium-based standard, a de facto standard, or even a licensing standard," says Dave Smith, network promoter and consultant, Siemens Energy and Automation, Norcross, Ga.
Official standards are generally developed and voted on by an organizational membershipusually an engineering bodysuch as IEEE, OSI, or ISA. Consortium-developed standards are more limited in scope and/or geography, and normally represent special supplier interests. DeviceNet and Profibus publish accepted standards that fall into this category. De facto standards are commercial productssuch as Microsoft Windowsthat have become common in their selection and use throughout a specific marketplace. Laws, such as those passed by the Federal Communications Commission, also define standards that are usually rigid in their application. And finally, licensing standards for use of a particular technology are generally controlled and granted by one source.
"In any open system, actual protocol stack testing is both necessary and foundational to the device testing that takes place later," says Smith. "Don't assume this has been done; ask for the results.
Most industry participants say the definition of an open system, what it conforms to, and the certification testing process are critical to open system implementation. "There may also be different degrees or subsets of conformance as in Ethernet and the various protocol stacks that are used in conjunction with it, says Horst Kolbert, network products marketing, Siemens E & A. For instance, TCP/IP is often used with Ethernet, but Fieldbus, Profibus, and other protocols may sit on top of it as well."
An open system provides freedom of choice. Ideally, it allows OEMs and integrators to buy hardware or software from a vendor of their choice, and still get standardized functionality and competitive pricing. Once developed, the logic to control the hardware should be the same from one supplier to another. And data should be capable of being shared seamlessly among the various networked devices.
Interoperable vs. Interchangeable
Most suppliers say that their products conform to a particular standard, but what does that really mean? "Major customers such as General Motors buy only products that have been conformance tested," says Steve Biegacki, vice president of commercial marketing, Rockwell Automation. "The average customer, however, is simply buying the best of breed in a mix-and-match environment. This is both an opportunity and a problem. The two main issues that must be sorted out involve interoperability and interchangeability."
Systems that conform to a particular standard can be expected to communicate with one another. "This means they should be interoperable," says Bob Rucinski, manager of the production integration team, GE Fanuc, Charlottesville, Va. But you can't even assume that. For example, some Profibus devices may support Intel processors, and others may support Motorola processors. Somehow, those bits have to be swapped for proper operation. In addition, suppliers may implement only a portion of an open system protocol, limiting its compatibility with other products that conform to the full standard protocol. So, even though product validation processes may be very thorough, they still may remove only about 85% of the bugs," adds Rucinski.
Interchangeability is perhaps even more difficult to assure in a mix-and-match open system environment. Although the protocols are identical and the processors totally compatible, suppliers are prone to add unique features that distinguish their product from others that conform to a particular standard. "For example, two photoelectric eyes may conform in all essential details to a certain standard. However, if one has a built-in counting/timing function and the other does not, they cannot be used interchangeably because of their functional differences," says Biegacki.
Every product has unique characteristics or features that distinguish it from other products in the marketplace. "That means there are degrees of interchangeability that make it a much larger, more complex issue than even interoperability," says Jeff Meyers, product marketing manager, system controls, Omron Electronics, Schaumburg, Ill. So, OEMs and integrators are well advised to familiarize themselves with key system components and standardize on them as much as possible."
Another issue affecting interchangeability is the use of synchronous and asynchronous call handling. Both may be mandatory in a particular application spec, but suppliers don't always implement asynchronous calls in a consistent manner, says Ryan McDonald, industrial automation product manager, National Instruments (NI), Austin, Texas. We work with the Open DeviceNet Vendors Assn., [ODVA] and other standards organizations to resolve such issues. But in the meantime, we may need to modify our procedures; we may use synchronous rather than asynchronous signals in order to temporarily remedy the problem."
Most of today's open control systems are built around Windows NT and often include real-time extensions," says Rucinski. "The tools that make all of this possible include OLE for Process Control (OPC), COM, and DCOM." Increasingly, suppliers are using these tools to bundle control and operator interface capabilities into integrated packages for application development as well as HMI and SCADA.
"Open systems work because of OPC," states Joe Aleman, software engineer, Daneb Corp., a Troy, Mich., builder of robotic and automatic assembly systems. "It provides the interface needed to connect PLCs to our robotic and automated assembly work cells. Thats the good news. But vendors have different implementations of the OPC standard, so its necessary to check each vendor's implementation personally," he adds.
The suppliers agree. "Without tools like OPC, interoperability and interchangeability in open systems would be virtually impossible," agrees Aldo Marcuzzi, software products manager, Nematron, Ann Arbor, Mich. "But basically, the integration issues with open systems aren't that much different from those involving closed systems or proprietary PLCs. His point: you have to deal with a company that knows the issues and how to resolve the problems. For instance, a new 3Com card probably requires a new driver, which in turn requires other system modifications, says Marcuzzi. It's like dominoes. Change one thing and a whole host of other things must be changed as well. Closed system or open system, specifiers have to be very careful in selecting suppliers they can rely on and work with over the long haul. Properly installed and tested, PC-based open systems should exhibit no more problems than PLCs," says Marcuzzi.
The Problem-Resolution Problem
OEMs don't care whether the hardware or software is at fault. They just need a fix. They look for someone like NI to take responsibility and find a solution," says McDonald. We often contact other vendors and works as an intermediary to resolve issues in a multi-vendor environment because it is a major supplier, and software is often the key. NI engineers first try to replicate and solve the problem. Second, they go to other suppliers in their standards group to solicit help in establishing a corrective action. Along the way, we may need to use remote monitoring and diagnostic techniqueseither Web or phone-basedto analyze actual online data, says McDonald. Sometimes software updates may be the real issue. At other times, improper third-party hardware or software substitutions may be at fault."
Some OEMs believe that because a product conforms to a particular standard, then that organization is responsible for resolving any problems. "But that's not how it works," says Omron's Meyers. "We are involved in ODVA, for instance, but its up to the members to voluntarily assume responsibility for problem resolution. ODVA provides the framework for problem resolution, and assists in product compliance, conformance, and performance testing. The ODVA task forces simply orchestrate the voluntary efforts of the vendors."
Smith and Kolbert of Siemens agree that good OEM configuration and diagnostic tools are essential in an open, multi-vendor system. "Most products have some type of self-diagnostic capability," says Smith. "This is an OEM's first step in identifying a specific problem area." Kolbert says most open system software also provides advanced diagnostic tools that offer further assistance in locating problems. If that doesnt identify the problem satisfactorily, then its time to contact the major vendor for assistance. This may take the form of telephone assistance, either online or offline, and Web-based assistance, including actual online machine monitoring in some cases."
Sometimes OEMs have their own definition of an open system, just as some suppliers have. This, in itself, requires some up-front resolution before an automation project is begun. "We talk to an OEM to understand his application," says GE Fanuc's Rucinski. From this, we derive a feature/function set that matches their worst-case performance scenario. Then using their definition of openness we develop a least-risk approach to control architecture implementation."
Customers often define their own open system, and the OEM develops a core open-control architecture. Suppliers then gather those pieces, and test and validate the concept using application testing that features component interactivity with the control logic right down to the kernel level. GE Fanuc used this procedure to develop a major system for the General Motors Power Train Division. Rucinski emphasizes that GE Fanuc's three-step approach involves initial technology assessment, component and architectural test and validation, and actual production testing.
OEMs and Open Issues
OEMs are aware of the limitations of even the best supplier companies. "No single supplier excels in all aspects of printing press production," says Howie Hoff, director of electrical development, Heidelberg Web Systems, Dover, N.H. "We work with quality suppliers whose performance is verifiable and trustworthy. Open systems allow us to take advantage of special expertise wherever it exists in the marketplace. It enables Heidelberg to find proven systems and subsystems that can be integrated into machine solutions. The downside is that their customers can easily add non-conforming hardware and software that may cause problems.
Heidelberg has a long-term relationship with Siemens and Profibusa standard for many OEMs in the printing industry. "Profibus allows us a lot of options and most of the devices are simply plug-and-play," says Hoff. "The few times we had any problems, Siemens quickly assumed responsibility for helping us resolve those issues. They even interceded on our behalf in working with other suppliers to find a solution. If its customers have problems, Heidelberg routinely offers remote diagnostics to help find the answer. To find a trusted supplier, we start with the warranty and liability provisions of the contract, and then evaluate the service and training provisions," says Hoff.
Many OEMs emphasize the importance of using select vendors whose performance and reputation can be verifiedrather than just any vendor with open system products. "We specify vendors that provide 24/7 support for open-system products," says Michael Wilmshurst, manager of controls engineering group, Hooper Engineering, a Sarasota, Fla.-based builder of automated form-fill and seal machines. "We depend upon remote diagnostics, the software downloads, and operating, service, and maintenance manuals available over the Web. Often, we test a supplier's remote service responsiveness before entering into a contract agreement, or during a machine prototyping stage."
Accordingly, Hooper has specified preferred providers for PLCs, servers, and touch-screen operator interfaces. "Company A may have an advantage for a year or so until the other companies catch up to them," says Wilmshurst. "Then, we may re-evaluate our position and go with another supplier as new technological advances become available.
PCs have offered many advantages to Hoopers machine controls. Windows NT has proved both efficient and reliable in our machine operating environments, says Wilmshurst. Embedded Web servers are increasingly specified because they allow remote access to online machine data anywhere in the world. Open systems are an essential part of Hoopers position as a flexible manufacturing machine builder.
IHI Turbo America, Shelbyville, Ill., manufacturer of a variety of turbochargers for off-road and automotive vehicles, recently implemented a PC-based open architecture system as the basis for machinery used to spin-test turbochargers. The device performs not only spin testing, but checks steady-state parameters and vibration limit testing as well.
"We make high-reliability machinery for the Big Three automotive manufacturers," says Paul Dykstra, project engineer, IHI Turbo America. "We needed an automated system to evaluate the quality of turbochargers for a new production line. We selected a PC-based system that uses LabView RT for test, measurement, and control. The machine clamps the part in position, brings it up to speed, analyzes vibrations, stops the part, and then releases it. During the test we also monitor axial and vertical clamps, and other critical variables. The real-time functionality of LabView RT assures us that if Windows crashes, the tests keep on running. Dykstra says the complicated controls in this test system would not have been possible using PLCs. In addition, we saved 50% of the cost by conducting the tests using a PC-based system rather than dedicated equipment," he adds.
Software Snares
Dealing with open systems is hardly a walk in the park. According to Aleman, Daneb is implementing both Rockwell Automation and Omron solutions on its machines. Using proprietary systems, the implementation effort required to get both supplier solutions to use the same logic and software would take weeks. OPC reduces that to hours. Thats the good news. However, there are differences for example in identifying and passing tag names back and forth. Rockwell uses both an access pass and a tag name. Omron does not use an access pass, so the user has to be aware of these differences. "You give it the PLC name and the tag name together," says Aleman.
Dirk is an engineer with a large manufacturer of machinery for the pulp and paper industries. He wont let us use his full name or reveal the company he works for, because they dont like to rock the supplier boat, but he offers a cautionary side to the story, too. "Ten years ago, we custom designed all of our own controls," says Dirk. "Today, we operate on totally open systems and, as a result, weve increased our efficiency, performance, and productivity by a factor of five using PCs, Windows NT, and LabView. There's no way that we could go back to using PLCs because of today's integrated, networked operating environment.
But hes paying a heavy price for our success, discovering that other suppliers hardware can be a nightmare to support and maintain. We have customers who expect us to support this equipment for the next 15 years, says our mystery man. If you can even get the same PC model two or three months from now, the chipset and boards will very likely be different than we have now. In one case, he says the physical dimensions of the PC even changed, making it too large for the original application. With changes like these where we have no control, spare parts, maintenance, and training are up for grabs," says Dirk.
The ever-changing nature of software is another problem. Revisions keep coming whether a user wants them or not. "But even with open systems, everything is tied into specific versions and interfaces," says Aleman. "Upgrading to new open system services or versions of software can be a problem. In some cases, you are still locked in unless you commit the extra funds to upgrade all aspects of the system to a newer technology.
These obstacles create a limitation on future modifications and releases. In each case, I have to test new revisions to see how well they fit into a particular customer's application, says Aleman. Assume nothing. If I had to deal with proprietary solutions for each PLC that we support, we would have to dedicate one person to each of those systems. We definitely don't want thatand we can't afford it."
Some vendors provide what help they can. "Thanks to a key supplier, we're able to stay on top of the applications software pretty well, but just try to get a necessary revision for the Microsoft operating system, says our mystery man. Sometimes these revisions may be put into a Microsoft service packet, but there's no guarantee how long you'll have to wait. He warns that this can force OEMs to work around problems, further delaying production schedules.
One result of this, according to some OEMs, is that its essential to write special long-term support contracts that will continually upgrade hardware and software to meet future operating and applications needs. A simple thing like a new video card in five years' time may require a new operating system, a new computer, and all-new applications software.
Wright Industries' Dave Riling sees open systems from a somewhat different perspective. "We do a lot of work with vision systems," says Riling. He may purchase a camera from one source, a frame grabber from another, and software from yet another. If you can't acquire an image, check the camera, he says. Chances are, it's all right, so check the driver next. If that's okay, check the frame grabber. If that seems to be the problem, don't be surprised to find out that it was written by a consultant who cannot be located. The same might apply to some of the more important parts of the application software, as well.
Riling's point is partnering is just as important with open systems as it ever was with closed systemseven though it opens up many more options and opportunities.
Another case in point: A customer brings a complete machine specification to an OEM, but explains that Company Xs product must used as part of the open system. The problem is that it may not be compatible with the OEMs other system components. If the device only uses NetBios, for example, and nothing else speaks directly to it, then you may end up using a lot of unnecessary protocol converters, explains Riling. In these cases, the customer's decision was probably based on lowest cost, or his supplier sold him on the virtues of that product, without proper regard to the specific application.
We have an in-house organization to integrate and test system components for compatibility, performance, and cost, says Riling. That way, we don't waste time on open-system components that are not optimal for our applications. As a result of this testing, Wright has developed excellent working relationships with preferred suppliers for motors and drives, and vision systems.
Riling also observes that the complexity of jobs has not changed much over the years, but the production schedules have become increasingly compressed. Every job comes in as a critical path system delivery, he says. And more frequently, customers come in without any staff engineering expertise because those employees have been laid off. They use a consulting firm to prepare the system definition and specifications even though the consultant may not have a good feel for the machinery, the application, and the performance requirements. That's when we become their open systems engineering department."
Open Systems: A New Opportunity?
Sometimes, the more things change, the more they stay the same. Both suppliers and OEMs emphasize that even with the many options offered by open systems, its still best to select strategic partners to work with over the long term. OEMs need some way to reduce the number of variables. Time and money would be wasted if every OEM and integrator began each new project with a clean sheet of paper and a wide-open field of suppliers and products from which to choose.
As open systems continue to evolve, the problem of keeping hardware and software current will become increasingly demanding. Continued service, maintenance, and operation of machines will consume larger portions of production budgets.
"This is a problem for some, but definitely an opportunity for us," says Rockwell's Biegacki. "We see an opportunity to provide support service to all brands of hardware and softwarenot just ours. He thinks this could even be pneumatic, hydraulic, or mechanical equipment that has little or no electrical content. We have the tools to be able to isolate problems and work them out with OEMs, integrators, and end users, adds Biegacki. It's a role were familiar with because we are already doing it. Whenever customers see our name on a control panel or inside a control box, they may call us, even if it's a software problem that has nothing to do with us."
Biegacki insists that it is rapidly becoming too expensive for OEMs to assume long-term responsibility for wide-ranging open-system components in their machinery, particularly when it sits in a customers factory. He envisions that some automation suppliers will become long-term contract service providers for machinery after it has been installed and is operating in a customer facility.
Whether open systems are viewed as a challenge or an opportunity, suppliers and OEMs alike agree that there is no turning back. The gains are indisputable; the problems inevitable.
What Exactly Is an Open System?
Ken Crater, the founder of the Automation List, recently conducted a poll on the http://control.com Web site. The question was simple: "What Does Open Mean?" Heres what the initial respondents to this survey think:
12.6% Whatever marketing wants it to mean
24.2% Protocol is published and free to replicate
1% Hardware is ISA bus, Compact PCI, PC/104, etc.
19.4% Software is open source
24.2% Answers 2-4 above
18.4% All of the above
Based on this, it appears more important than ever for supplier, OEM, and end-user to agree on a contractual working definition of an open system. : "All of the above was sort of a perverse choice that a number of people seem to think reflects the current reality, says Crater. It'd be nice if open could begin to mean something again.
