A Complicated Vision

Global Machine Builder Adds an Essential Machine Vision System to Engine Assembly Line to Improve Final Quality

By Anthony Ponton, ThyssenKrupp System Engineering

The design and installation of a new engine assembly line is among the bigger projects a machine builder can hope to win. That project gets a bit more involved when the builder, also acting as the system integrator, has to add a machine vision system to the line when the customer finds too many engines failing their final quality test because of operator errors during the assembly itself.

That was the challenge facing ThyssenKrupp System Engineering, Auburn Hills, Mich., a global designer and manufacturer of turn-key assembly and test systems for automotive powertrain components including engines, transmissions and axles. A leading manufacturer of construction and heavy equipment commissioned ThyssenKrupp to build an engine assembly line (Figure 1) for a new plant in the U.S. Southwest.

Acting as the system integrator, ThyssenKrupp supplied the vision, mechanical and electrical systems, PLC connectivity, and training of plant staff. The first steps of the vision solution were developed using six Matrox Iris GT smart cameras. As an engine moves down the line, it goes through multiple smart camera-based vision inspection stations. These applications vary from absence/presence to gauging/measurement.

ThyssenKrupp presented a camera-based solution similar to those they have done in the past with other engine builders. This solution would use vision inspection at the station where the parts were being installed, and then any errors could be fixed at the assembly station.

Adding Vision to the Engine Line

The heavy equipment manufacturer was a new customer for ThyssenKrupp System Engineering, and its managers were skeptical about using vision-based inspection as part of their assembly line. They questioned the reliability of the results as well as the dependability and maintainability of vision. “But, working with the customer, we determined that the part complexity, combined with the extensive teardown and rework required if an error is made, justified the use of in-station inspection,” explains Pat Coughlin, ThyssenKrupp’s program manager. “A camera was a perfect fit because it could handle the part variation and be located completely out of the operator’s workspace.”

Three applications on this new line use machine vision for inspections. One Matrox Iris GT smart camera verifies that the crank gear and water pump gear are aligned properly. The crank gear has a carrot feature and the water pump gear has a painted yellow line. The yellow line must be pointing at the carrot. Another application uses four Matrox Iris GTs. Cameras 1 and 2 both verify the presence and location of eight water seals (Figure 2). Camera 3 determines the presence and location of a drainback seal, another gasket and eight water seals. Camera 4 verifies that only a single head gasket is installed.

A third application uses a single Matrox Iris GT to verify that the timing mark (painted line) on the idler gear is between two painted teeth on the crank gear (Figure 3).

Smart Camera-Based Vision Inspection

The applications were developed with Matrox Design Assistant, an integrated development environment that is bundled with the smart camera. For this application, smart cameras are preferable because we believed that they provided better reliability of the hardware vs. a PC and frame grabber.

“Matrox Design Assistant is an integrated development environment (IDE) where machine vision applications are created by constructing a flowchart instead of coding programs or scripts using languages like Basic, C, C++ or C#. Once development is finished, the project (or flowchart) is uploaded and stored locally on the smart camera,” says Matt Maitland, ThyssenKrupp controls engineer. “The project then is executed on the smart camera, without the need for any companion PC and, in this case, is monitored and controlled from the PLC over an Ethernet link.”

A Siemens HMI panel is located at each assembly station that provides the operator with the engine build type, cycle time display and any specific information required at that station. This HMI also is used to display the camera image so that the operator can look at the inspection in the event of a failure. He then identifies the object that failed and takes corrective action.

Once development is finished, the project (or flowchart) is uploaded and stored locally on the smart camera. The project then is executed on the smart camera, without the need for any companion PC and, in this case, is monitored and controlled from the PLC over an Ethernet link.

The Network Connection is used for communication with the same Siemens PLC that runs the entire assembly line using the TCP/IP protocol. The camera acts as a server and the PLC is the client. The camera monitors the connection from the PLC and, if the connection terminates, the camera automatically makes the communication port once again available for the PLC to reconnect. The PLC is also responsible for sending the proper recipes to the camera based on the engine type in station.

In addition to building a flowchart, the IDE lets users directly design a graphical operator interface to the application, which presents the inspected image as well as inspection statistics and controls. The operator interface displays annotations and pass/fail statistics. No control was given in the operator interface that would allow changes to be made to the project.

A number of Design Assistant tools or flowchart steps were used. “Image processing operations enhanced and transformed images in preparation for subsequent analysis,” Maitland says. “As well, objects were analyzed using image intensity. Measurement tools, which are used to assess parts’ manufacturing quality, performed 2D geometric dimensioning and tolerancing. The metrology step was used for identifying features and to make measurements between these. For example, angle of crank was verified as ‘top-dead center’ using angle between center of crank and ‘arrow’ pattern. The measurement step was used for the water seal/head gasket detection application in which Camera 4 determined the thickness of the head gasket.”

A variety of tools were used for redundant detection of features so if one tool failed, a secondary tool would be used for detection. The pattern-matching step as well as the model finder tools that locate objects using their geometric features were used depending on the object for both fixturing and object detection/measurement.

Design Assistant’s sub-flowchart feature allows the inspection logic to be neatly separated from the communication and data logging logic. This helped simplify the design process and will facilitate future maintenance. “Sub-flowcharts allowed us to have a common programming architecture between cameras with the main program and custom sub-flowcharts for different program numbers/complexity,” says Fabio Perelli, smart camera product manager for Matrox Imaging. “Our sub-flowcharts contain only vision tools, which makes it easy to troubleshoot when there is a complexity of modifications required.”

Other smart cameras were evaluated by ThyssenKrupp for the vision system, but the Matrox Iris GT offered some key advantages. “Matrox products offer features best suited for this application at a competitive price,” Maitland explains. “Matrox Iris GT also has a lot more memory than other smart cameras. We can load multiple jobs (inspections) in memory simultaneously. Other smart cameras’ memory limitations would have required that we load one job at a time. As well, images would have to be stored on a separate PC, since there would be inadequate on-camera storage. Design Assistant’s flowchart logic also allows us to branch and execute different inspections within an application. In contrast, other smart cameras only support the development of standalone inspections—an application needs to be shut down before the next one can be loaded and executed.”

The engine line’s vision system was deployed in September 2010. “Deployment went smoothly,” Maitland says. “Application troubleshooting was very limited and handled remotely in Auburn Hills by way of emailing a few images that required a second look, and it’s easy to debug a flowchart when the logic is visually presented as interconnected blocks.”

ThyssenKrupp handled the majority of these technical issues and required support from Matrox Imaging on only a few occasions during the initial development phase. It was on those rare occasions, however, that ThyssenKrupp appreciated having direct access to experienced engineers at Matrox. “ThyssenKrupp vision developers are extremely experienced—in fact, they provided us with valuable feedback on our smart camera software, some of which has already been incorporated into Matrox Design Assistant,” adds Matrox’s Perelli.

Further, ThyssenKrupp needed to spend only a day to train the plant staff on the vision system. And so far, there have been no technical issues that have required ThyssenKrupp to send staff on-site to troubleshoot.

Moving Forward

At this point, no data has been gathered to confirm the positive results of the camera additions. This is partly the result of more vision applications being added and considered currently. Both ThyssenKrupp and the customer are excited to investigate the metrics once the complete vision solution has been implemented.

Three inspection stations on the engine assembly line currently use vision, and another station is being added in 2011. Since the first three stations have proven to be effective and maintenance-free, the engine manufacturer is looking to use Matrox Imaging vision components to ensure that many more assembly processes are being completed properly.