3D point cloud enables robots to ‘see’

Like many other industries, the woodworking sector faces a variety of challenges. These include quality assurance, untapped efficiency potential and a shortage of skilled workers. One possible response is further automation. Automation can reduce errors, improve quality and increase efficiency. In addition, production can be increased and accelerated because machines can operate 24/7 and perform quality control faster and more precisely than humans. Finally, the increasingly scarce human resources can be deployed more purposefully by automating monotonous and physically demanding tasks.

HOMAG Bohrsysteme has developed an automated solution that addresses these challenges. The company is part of the HOMAG Group and provides customers in the woodworking industry with a wide range of support options through its high-tech machines and systems. Its product portfolio includes CNC machining centers, through-feed drilling machines, drilling and dowel-insertion machines, as well as machines for drilling and fitting insertion technology.

“We want to continue developing with solutions that truly make sense,” explains Luca Zingg, member of the management board responsible for corporate development at MAB. “The further development of the cell with laser scanning and chaotic stacking was the function we had been waiting for. This allows the cell to meet our goal of batch-size-one production, and only then does automation make sense for us.”

Until now, an employee handled the loading of the CNC machining center. This involved picking up the workpieces, scanning the attached barcode, placing them into the CNC machine and depositing them on another pallet after processing. After several hours, this monotonous task becomes physically demanding and is not particularly efficient in terms of profitability.

Schwarz explains the goal of the automation: “MAB has set itself the objective of increasing productivity, deploying employees more effectively and, above all, in less physically demanding workplaces and thereby reducing costs. Another advantage of a fully automated production process is that the workpieces no longer need to be sorted before processing, since the application can also handle chaotically arranged stacks. This saves time in the upstream process step, which further increases productivity.”

The challenge during implementation was to develop a completely new solution, as nothing like this previously existed on the market. The task is also not easy for the machine vision system. This is due to the enormous variety of workpieces; different surface decors must also be processed.

"Cycle time was a primary design driver from day one, exactly because a vision pipeline must never become the bottleneck of a DrillTeq V 310," explains Schwarz. "Acquisition, point cloud processing, top layer extraction, pose estimation, barcode reading and stacking decision are pipelined and overlap with robot motion and CNC processing wherever possible. The result, validated in serial operation at MAB, is that the cell is paced by the CNC and the robot kinematics, not by vision. Engineering this overlap was, frankly, one of the harder parts of the project."

Secondly, the image processing must function under ambient light. Because not every area is fully illuminated, less powerful vision systems may have difficulty determining the exact position of the workpieces.

"Wood decors range from matte to high gloss, from dark to nearly white, with foils, lacquers and raw surfaces, all on the same stack," says Schwarz. "The chosen 3D acquisition principle is robust against ambient workshop light, which was a hard requirement from MAB. On top of that, we combine 2D and 3D HALCON methods so that the system does not depend on a single contrast cue. We do not require a controlled lighting cabinet around the cell, which is part of what makes the solution industrially deployable. Specific lighting parameters, exposure strategies and the way we mitigate specular artefacts are part of our engineering core."

Finally, it is technically demanding to separate the surfaces of relatively flat boards. “Despite these challenges, it was clear that such a fully automated solution had to be based on machine vision,” explains Schwarz. “We had to teach the robot to see. With other technologies, such as sensors, practical implementation would be virtually impossible, particularly in terms of speed.”

A 3D point cloud enables the robot to recognize individual workpieces

The application consists of several hardware components. At its core is a six-axis robot. A vacuum surface gripper system is used as an end effector (Figure 3). "The vacuum surface gripper was specified by HOMAG for exactly this workpiece spectrum: porous raw boards, coated panels, narrow strips, large formats," says Schwarz. "Vacuum monitoring is integrated into the safety-relevant control logic of the cell so that the robot only executes a high-speed move once a valid grip is confirmed. If the grip is lost or never reaches the threshold, the cell reacts deterministically; it does not hope for the best. The exact thresholds, sensor placement and the way we couple vacuum feedback into motion are part of our gripper know-how."

A 3D laser scanner is also mounted on the robot’s gripper arm. The drilling operations take place in the DrillTeq V-310 CNC machining center from HOMAG. The machining center offers a wide range of options for precise processing of wooden workpieces.

"The cell and the CNC are coordinated via a clearly defined handshake covering states such as machine ready, door open/closed, workpiece loaded/unloaded, error and reset," explains Schwarz. "We use a supervisory coordination layer, rather than letting the robot act as an unconstrained master over the CNC. This is essential for safety, diagnostics and serviceability. The exact distribution of responsibilities between robot controller, cell PLC and CNC is HOMAG’s integration architecture, and we treat it as such."

For the machine vision software, HOMAG chose MVTec HALCON. “We have been working with MVTec’s software for some time,” explains Schwarz regarding the decision. “HALCON has a huge pool of machine vision operators that allow virtually all machine vision applications to be implemented robustly. In addition, the software is flexible when it comes to combining different hardware components. And if technical questions arise, you can simply contact MVTec’s customer service.”

At MAB, the production process proceeds as follows: An employee places wooden workpieces onto an unknown and chaotic stack in the work area. The robot then moves over the stack so the 3D laser scanner can scan it from above. The laser scanner then creates a 3D point cloud—a highly precise three-dimensional representation of objects consisting of numerous individual data points.

"The 3D acquisition device delivers a Z resolution that is well inside the range needed to separate flat boards on a chaotic stack," explains Schwarz. "This was non-negotiable, since separating thin panels was exactly the problem the project had to solve. We characterize the system per cell and per workpiece spectrum during commissioning; publishing a single number out of context would be misleading and, more importantly, would invite an apples-to-pears comparison with sensors used in completely different settings. What matters for MAB is that the system reliably handles the panel thicknesses in their portfolio, which it does, in serial operation, since summer 2025."

Get your subscription to Control Design’s daily newsletter.

After image acquisition, the machine vision software MVTec HALCON extracts the top layer of wooden workpieces from the 3D point cloud and determines the spatial position of each individual workpiece. A stacking algorithm then calculates the optimal order in which the robot should remove the workpieces. This is an important detail because an unevenly unloaded stack could collapse.

"The stacking algorithm does considerably more than emit a pick point," explains Schwarz. "It evaluates the top layer of the chaotic stack, ranks candidates by accessibility and stack stability, so the remaining stack does not collapse, and provides the robot path planner with the information it needs to approach, grip and retract without colliding with neighboring tilted boards. Path generation itself remains in the robot domain; the vision system delivers the geometric and semantic context that makes safe path planning possible. The interplay between these two layers, and the fallback behavior when a candidate turns out to be ungrippable, is one of the parts of the solution that took us the longest to mature."

The cell uses industrially established communication mechanisms, nothing exotic, nothing that would surprise an automation engineer. The integration is designed for deterministic timing and for the diagnostics that a 24/7 production environment demands.

The robot then begins its work, removing the wooden workpieces according to the calculated order and transferring them to the CNC machining center. Before this, the 3D laser scanner captures a 2D image of the code. MVTec HALCON reads the code and transmits the information to the machine. The workpiece is then processed according to this information. Afterward, the robot picks up the workpiece again and places it on the target stack.

"The 3D acquisition device is mounted on the robot’s vacuum traverse, which means hand-eye calibration is a foundational part of the cell, not an afterthought," explains Schwarz. "We use a hand-eye procedure based on standard HALCON calibration concepts, hardened by HOMAG-internal routines that handle the specific mechanics of a heavy carpentry gripper. Acquisition is performed in defined motion states so that vibration is not a relevant error contributor for the measurement. Re-alignment can be triggered by service personnel through a guided routine; in normal production the calibration is stable over very long intervals. Beyond that, the procedure is part of our know-how."

MVTec HALCON performs multiple image processing tasks

“We are seeing machine vision becoming increasingly popular in the woodworking industry and among carpentry workshops,” says Jan Gärtner, product manager HALCON at MVTec. “Our software, MVTec HALCON, offers numerous methods—for example, for inspection tasks or for collaboration with robots—that can sustainably support automation and digitalization in this sector.”

For the robot in the MAB system to work autonomously and grasp the workpieces precisely, the machine vision software must perform several tasks. First, MVTec HALCON converts the 3D point cloud into information for further processing. For this purpose, HALCON uses 3D object models. This central container forms the starting point for creating a coordinate system within the machine vision software, which is then transmitted to the robot.

Various HALCON operators first determine the distance from the gripper to the pallet, then calculate the top layer of workpieces and finally determine the precise position of each individual workpiece. These positions are integrated into the coordinate system of the HALCON machine vision software and transferred to the robot.

During the 3D scanner’s capture of the top layer of the pallet, it also records 2D images. HALCON uses these images to read the information from the barcode attached to each workpiece.

"Barcode handling is part of the cell’s data flow, but not of MAB’s drilling logic," says Schwarz. "The barcode identifies the workpiece; the corresponding processing instruction is held in the relevant production data system and made available to the CNC through the established HOMAG data path. That keeps the barcode short, robust and decoupled from the manufacturing program, and it keeps recipe management where it belongs."

The challenge here is that the captured image is quite large, while the barcode region is correspondingly small. Reading such small barcodes is a major challenge for any industrial image processing software.

“The image-processing part of the implementation was not entirely trivial,” explains Schwarz. “Because of the flat boards, we had to combine 2D and 3D methods. This was possible with HALCON and significantly simplified the implementation.”

The cell distinguishes several failure classes and handles them differently, notes Schwarz. "A workpiece whose barcode cannot be read or whose geometry does not match any known plausibility check is treated as a reject and is moved out of the productive flow in a defined way, without stopping the line,” he says. “Conditions that genuinely require human judgement bring the cell into a controlled stop in line with the applicable safety category, with operator guidance on the HMI. The autonomy boundary—what the cell decides on its own versus what it escalates—was one of the design decisions we discussed most intensely with MAB."

System completed in summer 2025

The system went into operation at MAB Möbel in the summer of 2025. “Thanks to the close coordination with the partners involved, we were able to achieve very good results right from commissioning,” explains Zingg. “The system is now operating very reliably, which makes us very satisfied and gives us confidence for the future.”

The increased level of automation significantly relieves MAB, says Schwarz. “The employees who previously carried out this task can now focus on other, more important activities,” he adds. “At the same time, this solution represents an important development for us, because it allows us to significantly increase the automation level of our core machines and thus offer customers additional value. Machine vision plays an important role here, because the technology acts as an automation enabler. In our collaboration with MVTec, we see the opportunity to offer our customers first-class and reliable solutions.”

The value that customers buy from HOMAG is not a list of components or parameters, notes Schwarz. “It is an integrated, productized solution for an extremely heterogeneous problem space—random stacks, batch-size-one workpieces, varying decors, ambient light, flat boards, barcode handling, CNC integration and safety," he says.