AI-pick

AI+Pick: AI Picker for multi-geometry parts

CHALLENGES

The classical industrial bin-picking applications exploit a CAD of the object to perform the localization and computation of a suitable grasping point. CAD-based systems are widely used in industry for simple, low-reflective objects. Indeed, the recognition became complicated by the complexity of shapes, partial or complete occlusions, the object’s shine, and variation of lighting conditions. Furthermore, accurate recognition and grasping require high computation time. All these issues may lead to inefficiencies when relying solely on CAD-based systems. In addition, this approach is not scalable enough to perform well when objects of different types are in the bin.

On the other hand, DL has good recognition, and object classification results are more robust, with shiny objects and lighting conditions resolving the previous CAD issues. However, DL solutions face several challenges, including the need for large amounts of annotated data, which can be time-consuming and expensive to collect. There is also a risk of overfitting, where the model performs well on training data but poorly on new data, requiring careful design and diverse datasets to ensure generalization. In this case, fewer images and a self-supervised approach will be used, and the CAD model will be used to refine the location.

OBJECTIVES

AI+Pick is a hybrid solution for a multi-random bin-picking task that combines standard techniques based on CAD models with data-driven methods based on Deep Learning.

This approach combines the strengths of data-driven and model-based approaches, ensuring robust and precise performance, thus overcoming the usual problems of CAD-based only bin-picking solutions, i.e., robustness to changes in the environment, poor generalization to lighting conditions, and high usage of resources when solving for object candidates.

DL tools enable the system to identify and classify objects accurately, even for scenarios where objects vary in shape, size, and orientation, and shine and reflection may appear, such as in industrial environments. Combining this AI tool with more standard and locally stable CAD-based object localization gives the system more robustness since CAD-based approaches can provide detailed geometric information and accurate localization results when guided by a refined local search rather than relying on a top-down global search. The AI+Pick solution represents a huge innovation in the bin-picking market because it will integrate two different approaches into a unified product.

Moreover, each step of the object candidate computation will be stored, and data will be used to retrain the localization module to enhance the module’s performance from time to time, one iteration after another. AI+Pick will enhance product quality through consistent performance, and its flexibility allows for quick reconfiguration of existing production lines, fostering innovation and reducing time-to-market for new products.

APPROACH

The solution involves developing and providing hardware and software tools specifically designed to cover all the requirements. In detail, the system will be composed of an object localization module and a grasp synthesis algorithm based on the EyeT+ Pick product. Thus, AI+Pick builds upon a solid technological solution already in the machine vision systems market for robotic bin-picking.

The solution will combine a DL object detection algorithm executed on RGB/RGB-D images with a model-based 3D object localization algorithm executed on the point cloud and tailored to different objects. Using the CAD model, the candidate objects’ locations extracted by the DL module will be used to narrow down the search area for the final 3D alignment. This will drastically reduce the computation time, making the entire process more efficient, parallelizable, and fast.

The object candidates extracted by the enhanced localization module will be sent to the task planning module to select the next best action to minimize the cycle time, i.e., the best object to pick for each robot. In this phase, the gripper structure and the robot’s kinematics will be considered to generate and select the best grasping point and avoid collision with the surrounding environment. In addition, the system will be able to provide the robot with entry and exit points from the bin to make a full-collision-free grasp.

EU funded

This project has received founding under the COROB project which has been supported by the European Horizon Europe program under Grant Agreement No 101120640

 

“Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Commission. Neither the European Union nor the granting authority can be held responsible for them.”