Focus on First Open Call winners: HumAn-enhanced WeaK supervised learning for automatic data LABELing in the depalletization process

WHAT ABOUT THE EXPERIMENT? 
 

The proposed solution represents a cutting-edge vision system that seamlessly integrates both hardware and software components, aiming to significantly enhance image annotation process efficiency for robotized depalletization tasks where human operators will cover a central role. This solution will exploit a weak-supervised deep learning approach for auto-labeling that takes into account the human actions in the data annotation phase by observing his activities while performing the depalletization task just once the pallet arrives at the workstation.
The system features a high-precision 3D (RGB-D) camera positioned above the depalletization zone, capturing detailed images and point clouds for processing by HAWK4Label's software. Real-time imaging allows for capturing diverse, real-world scenarios, improving model robustness and adaptability to various environmental conditions. The system also encodes human operational expertise, integrating valuable insights into the data for improved training and operational effectiveness.
The main goal is to reduce the labeling time by exploiting real observations of humans and provide an efficient and accurate way to collect robust data to train the final robotic depalletization system. HAWK4Label will be tested in the food & beverage industry by depalletizing beverage bundles on a pallet. The secondary objective is to integrate the weak-supervised approach in the IT+Robotics (ITR) proprietary robotic-guided vision system EyeT+ Flex to improve its performance and flexibility. The deployment will be done using edge-computing devices on the production side that directly communicate with the already available ITR cloud infrastructure used only to train the AI model and return it to the user for deployment on the factory floor. Supported by digital twin technology, this integration facilitates rapid feasibility assessments by technicians and robotic integrators, with end customer data further optimizing testing processes. Demonstrations and tests at the ITR Padua facility intend to advance the Technology Readiness Level (TRL) from 5 to 7, demonstrating the system's practicality and applicability across broader domains.

​

WHAT IS THE EXPECTED IMPACT?  
 

The HAWK4Label system is designed to produce a step forward in current vision systems used in the food & beverage and logistics industries, where depalletization and bin-picking applications have a crucial role. The application of learning-based vision technologies in that industrial sector has a strong impact from many points of view, especially in production efficiency due to the high variety of available products that require to be handled, mostly each with a different approach. The HAWK4Label aims to develop a facilitating solution for the data acquisition and annotation part to be as general as possible and easily adapted to every product's depalletization operations. In addition, this technology can simplify and accelerate the deployment, allowing even non-skilled operators to train the system and enabling a wide adoption with very different objects and sectors. This is another crucial element that arises from the human-centric approach adopted in the proposed HAWK4Label system. Indeed, weak supervised learning will be developed to improve the generalization of EyeT+ Flex in the classification and recognition of objects, making the ITR product more competitive and attractive for the market.
The AI approach permits the system to be easily adaptable to a wide range of applications and industrial processes and could be rapidly transferred to other industrial fields such as logistics, food, and waste sorting and monitoring.