Robotics for waste sorting and recycling

Abstract

A key building block for creating a circular economy is the ability to eficiently recover waste. For recycling to be profitable the purity of the separated fractions must be very high. The aim of the project is to implement a robotised waste sorting system to complement the current commercial solutions. The objective is to improve the quality and quantity of material recovered while limiting costs and labour use. This can be achieved thanks to advanced computer vision and robot manipulation techniques. The system will consist of two main components: (i) a vision system based on Deep Learning (DL) that combines several cameras to achieve high accuracy in material recognition; (ii) a manipulator robot that will sort objects based on feedback from the vision system. Grasp planning will exploit Reinforcement Learning (RL) to learn how to handle complex situations such as singling objects from a stack or disordered flow. The goal of innovation is twofold: to develop Artificial Intelligence techniques to be able to use low-cost sensors and to make system training simple and flexible for high reconfigurability to diferent types of waste.

Keywords

• waste sorting
• robotic waste sorting
• circular economy

1. Introduction

Waste sorting is a complex problem which is addressed in several stages. First a gross manual separation is performed by the owner of the items before trashing the items into a specific litter box accordingly to the material of the item. Than, specialized sorting companies empty these boxes and process the separated fraction to increase the purity of the diferent waste fractions. Long chains of machinery are therefore used in waste sorting plants that include mechanical shredders, optical sorters, magnetic separators, rotating drums, vibrating plates and more [ 1 ].

Anything that these automatic systems fail to sort (in the

case of plastics about 15% of the total) is still sorted by hand by workers forced to perform repetitive work in an unhealthy environment. These plants have a very high cost because it is part of the potentially recoverable material (up to 20% in some cases) fails to be recovered.

Industrial robot manipulators are increasingly fitted with exteroceptive sensors and autonomous planning capabilities to modify their motion according to external stimuli. We are developing an AI-powered robot manipulator for waste sorting. The goal is to increase eficiency and lower the cost of waste separation in order to reduce the amount of waste directed to the incinerator. For the recognition of waste, Deep Learning,

Self-Supervised Learning and Data Augmentation tech

niques have been widely used in this project in order to deal with the complexity of collecting large amount of labelled data in real-world industrial settings. Reactive motion planning techniques and markerless human motion tracking techniques are under investigation to

We aim to develop a robotic system for sorting waste (Robotic Waste Sorting System, RoWSS) to be inserted process. panies have recently started to propose robot systems to economic and human cost. For this reason, several com- achieve an eficient collaboration between humans and robots. Diferent robot geometries will be investigated accompany or replace the current waste sorting systems. for increasing the speed and the eficiency of the sorting within the current plants sorting, where manpower is cur- of human operators, freeing up the workforce for other rently used to improve separation some materials. Sorting waste on a conveyor belt is repetitive work, wearing and unhygienic and also has an environmental/economic nEvelop-O 0000-0002-2945-8758 (A. Bacchin)

The use of an RoWSS allows to reduce or avoid the use more rewarding and value-added functions, and would make it more eficient and economically more sustainable material recovery. Moreover, it can reduce staf turnover which is very high in this sector of the unhealthy work environment. The development of waste sorting robots is not new. However, the products currently available have several limitations. In particular, it is assumed that the objects to be separated are placed on the conveyor belt in a not overlapping situation. This necessity limits the separation yield because it is not always possible © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License to guarantee that condition and it is necessary to put more machinery before it to better distribute the waste on the belt, which requires other investments to modify We already completed a couple of data acquisition the plant line. campaigns with diferent methodologies and in diferent

In our project, we aim to overcome these limitations sorting plants thanks to the collaboration of local compaby building a system able to manipulate complex and nies. Diferently from standards, our datasets have been disordered piles of objects. The product is under develop- collected with the aim to explore self-supervised and/or ment (TRL 2). The first prototypes and level installations weak supervised approaches. The first dataset have been industry are expected by the end of 2023. collected in a glass sorting facility. We installed two cameras above the conveyor belt, one before the human operator and one after. By leveraging the diference be2. Related works tween the scene before and after the human intervention, we aim to automatically label the contaminants indirectly Sorting through automated systems is a technological exploiting the knowledge of an experienced worker. A achievement of several years ago. Today, automation standard semantic segmentation model trained in this is very present in medium and large plants [ 1 ]. At the self-supervised settings showed an mIOU of 0.5. beginning these machines were built by adapting ma- On the other end, we built a second dataset in a paper chines from the mining or agri-food industry, while today sorting facility. In this case we targeted a diferent approducers are increasingly specializing, creating special proach. We collected around 2500 single waste objects systems for sorting waste: mechanical sorters, optical- from a real plant. Given a uniform background, it is pneumatic sorters, magnets, etc. These are able to handle relatively simple to automatically extract object image large flows of materials but have poor accuracy. For these patches and corresponding semantic labels. We aim to traditional plants, the separated outgoing fractions have use the obtained patches to generate synthetic cluttered a non-optimal recovery level (about 70-80%) [ 2 ]. In ad- scenes on a conveyor belt, through generative model as dition, many of these machines can perform a binary Difusion Models [ 7 ]. separation or separate the incoming flow into two frac- We also started to record a dataset of RGB-D and tions (the fraction to be recovered and the ”residue”) and hyper-spectral images to starting develop and test the are very sensitive to any contaminants that can even algorithms. Despite the promising preliminary results, block the plant. This is why today human operators are we consider the usage of hyper-spectral images as a secstill widely used in sorting plants to bufer the low yields ondary research line since the high hardware costs would of the machines currently on the market. limit the applicability in real-world settings.

There are also some companies producing robot waste The handling system is also very important since it sorting systems based on robot manipulators. Only a is the main limitation of current commercial solutions couple of these company seems to have mature systems1. which perform just standard pick & place operations The products they ofer, albeit with some diferences, use common in industrial robotics. We are developing an an advanced vision system to guide an industrial robot advanced algorithm for grasping and disentangling in in a pick & place operation. However, these machines order to be able to pick objects also from complex and need objects well distributed on the conveyor belt and intermingled piles of materials. In this direction, Reincannot handle complex and intermingled objects. forcement Learning and Self-Supervised Learning have shown promising results in laboratory settings [ 5 ]. We 3. Methods want to extend these approaches to work in real-world settings, e.g. considering objects with irregular geomeThe system we are developing is sketched in Fig.2. It is a tries. modular system to be adapted to the current machine in Another promising advancement we are going to test the sorting plant and especially to the conveyor belt. It in the next months is the use of tossing motion for the is designed to work alongside human operators and to robot. After the robot picks the objects, instead to move perform specific selections. In the first stage we devise the end-efector over to the correct container and place to have several installed along the conveyor belt, each the object, the robot is is throwing the object into the one devoted to selecting a specific material. The system container to save time in the cycle time. Tossing involves is composed of two main blocks: the vision system and a complex correlation between the shape of the object the handling system. and the release pose and speed, which are almost im

The vision system is mainly base on Deep Learning. possible to model in a closed form using equations of The first problem to be tackled is the data collection to motion. Deep learning can be used to capture this infortrain the models, then. mation citeb13, allowing to compute the right release specifications without an explicit motion model.

1https://zenrobotics.com/, https://www.amprobotics.com/

4. Conclusions

We presented an automatic system to process waste to sort the diferent fractions to be recycled with very high precision. The system is based on a modular solution with a robot manipulator and a vision system. The advanced algorithms we are developing in this project will enable us to overcome the current limitations due to the long cycle time and high costs of the machine vision systems.

We are designing a first prototype for careful and precise sorting of paper, cardboard, and TetrapackTM. We aim at achieving an accuracy of the sorting of 98% against the 80% of current opto-mechanical-pneumatic sorters.

References

[1] Sathish Paulraj Gundupalli , Subrata Hait, Atul Thakur , A review on automated sorting of sourceseparated municipal solid waste for recycling , Waste Management , Volume 60 , 2017 .

[2] Wilts , H. ; Garcia , B.R. ; Garlito, R.G. ; Gómez , L.S. ; Prieto , E.G. Artificial Intelligence in the Sorting of Municipal Waste as an Enabler of the Circular Economy . Resources 2021 .

[3] M. Terreran , A. G. Tramontano , J. C. Lock , S. Ghidoni and N. Bellotto , ” Real-time Object Detection using Deep Learning for helping People with Visual Impairments ,” 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS) , 2020 .

[4] M. Terreran , E. Bonetto and S. Ghidoni , ” Enhancing Deep Semantic Segmentation of RGB-D Data with Entangled Forests ,” 2020 25th International Conference on Pattern Recognition (ICPR) , 2021 .

[5] A. Zeng , S. Song , S. Welker , J. Lee , A. Rodriguez and T. Funkhouser , ” Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning ,” 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2018 .

[6] A. Zeng , S. Song , J. Lee , A. Rodriguez and T. Funkhouser , ” TossingBot: Learning to Throw Arbitrary Objects With Residual Physics,” in IEEE Transactions on Robotics , vol. 36 , no. 4 , pp. 1307 - 1319 , Aug. 2020 .

[7] Sarukkai , Vishnu , et al. ” Collage Difusion.” arXiv preprint arXiv:2303.00262 ( 2023 )