Proper waste management is not just about cleanliness; it is about safeguarding the environment, public health, and the overall well-being of communities. Trash detection and pickup play a pivotal role in preserving the natural beauty of our surroundings, ensuring the cleanliness of our streets, and mitigating the adverse effects of pollution. In this age of environmental awareness, these processes are integral to combating climate change, protecting wildlife, and maintaining the delicate balance of our ecosystems [1,2].
This vital task extends beyond mere aesthetics. Timely and effective trash detection and pickup are fundamental in preventing the spread of diseases, enhancing the quality of air and water, and creating spaces where people can thrive. Moreover, in the era of smart technology, the integration of innovative solutions like sensors, data analytics, and artificial intelligence has revolutionized waste management, making it not only more efficient but also more environmentally friendly [3].
This introduction sets the stage for a deeper exploration into the significance of trash detection and pickup, shedding light on the multifaceted benefits that these processes offer to our communities, environment, and, ultimately, our collective future.
In this paper proposes a method for detecting and picking up trash designed for autonomous cleaning robots to operate indoors. However, this innovative approach extends far beyond its initial scope. Its adaptability and potential for wider application suggest a transformative impact on various scenarios, promising a more efficient and comprehensive solution to waste management challenges [4][5][6].
Based on the literature search conducted on the topic of the study, eleven articles directly related to the research theme have been selected. They are summarized below in Table 1.
Objective: The primary objective of this research is to comprehensively investigate and develop strategies aimed at streamlining the cleaning process and elevating the operational efficiency of cleaning robots. This will be achieved through the implementation of advanced techniques such as garbage recognition and the precise identification of pickup points. 2023 [17] Research Object: The focal point of this study centers on cleaning robots as a critical component of modern cleaning practices. The research aims to delve into their capabilities, performance, and potential in terms of organizing and maintaining cleanliness within indoor spaces.
Research Subject: The core subject of this investigation pertains to the innovative approach of integrating garbage recognition technology and precise pickup point determination. These innovative methods are expected to serve as a catalyst for improving the overall effectiveness and productivity of cleaning robots, thus addressing the ever-growing demand for efficient and automated cleaning solutions in various indoor environments.
Proposed method consisted of a few parts, diagram provided in Figure 1.
At the inception of the workflow, the system initiates the data collection process, capturing valuable sensor data from IntelRealSense devices. This data serves as a comprehensive representation of the environment, allowing the system to discern its layout and the presence of various objects, with a particular focus on identifying trash items.
The subsequent phase of the operation involves intricate algorithms designed to not only detect the presence of trash but also classify it into specific categories and pinpoint its precise location on the floor. This multifaceted approach is pivotal for enabling the system to efficiently and accurately recognize various types of waste materials, an essential aspect of any cleaning and waste management process.
Following this, the data is passed on to a specialized component within the system, tasked with the conversion of the pixel-based position representation into SI units. This conversion serves to standardize the location information, making it universally comprehensible and facilitating seamless integration with other devices and software.
In the final segment of this complex process, the system utilizes advanced techniques to approximate the shape and attributes of the detected trash items. This analysis is crucial in determining the most suitable points for the robotic gripper to engage with the waste materials. By understanding the intricacies of the trash's structure, the system can make well-informed decisions about the optimal pickup points, ensuring a secure and efficient removal process. This step is instrumental in enhancing the system's overall efficiency and effectiveness in carrying out cleaning and waste management tasks.
The methodology employed in our proposed system leverages the formidable capabilities of YOLOv5 [18], a state-of-the-art object detection model, which has been trained on the TACO dataset [19]. This robust training process equips the model with a deep understanding of a wide array of object classes, allowing it to proficiently recognize and classify various items within its field of view.
However, when it comes to the crucial task of selecting the appropriate objects for pickup, the system needs to discern whether an object is squishable, solid, or unpickable, as these characteristics significantly impact the grasping and handling strategy. To accommodate this decision-making process, all the classes from the TACO dataset are propagated into a smaller set of overarching categories, which primarily include squishable, solid, and unpickable.
This strategic classification process is instrumental in ensuring that the robotic system can make informed decisions when it comes to the retrieval of objects, taking into account their physical properties and suitability for pickup. By categorizing objects into these key classes, the system can fine-tune its actions, optimizing its efficiency and effectiveness in handling a diverse range of items in real-world scenarios, making it an invaluable asset in the realm of automation and robotics (see Figure 2).
After receiving bounding boxes from our recognition model, we extract and focus on the specific regions of interest within the depth image. This process streamlines subsequent analysis, improves computational efficiency, and ensures our algorithms work effectively by concentrating on the most relevant areas. This targeted approach reduces noise, optimizes resources, and enhances the overall performance of our computer vision system.
Trash modeling proceeds in two stages, first stage we split all points from selected areas into regions with heights equal to heights of gripper clams.
Then for each region we project all points into the bottom of the region, and find minimal bounding box.
Rotating calipers is a geometric algorithm that can be used to find the minimum bounding box (also known as the minimum area rectangle) of a set of points in a 2D plane. This algorithm is efficient and works by rotating two perpendicular calipers to envelope the points. Here are the steps to find the minimal bounding box using the rotating calipers algorithm:
Find the convex hull of the given set of points. The convex hull is the smallest convex polygon that contains all the input points. Author use Graham Scan algorithm.
Initialie two calipers (long and short) that are initially aligned with two edges of the convex hull. The long caliper should be aligned with the longest edge of the convex hull, and the short caliper should be perpendicular to the long caliper.
Calculate the area of the rectangle formed by the two calipers. This rectangle represents a potential minimal bounding box.
Rotate the short caliper counterclockwise by a small angle (e.g., one degree or a smaller increment) while keeping the long caliper aligned with the longest edge of the convex hull.
For each rotation of the short caliper, calculate the area of the rectangle formed by the two calipers and check if it's smaller than the previous minimum area found. If it is, update the minimum area.
Continue rotating the short caliper until it completes a full rotation, which is 180 degrees. Make sure to keep the long caliper aligned with the convex hull's longest edge throughout.
After completing a full rotation, you will have the minimal bounding box with the minimum area.
To find the actual coordinates of the minimal bounding box, you can use the endpoints of the long and short calipers and the orientation of the calipers.
The rotating calipers algorithm guarantees finding the minimum area bounding box efficiently with a time complexity of O(n), where n is the number of points in the convex hull. This method is particularly useful when you need to find the tightest bounding box (see Figure 3) for a set of points in 2D space.
Then we extend the bounding box back to the selected region height.
After meddling we received stacks of cuboids then we went from top to bottom, checking cuboids for selection.
If the sides are close enough to be clamped by a gripper we go to the next rectangle.
We check two scenarios: if the cuboid's lower part is bound to parallel sides of the top cuboid, we mark it as pickable.
If no last cuboid becomes our pickable option. This algorithm is described below:
Since the algorithm consists of two parts, experiments were initially conducted with the recognition algorithm. When it achieved satisfactory accuracy, experiments were then performed with the second algorithm. Therefore, the recognition algorithm was first investigated using 10% of the data from the TACO dataset. Additionally, a small amount of data collected by the article's authores, which accounts for approximately 1% of the TACO dataset, was used for testing. This approach was chosen because the TACO dataset is not very large.
The model performance mainly depends on correct classification, not precision of the bounding box, because the bounding box is good enough to decimate any noise or unwanted objects from the frame.
This sequential approach to experimenting and validating the two parts of the algorithm allowed for a more structured and efficient development process, ensuring that the recognition component was working well before moving on to the second part. It helped in identifying potential issues and fine-tuning the algorithm's performance (see Figure 4). The second part of the discussion focuses on an algorithm designed to streamline the process of selecting a pickup point, which has been rigorously compared with a more straightforward, naive pickup approach. In the case of the naive pickup method, the primary strategy involves reaching for the object's center point and attempting to grasp it. Now, let's delve into the specifics of these two pickup methods. In the case of the naive pickup, the objective is clear: reach for the center of the object and grab it. However, there are specific criteria that determine whether the pickup is considered successful. If the gripper fails to raise the object and hold it securely for at least three seconds, the attempt is considered unsuccessful. This time threshold of three seconds serves as a crucial benchmark for evaluating the efficacy of the naive pickup strategy.
It's important to note that both of these methods have demonstrated commendable performance when it comes to picking up squishable objects. The testing primarily focused on items such as pieces of paper and plastic bags, where both the algorithmic approach and the naive method exhibited a high degree of reliability and efficiency (see Figure 5).
However, the stark contrast between these two pickup methods becomes evident when dealing with solid objects. In such cases, the limitations of the naive pickup method become pronounced. One of the significant shortcomings of the naive approach is its failure to consider the orientation of the object, which results in frequent failures. Solid objects often exhibit irregular shapes and varying orientations, making it a challenging task for the naive method to consistently achieve successful pickups. This limitation underscores the importance of the algorithmic approach, which takes into account various factors, including the object's orientation, in order to increase the success rate of pickups, particularly when dealing with solid objects.
In summary, while both pickup methods excel at handling squishable objects, the algorithmic approach shines when it comes to solid objects due to its ability to adapt to varying orientations. This contrast highlights the significance of adopting more sophisticated strategies when dealing with complex, non-uniform objects in the realm of pickup and manipulation. The automatic detection of waste on the floor using Intel RealSense technology developed in this paper has important environmental, economic, and social implications:
1. Environmental importance:
• Recycling efficiency (аutomatic waste detection can promote efficient collection);
• Pollution prevention (quick identification).
• Resource optimization (efficient waste detection);
• Cost savings (by automating waste detection and collection).
• Improved hygiene (automatic waste detection);
• Awareness raising (introduction of waste detection technology can increase public awareness of waste management issues).
In conclusion, effective trash pickup and detection systems play a pivotal role in addressing the ever-growing global issue of waste management. As our world continues to grapple with the challenges of population growth, urbanization, and environmental degradation, it is imperative that we embrace innovative technologies and strategies to manage our waste more efficiently and sustainably. Trash pickup systems, ranging from traditional municipal services to advanced automated solutions, help keep our communities clean and hygienic, ensuring a healthier and more pleasant living environment. Moreover, by reducing litter and waste in public spaces, they contribute to the preservation of our natural habitats, protecting wildlife and ecosystems from the harmful effects of pollution.
The integration of detection technologies, such as sensors and machine learning algorithms, into waste management processes offers a promising avenue for improving efficiency and reducing environmental impact. These systems not only enable early detection of overflowing bins, but they also optimize collection routes, reducing fuel consumption and greenhouse gas emissions. This not only benefits the environment but also lowers operational costs for waste management services.
Furthermore, the use of data analytics and real-time monitoring allows for a more datadriven and responsive approach to waste management. This means that resources can be allocated more efficiently, and waste collection can be tailored to the specific needs of each community. Such data-driven systems can even encourage more responsible waste disposal practices among individuals, fostering a culture of sustainability and environmental awareness.
In summary, the convergence of traditional trash pickup services with cutting-edge detection technologies represents a significant step forward in our ongoing efforts to create cleaner, more sustainable, and environmentally conscious communities. The collective action of governments, businesses, and individuals, in tandem with the advancement of these technologies, will undoubtedly contribute to a brighter and more sustainable future. By harnessing the power of innovation and data-driven decision-making, we can take meaningful strides towards addressing the challenges of waste management in the 21st century and leaving a cleaner, healthier planet for generations to come.
The authors would like to thank the Armed Forces of Ukraine for providing security to perform this work. This work has become possible only because of the resilience and courage of the Ukrainian Army.