Our research team has designed an educational technology called Waste Genie (WG) to support waste management learning. WG provides a series of technological features to engage informal learning, including interactive waste-sorting quizzes, a waste scanner to detect waste objects and the associated waste bin labels, and virtual carbon credits to help users quantify and visualize the environmental impacts. However, waste sorting alone is an immense and complicated problem. The recycling regulations vary by the city, the county, or even by the organization. We engineer an AI agent that utilizes GPT to provide adaptive feedback to support sorting. We hypothesize that the chatbot implementation is insuficient, because the users may experience limited vocabulary or have dificulty expressing their questions and concerns. The adaptive feedback is provided based on users' waste sorting attempts. A study was conducted with 54 participants to examine the efects of environmental awareness and waste management learning. The results showed increased waste sorting accuracy and eficiency, improved sustainability awareness, and positive efects of AI-generated feedback. Overall, the study demonstrated the feasibility of integrating the of-the-shelf AI agent to enhance educational technology and efectively support waste management learning.
In a world with escalating environmental concerns, it is
important to foster sustainable waste management practices.
Recently, researchers and organizations have undertaken
various eforts to support sustainability through improved
waste management. For instance, an assortment of
Artificial Intelligence techniques have been deployed to facilitate
waste sorting: deep learning algorithms to accurately
classify trash images [
With the rapid growth of the Large Language Model
(LLM) across fields in the past year, there is mounting
evidence to illustrate LLM-based applications and benefits
[
In this work, we design and conduct studies to investigate the following research questions: 1. Impact of AI-Guided Assistance: How does the integration of an AI agent impact the overall user experience? 2. Efectiveness in Learning: To what extent does the GPT-powered feedback contribute to users’ increased knowledge and awareness of waste management? 3. Usability and User Engagement: How do the users perceive the overall Waste Genie usability and to what extent do they engage in the WG platform?
Recent advances in large language models (LLMs) have
opened up new possibilities for educational content
generation. Several research studies have demonstrated the
benefits of employing LLMs. For instance, students felt
they learned and perceived less dificulty in solving math
problems with the help of LLMs [
Adaptive feedback dynamically tailors the instructional
guidance to individual learners based on their progress and
needs, reflecting a careful design consideration of all
available information [
There have been drastic technological advancements in
improving waste management these days. Research ranges
from (a) the power of deep learning to identify wastes in
medical fields [
Waste Genie (WG) is a web-based educational technology,
designed to support waste management learning through
reading and practicing waste sorting [
The objective of sorting wastes is to classify each waste object into one of four categories: Recycle, Landfill, Compost, and Hazard. Each quiz consists of three waste items 1–8 (Fig. 1b). Users interact with the quizzes by dragging and dropping the category labels onto the waste objects. The corrective feedback (correct or incorrect) will be indicated by the visual elements (green check mark and red x mark). Meanwhile, the AI agent is actively engaged when any incorrectness is detected. A quiz is completed when all of the objects are labeled correctly (Fig. 1c). Upon completion, users will be notified with an estimated amount of virtual carbon credits. An educational tip section will also be compiled and made available once the sorting is accomplished. The tip is an additional layer of information where the users can consolidate their learning by reading more detailed information on how to properly process the waste items featured in the quiz. It is intended to promote deeper reflective learning in the practices. Finally, a leaderboard (Fig. 1d) of virtual carbon credits is accessible for the users to comprehend and visualize the real-world impact of the properly sorted wastes. Two primary metrics, mileage-travel-by-car and tree-days equivalent (the amount of CO2 sequestered by a mature tree per day), are employed as the quantitative estimations to represent the significance of the carbon saved.
To investigate the efects of GPT-guided feedback for waste management learning in WG, we specifically engineered the AI agent to support feedback adaptation in the context. Fig. 2 demonstrates the overall architecture of our research platform - Waste Genie.
Interactive waste sorting quizzes are the essential learning
content in WG. A quiz comprises two key elements, the
waste items and their corresponding category labels. The
association between the item and the label constitutes the
corrective information for proper sorting (A.K.A. feedback
for the users). In this study, we sourced 96 waste items from
California’s recycling guidelines [
To supply unlimited interactive quizzes in WG, we first
exploited the option of generative AI quizzes, so users can
refresh the content feed to demand more content whenever
they want. Many text-to-image models have demonstrated
the capability of textual guidance in creating visual content,
such as DALL· E 2 [
(b) Interactive quiz with real-time
AI assistance.
In this paper, since we have already collected the ground truth labels of the waste items, we just need to assemble the ground truth information with the associated imagery without generating everything artificially from scratch.
Thus, in our study, each quiz used was generated by
randomly selecting three items from our pre-collected dataset
(Section 3.2.1.1). The selected waste items were firstly
associated with corresponding images crawled from Google
open image search. Next, The waste items were placed on
a canvas with a clear background generated by DALL· E 2
[
The AI agent, powered by GPT-3.5 1, plays a pivotal role in guiding the users during waste sorting. When a user makes a mistake, the AI agent receives the context of the current quiz, including the item that causes the user’s confusion, the user’s erroneous choice, and the correct answer. It is then instructed to generate a concise explanation to help the user move towards the correct answer. For instance, when a user misclassifies a recyclable paper bag as “landfill", the AI agent will be instructed that the user is confused about labeling a paper bag to recycle, and falsely labeled it as a landfill. The agent is asked to generate a concise explanation in a few words, and to avoid direct disclosure of the answer, to guide the user to correct their mistake. In this way, the agent will ofer insights into the item’s materials and the environmental impacts of disposing of them so that users can make another choice based on the clues they are ofered. Additionally, a more detailed tip will be summarized and made available after the completion of the quiz. The AI agent will also compile a comprehensive and informative summary as a tip (Figure 1c), to reflect and illustrate all the information about the waste items in the quiz, their corresponding categories, and nevertheless, the users’ sorting processes.
Waste Genie introduces the concept of virtual “carbon
credits”, which was initially issued by the United Nations [
A total of 54 students were recruited from two web usability classes at the author’s university. Students were introduced to the study as part of the learning activity. Prior to using the Waste Genie platform, each participant was asked to complete a pre-survey that included basic sustainability awareness questions and waste categorization knowledge tests. There were 15 waste items being asked to classify into one of four categories: recycle, landfill, compost, or hazard. This pre-survey collected a baseline understanding of the participants’ existing knowledge and attitudes toward environmental sustainability.
Participants were then asked to use Waste Genie over two days to complete a series of 30 pre-generated interactive waste sorting quizzes. Each participant received the same quizzes in the same order. During this period, participants’ activities, including the time spent to complete each question, the steps used to complete a quiz, the mistakes they made, and the interactions with the AI agent, were recorded for the following data analysis.
A post-survey was distributed among the users after the
completion of quiz sorting. The questionnaire included the
same set of sustainability awareness questions, a
reassessment of their waste classification knowledge, and a WG
usability survey. The system usability scale (SUS) questions
[
During the study, we collected 4,914 waste-sorting actions performed by the 54 participants on WG. We examined these participants’ pre-knowledge distribution to analyze the diferences and learning efects. A threshold of waste categorization accuracy of 0.8 was found based on the median accuracy in the pre-survey. We then distinguish these users into two groups, lower performing group (LPG, 33 users) and higher performing group (HPG, 21 users). This classification enabled us to analyze and compare the performance and responses of participants with varying levels of proficiency in waste classification. The participants with accuracy below or equal to the threshold were assigned to the LPG, while those with accuracy above the threshold were assigned to the HPG. To evaluate the efectiveness of the Waste Genie platform for improving waste management skills (RQ2), we measured the accuracy and time taken per quiz for each user and conducted an ANOVA analysis. The results from the waste sorting knowledge test and the sustainability awareness questions in our pre- and post-study surveys were also compared to evaluate the knowledge growth. System usability (RQ3) was assessed by calculating the SUS score from the post-study survey responses. The open-ended feedback also provided qualitative insights into our evaluations of users’ experiences. As for the impact of AI-guided assistance (RQ1), we evaluated it by comparing users’ accuracy in sorting waste items before and after receiving AI feedback. The number of AI guidance triggered per user was also tracked to gauge their engagement with the agent.
One of the underlying hypotheses in WG is that people can learn waste management from sorting virtual waste and reading organized waste management information. To collect the evidence of learning, we first measured each user’s sorting accuracy throughout the 30 assigned quizzes. We then analyzed the trajectory of the waste sorting accuracy over time. We found that the overall sorting accuracy was 0.78, the average total sorting time spent per user was 294.8 seconds, and the average time spent on WG was 914.6 seconds (roughly about 15 minutes).
To dive deeper into the analysis, we found that the users showed an improvement in accuracy as they progressed through all the quizzes. As shown in Table 1 and Fig. 3, the average accuracy significantly ( < 0.01, Cohen’s = 0.44) increased from the first 6 quizzes ( = 0.79, = 0.18) to the last 6 quizzes ( = 0.87, = 0.18). The users also illustrated a steady improvement. To put it in perspective, we also used the amount of time spent per quiz to examine the waste sorting eficiency. We found that the users demonstrated a significant time reduction in waste classification ( < 0.01, = 0.71) at the end of the sorting, from 15.34 ( = 12.99) seconds to 7.5 ( = 8.48) seconds. Such a finding is not trivial, it showed eficient waste sorting practices just within less than 5 minutes of overall use in WG. This trend was consistent across both the LPG and HPG. Despite variations in initial sorting proficiency, both groups showcased similar patterns of improvement in accuracy (0.79 to 0.88 for LPG and 0.80 to 0.87 for HPG) and decreasing sorting time (14.97s to 6.81s for LPG and 15.88s to 8.52s for HPG) as they engaged with WG. It is worth noting that the entire participant group illustrated a more coherent sorting eficiency after they had progressed 40% of the quizzes (the time spent was less than 9 seconds and the standard deviation was small, ranging from 5.04 to 8.48).
Additionally, based on the knowledge tests in pre- and post-study surveys (see Table 2), the users showed overall knowledge growth to sort diferent types of waste. The overall average score improved from 0.79 in the pre-test to 0.91 in the post-test ( < 0.01). The compost type achieved the highest score in the post-test, nearing 100%. This suggests that the users comprehend well the concept of composting throughout the entire WG experience (sorting exercises and feedback). On the other hand, the Landfill category presented the greatest challenge among all types. The users exhibited a low average pre-knowledge score (67%) to begin with, and the LPG (57%) and HPG (83%) groups appeared to have a big accuracy gap. Although all users managed to increase their overall average accuracy to 81% in the post-test, HPG showed a slight knowledge score drop, which means the overall knowledge growth was predominately attributed to the LPG’s growth. Such a result may be attributed to the unique definitions of compost, recycle, and hazardous wastes compared to the more ambiguous nature of landfill wastes. For instance, a piece of food-stained plastic consists of the key ingredients to dispose of in the compost and recycling categories. However, depending on the materials and the cleanliness, the item will be expected to be sorted into the landfill bin. Overall, these findings underscore the complexity and ambiguity of waste types, and the necessity of adaptive feedback, and suggest avenues for further research to enhance learning outcomes in waste management education.
With the improved waste sorting accuracy and eficiency over time, we were motivated to find out how much of the improvement was attributed to the AI-generated feedback. We looked into the statistics of tips and feedback usage to analyze what happened before and after the feedback was provided. We found that on average, each user received approximately 30.96 times AI-generated content. This number means that on average a user will interact with the AI agent at least once per quiz, either by making the mistake and receiving the feedback or explicitly clicking on the tips to review the sorting summary. It showcased their active engagement with the AI agent. Notably, users exhibited an increase in accuracy after receiving AI guidance, with an average ifrst-attempt success rate of 0.76 compared to 0.69 before the assistance (Table 3). Such an efect was found in both LPG and HPG groups. Specifically, HPG showed significant accuracy improvement after the AI guidance ( < 0.0167), from 0.68 to 0.79 (Cohen’s = 1.16). The p-value was adjusted with Bonferroni correction with = 0.05. It is reassuring to learn that AI-generated feedback has a positive influence in helping users achieve a higher first-attempt success rate. The finding is also encouraging to learn that HPG which has already had higher pre-knowledge can capitalize on the AI guidance and improve itself further.
Furthermore, based on the post-survey, users also provided high ratings on the helpfulness of the AI-generated content ( = 3.64, = 1.11) and the AI agent’s contribution to their understanding ( = 3.68, = 1.04). These subjective assessments along with the observed improvements in the data logs suggested that the AI agent indeed enhanced the waste sorting efectiveness.
The pre- and post-study sustainability awareness questions from the surveys revealed that our platform had a positive impact on increasing users’ knowledge and attitudes to sustainable waste management practices (Fig. 4). Users’ core concepts such as “distinguishing between compostables and landfills" increased from 3.54 (standard deviation = 0.98) to 4.05 ( = 0.81), “understand the diferences between composting and recycling" increased from 4.00 ( = 0.96) to 4.27 ( = 0.81), ratings on “know how to do proper waste sorting" increased from 3.48 ( = 0.84) to 3.93 ( = 0.86), and ratings on “confidence in my ability to properly sort the daily waste" increased from 3.55 ( = 0.88) to 3.79 ( = 0.86).
Interestingly, the users found an increasing understanding of carbon emission on waste sorting and their daily waste carbon footprint (Q1 & Q2 - carbon emissions with the sorted waste ( = 3.38, = 1.09) and daily waste carbon footprint ( = 3.04, = 1.10), but the ratings of these two questions were still considered relatively low. After the exposure to sequences of waste sorting quizzes and AI feedback in WG, the users indicated their awareness and consciousness level increased, but the concept of the association between waste sorting and carbon emission is still arguably insuficient (just a bit higher than the neutral reference point in the Likert scale).
According to the System Usability Scale [
In this work, we proposed to leverage the power of of-theshelf GPT to generate adaptive feedback to facilitate waste sorting. We implemented the AI agent and integrated it into the web-based application Waste Genie (WG). We designed and conducted a user study to evaluate the efectiveness of learning and the impact of AI-generated feedback. The results demonstrated the feasibility of leveraging such AI agents to enhance educational technologies and efectively support waste management education.
Participants exhibited increased waste sorting accuracy and eficiency as well as improved sustainability awareness after engaging with WG. The AI-generated feedback had a positive impact on users’ first-attempt success rates for waste sorting (RQ2), highlighting the potential of using of-the-shelf GPT models to provide concise guidance in the complex domain of waste classification. Furthermore, users demonstrated positive knowledge growth and heightened sustainability awareness (RQ1), underscoring the educational implications of WG, particularly in the context of informal lifelong learning and the global concern surrounding waste management and sustainability education. Waste Genie’s usability was again validated by the users with overall high ratings (RQ3), it showcased the viability of the integration of AI feedback in WG.
Several limitations and avenues for future research have been identified during our study. For instance, the AI explanations were sometimes too brief and could have been better if there had been more conversational interactions. One user suggested that a smarter arrangement of quizzes and pace of explanations, such that sorting similar wastes together may aid in the reinforcement of knowledge.
Future iterations of our work could address these issues by further tuning the AI agent for dynamic and deeper dialogue; optimizing the quiz generation algorithms to add adaptive features for personalized sequences and paces; and providing more types of challenges to boost the learning process. 1–8