Competitive programming platforms like Codeforces are widely used for self-directed learning and assessment in computer science education. However, students often struggle to identify relevant practice problems or receive targeted help without personalized guidance. In this paper we present AlgoAce, an AI-powered assistant that integrates with Codeforces and uses retrieval-augmented generation (RAG) to provide support while attending at two key sources of information: user submission history, and problem metadata. The system leverages large language models to fetch relevant problems and past user submissions, grounding user queries in retrieved content before generating responses. AlgoAce adapts to user preferences-such as programming language or problem tags-and supports reflective learning through natural language interactions. We describe the tool's architecture and discuss its potential for integration into the competitive programming learning ecosystem.
LLMs have been widely applied to educational tasks, including student performance prediction,
personalized practice generation, and automated feedback [
Recent studies have explored programming exercise recommendation systems [
AlgoAce addresses this gap by incorporating users’ historical problem attempts and performance data into its recommendation pipeline. This enables the system to dynamically adapt to a learner’s evolving skill set, ofering targeted practice.
On the open source platform AlgoAce, users input a query, e.g. “What should I learn next?”. They can provide their Codeforces handle, which allows the agent to retrieve their submission history—notably successful and unsuccessful problem attempts—via the Codeforces API. If no handle is provided, generic recommendations are made based on the query alone. Sample outputs are provided in Appendix A.
Each Codeforces problem is represented by the following metadata: its title, associated tags (e.g., “shortest paths”), Elo rating, and URL, see an example in Table 1 in Appendix C. The diference in Elo rating between a user and a problem is proportional to the logit of the probability that the user can solve the problem. AlgoAce embeds this available metadata into a dense vector space to enable semantic retrieval. The user submission history contains metadata of the attempted problem and the result of the submission (OK, Wrong Answer, or Time Limit Exceeded), see an example in Table 2 in Appendix C. It is similar to the data encountered in knowledge tracing scenarios.
AlgoAce runs two parallel retrievers, one over user’s past submissions and another over the Codeforces problem set, both using vector similarity search. The retrieved results are merged and then fed into a structured prompt to guide the LLM’s generation. The system is designed with modularity in mind, currently supporting both API-based large language models such as OpenAI’s GPT models and locally hosted open-source alternatives, so that the data never leaves the device. The complete prompt templates are provided in Appendix B. A diagram of the whole pipeline is also provided in Figure 2 in Appendix D.
To evaluate the retrieval of submission history, we prioritized ensuring the relevance of retrieved documents. Our initial observations revealed a critical need for query reformulation: queries that didn’t explicitly define terms like "OK" or "time limit exceeded" consistently yielded irrelevant submissions. This led us to implement a two-way prompting strategy, where the first prompt is specifically designed to reformulate the user’s query, thereby improving the relevance of the retrieved submission history. Implementation details AlgoAce is implemented using the Pathway framework, which provides eficient Retrieval-Augmented Generation (RAG) pipelines. For approximate nearest neighbor search, we use Pathway’s KNNIndex class, which applies locality-sensitive hashing to perform fast similarity search across embedded problems. The user interface is built using Streamlit, with a Pathway RESTful API connecting the front-end to the backend services. All code from the platform is released on GitHub.
As future work, we plan to incorporate problem solutions into the retrieval and recommendation process, and conduct user studies to evaluate our tool and see the common queries made by users. By analyzing oficial or community-provided solutions, we can better estimate the prerequisite skills required for each exercise, leading to more accurate and pedagogically meaningful recommendations. Additionally, we aim to align the objective of the LLM more closely with learning outcomes, ensuring that the system prioritizes educational progress rather than optimizing user satisfaction. This could involve fine-tuning prompts to emphasize skill development and problem-solving strategies.
During the preparation of this work, the authors used LLMs like ChatGPT or Mistral in order to perform grammar, spelling check, and help with formatting. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.
Examples of the output generated by AlgoAce are provided in Figure 1. We observe that the LLM is explaining the recommendations made by looking at user submission history.
The query rewriting prompt is used to reformulate user queries with explicit metadata terms, improving retrieval accuracy: You are a query rewriting assistant for a competitive programming tool. The goal is to rewrite user queries so that they include concrete terms that match problem metadata stored in a database. The database retrieves problems based on word overlap with the query - so include exact terms like problem ’tags’ (e.g. ’greedy’, ’dp’, ’graphs’), verdicts (e.g. ’WRONG_ANSWER’, ’OK’, ’RUNTIME_ERROR’), difficulty ’ratings’ (e.g. 800,1000, 1700), or specific problem names if known. You do NOT have access to the user’s past submissions, but you must guess the user’s intent and include as many relevant keywords as possible to help the system match the query to relevant problems. Your output should be in the following format: Retrieval terms: <insert keywords for retrieval> Make sure the retrieval terms are rich in specific tokens, and the user query is clear and helpful for the LLM.
The response generation prompt guides the LLM in analyzing user submissions and generating tailored recommendations or explanations: You are a competitive programming coach.
I will give you the user code submissions and a list of all available problems after "Data:". The data lines with verdict are the user code submissions. The other lines are the available problems on codeforces. Answer the user query after looking at the user code submissions and the questions he got a wrong answer in. The problems you analyze and suggest should only be from the given data lines without verdict and should be similar in rating to the highest rated problems that the user has attempted. The answer should also explain how you arrived at the answer looking at the user code submissions if necessary. Keep your thinking short.
Example query : "What problems should I practice next?" Ideal Answer : "Looking at your submissions, you solve questions rated around 1500 and you are facing difficulty in implementing binary search. So here are some binary search problems you can practice: [Links to Binary search problems]." Example query : "How to implement dynamic programming?" Ideal Answer : "Here’s an implementation of dynamic programming. [Dynamic Programming Code].\n Data: \n {context} \n{query} /think.
Examples of Codeforces problems are provided in Table 1. An example of user submission history is provided in Table 2.
Our pipeline is provided in Figure 2.