Several research eforts have addressed the problem of characterizing student mistakes in database query writing, among which the SQL error taxonomy proposed by Taipalus et al. [1] has become a widely adopted framework. In this paper, we report ongoing work stemming from our experience in operationalizing this taxonomy within an automated SQL error-detection pipeline, where we encountered practical limitations and ambiguities arising when taxonomy categories are interpreted algorithmically rather than by human annotators. Building on extensive manual annotation and iterative refinement conducted during the development of an automated SQL error correction tool, we outline a practice-driven revision of the taxonomy aimed at supporting fine-grained automated error classification. The proposed revision focuses on refining error definitions, regrouping certain categories, clarifying labels, and addressing gaps and redundancies identified in the original taxonomy. We conclude by discussing how the revised taxonomy can support educational tools and by outlining future directions for validation and empirical assessment.
Data systems education represents a challenging component of many computer science degree
programmes. While research on query language education has recently gained increased attention, it
remains less developed than research on programming language education. Declarative query languages
such as SQL may appear intuitive, yet students frequently struggle to master them. A well-documented
source of dificulty lies in the contrast between SQL’s declarative, set-based foundations and the
imperative, instance-based paradigms typically encountered in CS1 courses (see, e.g., [
Research on SQL education has produced valuable results, including taxonomies for classifying student
errors [
Our work builds on these contributions through the development of the LenSQL tool [
This paper presents ongoing research aimed at addressing this gap by revising the taxonomy in [
The revised taxonomy aims to bridge educational research and intelligent support systems. By grounding error categories in machine-detectable properties while retaining their educational meaning, it supports reliable automatic classification, targeted feedback generation for students, and aggregated analyses that can inform instructors and, in future work, support reasoning about underlying misconceptions. At the same time, the taxonomy remains interpretable for researchers and educators conducting manual analyses of student performance. Thus, we strengthen the pedagogical value of error classification by making categories actionable for targeted feedback, personalized tutoring, and data-driven exercise generation.
The contributions of this paper are as follows: (i) a systematic, practice-driven revision of an established SQL error taxonomy to support fine-grained automated classification; (ii) refinement and reorganisation of error categories to remove ambiguities and redundancies while introducing machinedetectable distinctions; and (iii) the definition of a framework that connects educational research with automated assessment tools, enabling actionable feedback and data-driven support.
The remainder of the paper reviews related work, outlines the requirements and principles guiding the revision, presents preliminary revisions to the taxonomy, and discusses ongoing and future research directions.
In this section, we briefly discuss the relevant related work in SQL error classification, identification of frequent students misconceptions, and automatic correction of SQL queries.
to more advanced semantic and logical misinterpretations, errors in SQL queries can take many forms.
The idea of analyzing and classifying (frequent) student errors is not new: Brass and Goldberg [
Identifying Misconceptions Taipalus et al. [
The methodology adopted in this work is grounded in the practical task of implementing an automated SQL error detector based on a widespread taxonomy. Rather than proposing theoretical refinements in isolation, the revision process emerged from a systematic, error-by-error operational analysis conducted during the development of the detector. This section describes the analytical procedure followed, the criteria used to identify limitations of the original taxonomy, and the principles guiding the proposed revisions.
As a first step, each error type in the reference taxonomy was translated into a set of conditions that an automated system could verify on a student query. This process required making explicit the assumptions underlying each error label, such as whether the query must fail to execute, whether comparison with a reference solution is required, or whether schema metadata inspection is necessary.
During this phase, several ambiguities between errors and student misconceptions became apparent. For instance, the syntax error “restriction in SELECT clause (e.g., SELECT fee > 10)” refers to a misconception about SELECT and WHERE clauses, rather than invalid SQL syntax (SELECT fee > 10 returns a boolean value for each row). These cases motivated a shift from intent-based definitions (i.e., maybe the student is confused about the role of the clauses) to criteria grounded in observable query behavior (i.e., the query cannot be executed because of a syntax error).
The detector was incrementally implemented by testing real and synthetic student queries against each error definition. For each query-error pairing, we assessed whether the taxonomy allowed for: (i) a clear and unambiguous classification, (ii) a feasible algorithmic detection strategy, and (iii) a meaningful interpretation in view of future mappings to student misconceptions.
This analysis led to the identification of three recurrent situations: (i) queries that could reasonably be assigned to multiple error categories depending on interpretation, (ii) queries for which no existing error category provided an adequate description, (iii) error categories whose definition hindered a principled alignment with misconceptions rather than query properties.
These situations were systematically documented and used as evidence for revision decisions.
Based on the identified issues, revisions were guided by a set of methodological principles. First, precise naming criteria were adopted to prevent ambiguous error definitions. Second, error categories were redefined to rely exclusively on properties that can be derived from the query itself, optionally in relation to schema metadata or correct reference queries1. Third, overlapping or weakly diferentiated categories were merged or re-grouped to reduce classification ambiguity.
Importantly, no revision was introduced solely to simplify implementation. Instead, changes were adopted only when the original formulation systematically hindered consistent detection or obscured the pedagogical interpretation of the error.
Although the present work focuses on taxonomy revision rather than misconception modeling, the methodology explicitly considers future mappings between errors and misconceptions. Categories that encode multiple conceptual failures or that depend heavily on inferred student intent were flagged as problematic for such mappings. By enforcing clearer, behavior-based definitions, the revised taxonomy is designed to act as a stable intermediate layer between raw student queries and higher-level cognitive interpretations, ensuring that it remains pedagogically meaningful while being suitable for automated error detection and scalable educational tools.
A comprehensive and up-to-date version of the taxonomy is available online2, where illustrative
examples are provided for each error. Representative queries highlighting the relevant revisions to
the taxonomy are reported in Table 1. The underlying schema, described in Table 2, is adapted from
Miedema et al. [
We propose using the following naming criteria to reduce ambiguity in the error definitions: Table Reference We suggest renaming “join” into “table reference” whenever we mean which tables are included in the FROM clause, since “join” fails to capture errors in the first element of a FROM clause. For example, if a student wrote SELECT * FROM customer JOIN store ON ... instead of SELECT * FROM inventory JOIN store ON ..., technically it is not a join on an incorrect table error, since the joined table is correct. However, this error can still be described as an incorrect table reference. Join Condition We suggest renaming “join” into “join condition” whenever we mean a condition applied between columns of two diferent tables. This condition can be either in the JOIN ... ON ... clause or at the top level of the conjunctive normal form of the WHERE clause (i.e. under implicit join syntax). This clarifies whether “join” refers to the presence of a table in the FROM clause or to a condition between columns of these tables.
Ambiguous Columns We noticed that error #1 “omitting correlation names”, as discussed in [27] and [28], refers to referring to a column present in at least two tables referenced in the query without specifying which table the column belongs to. Similarly, error #2 “ambiguous column”, as discussed 1We assume that (at least) one correct reference query for each data demand is available, which is reasonable in our intended usage scenarios. 2https://github.com/DavidePonzini/sql_error_taxonomy
SQL statement SELECT
pID, P unit_price > 10 FROM inventory; SELECT street FROM customer
GROUP BY street; 1 2 3 4 5 6
Request Select all items IDs and whether they have a unit price greater than 10.
Select the IDs of items have a unit price greater than 10.
Select all street names without duplicates Select all street names in alphabetical order Select how many customers live in each street
Original taxonomy 28. Restriction in SELECT
clause 28. Restriction in SELECT
clause 31. Confusing the logic
of keywords 31. Confusing the logic
of keywords 31. Confusing the logic of keywords
Revised taxonomy (No errors) 70. Extraneous column in
SELECT 66. Missing expression 97. GROUP BY can be
replaced with DISTINCT Extraneous GROUP BY Missing ORDER BY 13. Data type mismatch 17. Strange HAVING (Not relevant) (No errors)
Missing quantifier (No errors) (No errors)
Diferent tuples in set operation Correlation name identical to table name SELECT street, COUNT(*) FROM customer HAVING street; SELECT * FROM inventory WHERE unit_price >= (
SELECT unit_price FROM product NATURAL JOIN inventory
WHERE pName = ’Banana’
);
SELECT city, street
FROM store
INTERSECT
SELECT city
FROM customer;
SELECT * FROM store AS store;
(Not relevant)
(Not relevant)
in [
Restriction in SELECT Clause Error #28 “restriction in SELECT clause” corresponds to a query
which is actually syntactically correct. A SQL query can use operators and comparisons in the SELECT
clause: the returned value is the result of the expression. For instance, using the example provided
in [
Projection in WHERE Clause Error #29 “projection in WHERE clause” refers to a student misconception, rather than to an actual syntax error. We suggest using error #32 “confusing the syntax of keywords” for this error, since the problem can be described as incorrect syntax for the WHERE clause, since it should be provided with an expression that can be evaluated to a Boolean.
Confusing the Logic of Keywords Detecting error #31 “confusing the logic of keywords” can be very complex, even for a human, and does not always result in a syntax error. For instance, query 2 executes correctly, and as such presents no syntax errors. As shown in the example, categorization depends heavily on the data demand. On the other hand, query 3 cannot be executed, since the condition in the HAVING clause cannot be evaluated to a Boolean value. We can argue that the user could have meant to group by street instead of applying a HAVING condition on a string, but without further context (which is not needed for detecting syntax errors), we have no way of knowing for sure. Our suggestion is simply to apply the other categories, focusing on the actual query, instead of a supposed user intention. Subquery Errors We suggest creating a new category, called “Subquery errors”, that contains error #26 “too many columns in subquery” and a new error: “missing quantifier”, which refers to not using a quantifier ( ANY/ALL) with a subquery that could return more than one row. Consider query 6: if the join between product and inventory contains at most one ’Banana’, the query can execute successfully, otherwise the DBMS throws an error. Even though the query is not necessarily syntactically incorrect, tagging a query with this error if a comparison is performed without a quantifier and the subquery can potentially return multiple rows (i.e., the subquery is missing a condition on primary/unique keys) would prevent potential future execution errors (i.e., upon changes in the database instance).
brackets” into two more detailed errors, namely “curly or square brackets” and “unmatched brackets”. These errors reflect diferent misconceptions in students: the former means that students are not aware that they can only use parentheses in SQL, while the latter could be, for example, attributed to distraction.
IS where not Applicable We suggest moving error #35 “IS where not applicable” from SYN-6 Common syntax error to SYN-3 Data type mismatch, since using the IS operator expects either NULL or a Boolean value. Any other value, e.g., a string, triggers a data type mismatch, and as a consequence, we believe this category to be more appropriate.
Diferent Tuples in Set Operation We suggest adding a new error, namely SYN-6 (Common Syntax Error): diferent tuples in set operation. This syntax error is caused by having two queries that return tuples with diferent cardinalities combined with a set operation ( UNION/INTERSECT/EXCEPT). For example, query 9 is invalid, since the first SELECT returns two columns, while the second only one.
AND instead of OR (Empty Result Table) An invalid AND condition which always results in an empty result table is already accounted for in error #40 “implied, tautological or inconsistent expression”. Moreover, a valid condition which uses AND instead of OR and does not result in an empty result table (i.e., non mutually-exclusive conditions) is not accounted for in the taxonomy (the only similar error is “using OR instead of AND”, but this is the opposite scenario). In light of this, we suggest renaming this error to a generic “AND instead of OR”, independently from the cardinality of the result set, and considering it a logical error.
Wildcards without LIKE We suggest considering error #43 as logical, since we cannot determine a priori if a user actually meant to equate a string to a wildcard character.
Incorrect Wildcard Similarly, we suggest considering error #44 as logical, since using a wildcard character in a string equation is not always wrong. Additionally, we suggest splitting this error into two more detailed errors, namely “wrong wildcard” and “invalid wildcard”. The former error refers to using ‘_’ instead of ‘%’, or vice-versa, and highlights dificulty in understanding which wildcard to use. The latter refers to using characters which are interpreted as wildcards in common pattern matching languages but are treated as regular characters by the LIKE operator (e.g. using ‘*’ instead of ‘%’, or ‘?’ instead of ‘_’).
Omitting a Join We suggest considering error #48 as logical, since Cartesian products between tables are supported by the SQL standard and may be desirable in some scenarios. However, by knowing the data demand, we can easily determine whether a Cartesian product is expected by the data demand. Additionally, we suggest renaming this error to “missing join condition”, to better align with our naming convention.
Putting NOT in Front of Incorrect IN/EXISTS Error #56 is a subset of errors #53 “extraneous NOT operator” and #54 “missing NOT operator”. This error is more specifically related to queries that have two nested subqueries that use IN/EXISTS operators. Since, in our experience, this is a pretty niche scenario, we believe no further distinction is needed. Additionally, the error definition is not entirely clear: are at least two IN/EXISTS, one of which should be negated, necessary for this error? What if the query contains only a single IN/EXISTS operator or if both operators are supposed to be negated but only one is? To solve these problems, we suggest counting only if the NOT operator should (not) have been present in the logical expression.
to better align it with the other error names.
Missing Table Reference We suggest renaming error #62 “missing join” to “Missing table reference”, to better distinguish it from error #48.
Clause Errors We suggest adding a new error category for when a SQL clause is used when not needed, or vice-versa. This category contains the following errors: • Missing clause: a SQL clause, which is required by the data demand, is not present at all. Not using a clause highlights a diferent misconception in students than using that clause incorrectly, and as such should be treated separately. • Extraneous clause: a SQL clause not required by the data demand is present in the query. This scenario is only accounted for the ORDER BY clause (error #76), but we suggest considering all SQL clauses. • Incorrect limit: a LIMIT clause is required by the data demand, and the value provided in the query is incorrect. • Incorrect ofset: an OFFSET clause is required by the data demand, and the value provided in the query is incorrect.
Correlation Names are Always Identical We suggest splitting error #86 into two more specific
errors, to better align with the two possible interpretations of this error. On the one hand, as described
by [
Condition on Left Table in LEFT OUTER JOIN As the name suggests, error #104 does not consider
conditions on the right table for LEFT OUTER JOINs, as well as any condition in RIGHT/FULL OUTER
JOINs. Brass et al. state that “conditions on the right table do make sense in the left outer join condition” [
AND s.sname = ’Coop’; • SELECT c.*, s.* FROM customer c RIGHT OUTER JOIN store s ON s.city = c.city AND s.sname = ’Coop’; First, we suggest expanding this error to consider all conditions in the JOIN ON clause that involve either the left or the right table, as well as considering all outer join operations (LEFT, RIGHT, FULL). Additionally, we also suggest considering this error logical, since it can afect the resulting data set. Unused CTE We suggest adding a new complication for defining a CTE (Common Table Expression) in a query and then not referencing it anywhere else.
In this paper, we presented a practice-driven proposal for revising the error taxonomy in [
The reported work is ongoing, as further refinement is needed within the Logical Errors category, which was only partially addressed in this study. In particular, operator- and expression-related errors still exhibit conceptual overlap and boundary ambiguities. Errors involving redundant, missing, or inconsistent logical expressions are undergoing systematic revision grounded in formal properties of expressions and empirical observations from student queries.
Moreover, the revised taxonomy should be regarded as a grounded proposal rather than a consolidated or exhaustive classification. While the changes presented here are informed by concrete implementation experience, further work is needed to examine how the taxonomy is applied by human annotators in practice and how it performs in fully automated settings. Studying annotation behavior, sources of disagreement, and the interaction between manual and automatic annotation can guide iterative refinements of category definitions and guidelines. In this perspective, the creation of a publicly available benchmark of manually annotated SQL queries would be a valuable contribution, supporting reproducibility and comparative evaluation of future error detection approaches, and strengthening the taxonomy’s reliability.
During the preparation of this work, the authors used ChatGPT in order to: Paraphrase and reword, Improve writing style, Abstract drafting. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. Autograder for Assessment of SQL Queries, International Journal of Artificial Intelligence in Education (2025) 1–31. [24] C. Kleiner, F. Heine, Enhancing Feedback Generation for Autograded SQL Statements to Improve Student Learning, in: Proc. of the Int’l Conf. on Innovation and Technology in Computer Science Education, ITiCSE 2024, 2024, p. 248–254. [25] Z. Chen, S. Chen, M. White, R. Mooney, A. Payani, J. Srinivasa, Y. Su, H. Sun, Text-to-SQL Error
Correction with Language Models of Code (2023). arXiv:2305.13073. [26] A. AlRabah, S. Yang, A. Alawini, Optimizing Database Query Learning: A Generative AI Approach for Semantic Error Feedback, in: ASEE Annual Conference and Exposition, Conference Proceedings, American Society for Engineering Education, 2024. [27] C. Welty, Correcting User Errors in SQL, International Journal of Man-Machine Studies 22 (1985) 463–477. [28] J. B. Smelcer, User Errors in Database Query Composition, International Journal of HumanComputer Studies 42 (1995) 353–381.