Preface
The notion of intelligent systems that “care” about students is at the center of ITS
research [
Caring assessment systems are defined as systems that provide students with a
positive assessment experience while improving the quality of evidence collected about
the student’s knowledge, skills and abilities (KSAs) [
Taking a test is not just a passive mechanism for assessing how much people know.
It actually helps people learn, and it works better than a number of other studying
techniques [
This workshop provides a great opportunity for ITS and assessment researchers to
share information about the potential of applying ITS techniques and approaches in the
development of a new generation of caring assessments. Examples of ITS technologies
that have been successfully used for assessment purposes include automatic scoring of
essays and short responses [
Eight recognized members of the research community were invited to serve as members of the program committee. Each member reviewed up to two submissions. The program committee members are: Ivon Arroyo, Worcester Polytechnic Institute; Ricardo Conejo, University of Malaga; Vania Dimitrova, University of Leeds; Sidney D’Mello, University of Colorado Boulder; Art Graesser, University of Memphis; G. Tanner Jackson, Educational Testing Service; Irvin R. Katz, Educational Testing Service; and Steve Ritter, Carnegie Learning.
Seven papers were submitted and all of them were accepted for presentation at the workshop. Each paper received feedback from at least two reviewers. The accepted papers include: When Should an Adaptive Assessment Care? (Blair Lehman, Jesse R. Sparks, and Diego Zapata-Rivera); Incorporating Emotional Intelligence into Assessment Systems (Han-Hui Por and Aoife Cahill); Diagnostic Assessment of Adults’ Reading Deficiencies in an Intelligent Tutoring System (Genghu Shi, Anne M. Lippert, Andrew J. Hampton, Su Chen, Ying Fang, and Arthur C. Graesser); Tower of Questions: Gamified Testing to Engage Students in Peer Evaluation (Nafisul Islam Kiron and Julita Vassileva); Exploring Gritty Students’ Behavior in an Intelligent Tutoring System (Erik Erickson, Ivon Arroyo, Beverly Woolf), Disengagement Detection Within an Intelligent Tutoring System (Su Chen, Anne Lippert, Genghu Shi,Ying Fang, and Arthur C. Graesser); and Assessments That Care About Student Learning (Stephen E. Fancsali and Steven Ritter).
These papers offer different perspectives and current research toward the goal of making “caring” assessments part of the educational milieu.
The workshop included a thought-provoking discussion section that covered topics such as: • The need for educating the public on the characteristics of different types of assessments and their appropriate use. • Alternate criteria for adaptive testing that not only take into account the difficulty and the sequencing of questions but also other aspects of the student and the learning context and way of interaction. • Assessments that provide additional feedback/guidance on content related issues and testing strategies (e.g., time management warnings). • Using student model information from formative learning environments to inform the assessment systems. • Possible approaches for integrating emotion data into assessment. • Strategies for engaging students in peer assessment gaming activities. • Exploring connections with other research areas (e.g., persuasive technologies). • Evaluating the effects of additional features on test reliability, validity, and fairness.
We thank the authors for submitting relevant papers to the topic of the workshop, the program committee members for their time reviewing and providing constructive feedback to the authors, and the ITS workshop organizers, Nathalie Guin and Amruth Kumar, for providing us with this great opportunity to convene and address this topic. Best regards, Diego Zapata-Rivera and Julita Vassileva