Conducting a student centered discussion in a mathematics classroom is not a trivial task. A teacher must follow their students work, and then use the relevant information in order to conduct a meaningful discussion. This challenge is greater when digital environments are involved, or when the students work remotely and only submit their work online. One possible solution for this challenge could be in the form of accessible learning analytics that could assist the teacher to gain insight about their student's work. We report on a formative assessment platform that automatically analyzes student submissions and characterizes them according to preset conditions that are topic specific. These characterizations are then used in several interactive reports that form a teacher's dashboard that is designed to enable multiple levels of analysis by the teachers in planning a classroom discussion. We describe the different components and demonstrate their use in mathematics classrooms in Israel.
Teaching mathematics with technology requires specific competencies [
In Blended learning settings teachers are required to assess in real time their student’s understanding
in multi-participant classes and direct their teaching accordingly. Many technological learning
environments have been developed to provide the teacher with an immediate snapshot of students' work
[
Technological learning environments could provide solutions for teachers when they provide the
teacher with an immediate picture of their students' work. Each environment offers different learning
analytics (LA) for the teacher. LA is sometimes referred to as a single display that aggregates various
indicators about learners, learning processes, and learning contexts into one or multiple visualizations.
More studies are needed on the long-term effects and affordances of learning dashboards if these
technologies are to become part of the common toolbox of teachers and learners in years to come [
Researchers have different attitudes toward the potential of technology in changing teaching
practices. On one hand, some researches realized that the potential of technology to change teaching is
unlikely to be significant [
In the following we present STEP’s dashboard components. STEP currently has six different reports include: a table report, grid report, histogram, Venn diagrams, perceptual landscape, and bubbles report. Each of these components was designed to address specific pedagogical needs, and with these design principles and usage ideas was introduced to mathematics teachers in professional development programs. In this report we will focus on four of the components: table report, grid report, histogram report, and Venn diagrams. For each component we will present separately the goals, mechanisms, and use cases for a task that was enacted in Israeli middle schools (grades 8 and 9) on the topic of functions, during the 2020-21 school year.
The grid report presents a snapshot of each one of the student submissions in a collage (Figure 1), in a presentation that resembles many other platforms (e. g. TI-Inspire). The goal of this report is to enable teachers to have all of the classroom’s student’s work accessible in a single report, while enabling the teacher also to filter student work according to pre-designed characteristics. This filtering process can assist the teacher in locating different phenomena in the students’ work rather than just go over all of the snapshots trying to locate work that is relevant for the ensuing classroom discussion of next task.
The report has an anonymous mode as well as an identified mode for the display of student answers so that a teacher can choose between a general discussion to specifically highlighting different student’s work. In addition, STEP enables to determine which characteristics will be taken into consideration when assessing whether the submission meets the requirements of the task, and these characteristics are displayed in a different color.
The teacher presented examples for student submissions with this characteristic (Figure 4), and asked the students what did these students do? What type of function do you get in this situation? Do they think it is easier? How would you find the equation? Furthermore, the teacher also addressed submissions in which the two point coincide which enables students to have non-constant functions. The teacher indicated that this choice of points enabled students to understand that an equation can be written without calculating the slope and plugging in the X and Y values of the points.
The table report presents a row for each student, and a column for each characteristic indicating which characteristics are evident and which are not for each student submission (Figure 5), in a presentation that is quite similar to a conventional spreadsheet. The goal of this report is to provide the teachers with elaborate information for each task for every student in the class. The table report was designed following requests from teachers to have a detailed description for each student separately.
The table report enables the teacher to catch in a glimpse which characteristics are prominent in their student’s submissions and which are not. It also enables teachers to see which students did not submit an answer at all and potentially approach them.
The histogram report presents a distribution of the characteristics across the different submissions. A single report shows for each characteristic a visual representation of a column which height corresponds with the number of submissions which have this characteristic (Figure 6). Note that characteristics that are not present in the student’s submissions do not appear in this report. The goal of this report is to provide a relative visual representation of all of the characteristics for a given task, providing insight into the relative distribution of characteristics among student submissions. This interactive report also enables to choose (by clicking) a certain characteristic and then display a filtered grid report below the histogram, enabling the teacher to further analyze student work directly.
The use case for this report is for a task on quadratic functions, identifying the extremum and calculating the distance between to extrema of two different quadratic functions. The task: “Given the functions f(x), g(x) from the family y=a(x-p)^2+k. Claim: there is only one option for the functions f(x) and g(x) such that the distance between the extremum points of each of the functions is 5 units. If you think this claim is true provide the algebraic expression for each of the functions. If not. Give 5 examples for different functions using the interactive diagram”. The teacher is this case noticed that most of the students gave correct answers, and also that most of the students gave a distance that was not a vertical distance, which she did not expect. Initially she presented the filtered grid report for the characteristic “vertical distance” and conducted a discussion around it, and then moved on to the filtered grid report of a distance that has a slope (Figure 6) 2.4.
The Venn diagram presents the interrelations between characteristics in the student’s submissions (Figure 7). The report shows up to three different characteristics simultaneously (due to the circles representation constraints of Venn diagrams), which are color coded to indicate which one represents which characteristic. The goal of the Venn diagrams is to show the distribution of more complex phenomena in the submissions of students. Phenomena that could not be captured by only one characteristic but rather in the relations between several characteristics. The diagram provides indication whether certain characteristics coincide, or perhaps do not exist together, etc. The presentation provides also the total submission’s number thus enabling to analyze which phenomenon is more prominent in student work. Similar to the histogram report, clicking on a part of the diagram displays a filtered grid report according to the selected region (e. g. intersection of 3 characteristics).
In this report we described dashboard components of an online formative assessment platform in mathematics (STEP). This platform is designed to facilitate blended learning by combining both automatic analysis of student answers for online interactive tasks, as well as accessible learning analytics in various levels of complexity available for teachers. The learning analytics are accessible through different interactive reports that could be used both in real-time teaching in the classroom and also in preparation and review of homework assignments towards a classroom discussion in the following lesson. The use cases presented show that teachers use the different reports while employing different strategies ranging from locating work with specific characteristics (e. g. choosing points with the same Y value), to meaningful complicated logical conjunction of characteristics (e. g. not many students both solved correctly and used a distance with a slope). These examples show that the design principles embedded in the platform indeed facilitate different usage schemes, and enable teachers to gradually gain expertise and address complex phenomena in their students’ work, phenomena made accessible by the different learning analytics components in the platform dashboard, and which exceeds well beyond the correctness of the answer.
This research was supported by the Israel Science Foundation (Grant 522/13) and the Trump Family Foundation (Grant 191). [10] Drijvers, P., Doorman, M., Boon, P., Reed, H., & Gravemeijer, K. (2010). The Teacher and the Tool: Instrumental Orchestrations in the Technology-Rich Mathematics Classroom. Educational Studies in mathematics, 75(2), 213–234 [11] Olsher, S., & Raya, K. A. Teacher's attention to characteristics of Parabola sketches: differences between use of manual and automated analysis. In Conference on Technology in Mathematics Teaching–ICTMT 14 (p. 213). [12] Bagdadi, J. (2019) Let Students Choose the Given: Design Principles for Technologybased Assessment. Paper presented at CIEAEM71, Braga, Portugal.