-Since the advent of tablet computers with styluses, there is research on how to integrate such devices into teaching. Previous work focuses on the students' perspective, (visionary) usage in the classroom, or efficient exam marking. In this article, we consider the perspective of tutors (or student teaching assistants) that are supposed to give feedback on students' homework. We investigate the usability of a particular tablet configuration used by five different tutors in two years, and the (often neglected) aspect of overall maintenance effort.
Our introductory course on software engineering employs, like many other similar courses, exercises for homework and a discussion thereof in so-called tutorial sessions. A part of this activity is to mark the students’ submissions and to give in-place feedback on the submissions that enables students to improve their skills in problem solving and scientific writing.
Students’ submissions on exercises in our software engineering course include a good amount of diagrams and mathematical notation. We have the feeling that good feedback on such submissions needs the expressive freedom of handwriting, which is perfectly available with a classical, paperbased submissions scheme (students submit paper and get the paper back with annotations). For many years now, students can (and are asked to) submit electronic documents using the e-learning platform offered at our university. The digital workflow was broken when it came to the correction work by tutors (or: student teaching assistants). By the means available at our department, we could only offer to use ordinary PCs with keyboard and mouse for the annotations (which was reported to not be perfectly appropriate for diagrams and mathematics), or print out the submissions (which was reported to be annoying due to the necessary handling of the papers).
In this article, we argue for the opinion that the recent generation of high-resolution tablets with styluses gives a good opportunity to get the best of both approaches. We report preliminary results on the following research questions to gain a better understanding of the costs, feasibility, and usability of stylus-enabled tablets in homework marking:
software support stylus-based homework marking?
homework marking (pre-semester and in semester)?
The research method that we use is an expert questionnaire including questions for percentage estimates, multiple choice fields, and open questions. We consider the tutors who answered our questionnaire to be experts in the following sense: All of our tutors are familiar with the usability of a broad range of software and devices (from PCs to touch-enabled, mobile devices). The majority of our tutors held similar positions beforehand and are thus familiar with different workflows, and they are, as senior computer science students, familiar with the students’ perspective, i.e., with the form of useful feedback.
The article is structured as follows. Section II discusses related work. In Section III, we characterise the role of a tutor in our correction process. Section IV describes the hard- and software setup that our tutors used and gives rationales for our choices. Section V discusses the responses we received on our expert questionnaire and Section VI concludes.
In summary, most related works address research questions similar to (RQ 1) yet mostly in the case of exam marking (which is different from homework feedback). Some works address the student’s perspective, yet the administration effort for providing each tutor with a stylus-enabled tablet (our (RQ 2)), and usability and usage from the tutors’ perspective (our (RQ 3) and (RQ 4)) have received less attention.
Popyack et al. [
Closer related to our work are [
III. BACKGROUND: TUTORING ROLES AND PROCESSES In the case of our course, the tutor role participates in the following activities wrt. homework assessment: Download the PDF submissions (possibly handwritten and scanned by the students) from the ILIAS e-learning platform, add the submissions to a revision control system (accessible by (staff) teaching assistant and tutor), assess the submissions, provide feedback and preliminary markings, discuss the markings with the teaching assistant, and finalise the marking and upload a PDF to ILIAS (or hand out a marked printout to the students).
This workflow has been used for many years for our course (and is, we believe, used at many other universities), yet we had to leave the means to be used for annotations and markings to the discretion of the tutors. Some tutors preferred the classical approach of working on a printout, some used PCs with keyboard and mouse, and a few had private tablets with stylus that they deliberately used on their job.
The course we report on here is an undergraduate
introduction to software engineering [
With this regard, we found the existing process unsatisfactory: Using paper is tedious, preparing the annotations that we need with keyboard and mouse is cumbersome, and only hiring tutors with a private tablet was not an option. To obtain a more satisfactory, completely digital process, we started to provide each tutor with a tablet with stylus for the whole semester, so to combine the advantages of paper-based correction with the benefits of a digital workflow.
Regarding hardware, we are limited to the market offerings.
For the software setup, previous work seems to have a slight
preference for newly built, dedicated tools [
When we needed to choose devices, there were three broad options: Tablets designed for the Android operating system, iPads coupled with iOS, and PC-like hardware in tablet form. Android devices with stylus are available from different vendors, predominantly with 10′′ screen and with a weight of 500-600g (without keyboard). iOS devices are available with 10′′ screen (and a weight of about 500g without keyboard) or with a 13′′ screen (and a weight of about 700g without keyboard). We early on decided against Android or iOS devices for privacy reasons: To be useful, these systems almost need a registration with the respective OS provider and our time resources didn’t allow to investigate further how exactly our student employees would be tracked and whether the tracking is allowed by labour laws. Another aspect that needs to be cared for are the privacy interests of the students, whose individual-related data is necessarily processed by us.
The market offering on PC-like tablets started at slightly higher prices at the time of our market research (around 1,000 e), and easily reached regions from 1,500 e to over 2,000 e which was outside our budget. The devices that we report on here, have been acquired in two phases. In 2016, we acquired one tablet (a 2016 model) that we refer as ‘Device L’ in the following, and in 2019, three ‘Device M’ tablets (a 2017 model) from a different brand. Both devices have a 12′′ high-resolution, touch-enabled screen, support a stylus and wireless networking, and come with a detachable keyboard. We have chosen similar, rather low-end configurations wrt. memory, CPU, and Solid State Disk (cf. Table I). In addition, we acquired one set consisting of a wireless keyboard and
TABLE I
DEVICE SPECIFICATIONS.
L mouse to experiment with their usability (around e 30), one set of adapters (HDMI and VGA (for projectors), and LAN) to be borrowed as needed, and one sleeve bag (made of thick felt) for each device (ca. e 35).
Both devices support different operating systems. The data
we report on here refers to two setups: One data point from
2018 is from a tutor who used some Windows 10 edition,
and four data points from 2019 are from tutors who used a
Linux-based setup. In the following, we describe the
Linuxbased setup. As base system, we use Xubuntu, i.e. the Ubuntu
distribution with the XFCE desktop environment, in an LTS
(long term support) release. Our hope is that the installation
media and configuration files can be re-used over many years.
Device L is fully supported by the standard distribution,
Device M uses some non-standard hardware that needs dedicated
kernels and drivers (that are freely available). We will continue
to offer the tablets with this setup for the following reasons.
One of our concerns is administration effort (which is often
neglected when acquiring new technology (with [
For the re-installation, we follow a 2-page, step-by-step instruction sheet that we did prepare with the initial installation.1 The initial installation and the preparation of the instruction sheet took about two to three working days net (plus about one day in total to choose and order the devices). We mostly use the setup proposed by the standard installer (with encrypted disk), add some pre-downloaded software from a prepared memory card, copy over a few prepared configuration files, and change a few aspects of the setup, e.g., for the high-resolution screens. For our course, we need the following software: Xournal (the PDF annotation tool); Make; Subversion (file versioning); Vim (text editor); Xpdf, Zathura (PDF viewers); Xscreensaver; Florence (onscreen keyboard); Arandr (screen setup); Acpi (battery information); TeXlive (LaTeX); a Java Runtime Environment; and Gimp (raster graphics editor). Students are allowed to install further software if needed. We copy over a configuration file for Xournal, and for the window toolkits to, e.g., let scrollbars appear wider and hence easier to hit with the pen. A complete re-installation of the tablets then takes about 15 minutes (some of the time unattended).
1All media (for one bootable USB flash drive and one microSD memory card) and the instruction sheet could be made publicly available on popular demand, since our setup does not rely on any non-free software. V. HOW DID THE TUTORS USE THE STYLUS-TABLETS?
AND DID THEY LIKE THEM?
We have selected the devices so that our tutors can experiment with different setups for their workplace: The keyboards are detachable, the screen can be rotated to support landscape and portrait views, and one tutor was provided with external, wireless keyboard and mouse.
After finishing the summer semester of 2019 (and after the jobs had been completed), we conducted a survey among the four student teaching assistants that used the tablets with the Linux setup and one who used Device L with the Windows setup in the season of 2018. The data points interestingly do not differ much for the setups: With the setup of 2018, only the stylus has been reported to be lagging in a slightly annoying way for almost all annotation software and, not being well familiar with this system, we had some difficulties to get all configuration settings to our needs.
The survey questionnaire asks for feedback on four areas: The way the device as such is used and how the workplace is set up, the workflow for the marking and which kinds of annotations are used, satisfaction with the device and setup, and free text questions on the overall experience. In the following, we report the collected data from these areas.
The majority of tutors used the tablet on a large desk (so there would be space for an external keyboard and mouse), some also squeezed it onto a smaller desk or use the device in another setup (‘lounge style’), cf. Figure 1(a). In the questionnaire we asked for the percentage of the net correction time, i.e., excluding the time when the submissions were downor uploaded, or when taking notes in tutoring meetings. The percentages are estimates and recalled from memory, we are not aware of measurements being taken.
The preferred orientation was landscape, where some tutors used a mixture of landscape and portrait, hence to suit all tutors’ needs, devices should support screen rotation (cf. Figure 1(b)). The free text responses indicate that the preferred orientation may depend on the workplace setup: Some tutors kept the correction instructions on the tablet screen and some tutors reported to have used a separate screen.
There seems not to be a strong correlation between the screen orientation and keeping the keyboard attached or not: The tutor who gave the top-most response in Figure 1(b) kept the keyboard attached 90 % of the time, and the second response in Figure 1(b) reported only 30 %. Of the three tutors who preferred landscape, two kept the keyboard attached 0 % or the time and one of them 90 % of the time. We would have expected the attached keyboard to come in one’s way in the landscape setup; again, a stylus-enabled laptop would not suit all tutors’ ergonomic needs.
There was no clear preference on whether to put the tablet flat on the table or to use the built-in kick-stand, which allows different angles between a steep, laptop screen-like setup and a flat-angled, desk-like setup (cf. Figure 1(c)). Almost all tutors, though, had a personal strong preference. Free text feedback 70 % 30 % 40 % 60 % 100 % 100 % 100 % (b) portrait, landscape 100 % 90 % 20 % 100 % 50 % (c) indicates that the flat-angled setup was much appreciated, a setup that is not easily supported with stylus-enabled laptops.
For each exercise sheet, each tutor received between 10 and 15 submissions (about 12 in average over the semester). Each exercise sheet includes multiple tasks with subtasks. The majority of our tutors prefers the strategy to mark by task (cf. Figure 2(a)), i.e., first consider one task in all submissions, then the next task, etc. One tutor went team-by-team, i.e., first consider all tasks of one submission, then the next submission, etc., and one tutor employed a mixture depending on the task (whether the task is more technical or more open).
Regarding screen orientation, the tutors who preferred taskby-task used 100 % portrait, and 70 %, and 60 % landscape, respectively, the tutor who preferred team-by-team used 100 % landscape, as well as the tutor with a mixed strategy. We would have expected a stronger correlation between task-by-task and landscape (because a smaller area of the submission needs to be visible), and team-by-team and portrait (because it could be useful to have a larger area of the submission in sight).
The questionnaire offered seven choices for the kinds of annotations that tutors used at least once: (i) short, handwritten ‘okay’/‘not okay’ annotations (like check marks), (ii) highlighting (underlines, circles, etc.), (iii) lines, arrows, circles, etc. to refer to different parts of the considered document, (iv) short, handwritten comments, (v) long, handwritten comments, (vi) typed comments, and (vii) others. Figure 2(a)) gives the total numbers of responses for the seven options. Each tutor used at least three of these, the median is at four (two responses), the maximum at six (one response). Hence, each of our tutors used the stylus, some of the tutors seem to have almost always used the stylus due to low number of responses for case (vi), the typed comments. Responses to questions on mouse, touchpad, and touchscreen usage indicate that these input devices have almost never been used for annotations.
The previous data only indicates that the tutors have used non-textual annotations. Recall, that one of our hypothesis for the whole project was that software engineering exercises are often graphical or mathematical in nature, and hence tutors need a way to include graphics and mathematical symbols in their annotations. To investigate this hypothesis, the questionnaire asked for the percentage of annotations in graphical or mathematical form. Figure 2(b) shows a clear need for graphical and mathematical annotations for the kind of tasks in our exercise sheets.
An important aspect when using styluses is the feeling of the handwriting. We asked for two characteristics: Whether the cursor lagged behind the tip of the stylus and whether the device faithfully captures the strokes. Four out of five responses considered the lag good, one response was ‘okay, but not disturbing’. The strokes were considered good by three and ‘okay’ by two responses, the latter for two different devices.
A free text question on the provided setup and software suggested a very broad range of computer system aspects to consider (like speed, display resolution and brightness, etc.) The tutors seem to very satisfied with the setup, since the remarks were all related to the form of the styluses (shape, ergonomics of the button), and the stylus holder on Device L. Regarding the screen size, our respondents have the feeling that 12′′ displays have a good size (they would not want to trade smaller size and lesser weight for a smaller screen), if at all, they would recommend a 14′′ device, but not larger.
The felt cover was considered to be a good addition to the overall setup, and the responses indicate that the covers are considered to provide a good protection for transport.
The questionnaire also asked for an opinion on whether the tutors would prefer to obtain their tablet in a pre-installed fashion (including all necessary software and useful configurations), or whether they would prefer to install an own working environment. In contrast to common prejudices (that computer scientists are reluctant against devices set up by other people), all tutors prefer the pre-installation approach. Free text comments name ‘economy of thought’, ‘laziness’, and state that they prefer to dedicate their working hours to tutoring. Some responses raised the strong point that a similar setup for all tutors allows everybody to share experience and help each other (which we did observe during the semester, in particular via a dedicated forum on the ILIAS platform).
What did our tutors dislike about the tablets? We have space to quote all answers: ‘depends on battery’ (two times; both times not causing inconveniences), ‘nothing’ (three times).
What did our tutors like about using tablets? The feeling of handwriting was named twice (once in comparison to personal experience with stylus pads (without display)). Three respondents liked the aspect that the submissions were available at any time and place together with the tablet (in contrast to a paper-based marking). Two respondents were able to compare their experience to a digital workflow where the annotations are prepared on an ordinary PC with keyboard and mouse, and found the tablet with stylus much more convenient. A final aspect, that was mentioned three times, is that the annotations can be easily corrected ‘without the submission looking horrible afterwards’ (which would also be possible in a digital workflow without styluses).
If our tutors were to take a student teaching assistant’s job again, would they prefer to have a tablet with stylus again? Five (of five) times ‘yes’, two times with the wish to try another device, not out of dissatisfaction but out of curiosity.
On (RQ 1), our experience indicates that today’s stylus and tablet technology is ready to support a fully digital workflow for homework assessment in our course. There do exist device setups that can be maintained with a, for us, acceptable effort (cf. (RQ 2)). On (RQ 4), the data reported here shows that our tutors need means to add graphical or mathematical annotations and that a variety of kinds of annotations is used (cf. Figure 2). Hence providing a stylus does enable expressive annotations. Regarding (RQ 3), the responses indicate that our tutors are overall contented with the device characteristics. Responses on the handling of devices indicate that our tutors use different workplace setups that would not all be equivalently well supported by a laptop with stylus. Using tablets provides the additional value, that these devices can support a broader variety of workplace setups and thereby increase overall (if only subjective) usability for (student) workers.
Previous research [
Future work includes a further improvement of the workflow
and the workplace setup. In our opinion, student teaching
assistants need a professional working environment, just as
professional video editors or digital graphics artists may use
a different workplace setup and may have different working
habits (e.g., the extensive use of keyboard shortcuts) compared
to the casual user. The setup we presented in this article has
shown not to be unusable, yet the devil is in the details: We
see that some user interface elements could be improved for
the use with styluss on high-resolution displays, that some
keyboard shortcuts are at inconvenient places, and we see a
strong potential in complementing the annotation software that
we use with ‘stamps’ (cf. [