The carbonized Herculaneum scrolls represent a unique challenge for text recovery due to their fragile state and the visual similarity between ink and papyrus substrate. This study presents an iterative, human in-the-loop approach for ink detection in Scroll 5 (PHerc. 172) using a TimeSformer-based deep learning model. Building upon preprocessed X-ray Phase-Contrast Tomography (XPCT) data from the Vesuvius Challenge, we employ progressive model refinement and expert-verified labeling to improve the identification of ancient Greek letterforms. Our approach demonstrates that precise, trace-based annotation combined with repeated training cycles leads to a significant increase in legible character recognition from 169 to 368 letters across 15 scroll segments. The results underscore the value of interdisciplinary collaboration and iterative feedback in advancing the digital decipherment of ancient texts.
The eruption of Mount Vesuvius in 79 AD sealed away a unique literary treasure: the carbonized papyrus scrolls from the Villa of the Papyri in Herculaneum. Preserved under layers of volcanic material, these ancient manuscripts ofer a rare glimpse into the intellectual life of the Roman world. Yet for centuries, their charred and fragile state rendered many of them unreadable. Recent advancements in non-destructive imaging, particularly X-ray Phase-Contrast Tomography (XPCT), have created a new opportunity to digitally access their contents without physical unrolling.
To advance this opportunity, the Vesuvius Challenge [
In this study, our primary focus is on Scroll 5 (PHerc. 172), where we utilize the preprocessed outputs of the Vesuvius Challenge to train and refine an ink detection system based on the TimeSformer model. The proposed methodology involves a human-in-the-loop approach, wherein initial model predictions are systematically reviewed and refined by domain experts. These refinements then serve as a foundation for subsequent training iterations. This iterative feedback loop has been demonstrated to result in progressive improvements in model accuracy and output clarity.
The primary metric employed is the quantity of clearly identifiable Greek letters, as verified by a specialist, across successive model iterations. This approach maintains a balance between technical precision and historical authenticity, ensuring that detected ink traces correspond to plausible ancient letterforms. The results demonstrate that through careful annotation and iterative refinement, even faint textual signals embedded within the scrolls can be successfully recovered and rendered legible.
The remainder of this paper is structured as follows: Section 2 provides an overview of related work, including past Vesuvius Challenge contributions and the historical development of letter shape analysis and transformer-based models. Section 3 introduces the dataset and preprocessing pipeline, with a focus on XPCT scanning and the segmentation method used to flatten the scroll layers. Section 4 details our methodological approach, including labeling strategies, iterative training, and implementation details. In Section 5, we present our experimental results and evaluate model performance across iterations. Section 6 concludes the paper with a discussion of findings and outlines directions for future research, including semi-supervised techniques and improved preprocessing workflows.
The data of the Vesuvius Challange is an application of computer tomographic (CT) measurement and imaging in the context of the evaluation of man-made artifacts. The description and estimation of these artifacts is related to man-made scientific reading and assessment schemes. In its entirety it covers topics of material distinction and description, physical, chemical description of material change under conditions of temperature and time, material representation through measurement and its computational estimation, also the description of change of human labor habits related to production and usage of papyrus, the development of writing, reading and storing information. Out of that we choose three aspects, that we will cover in the section of related work: (1) Ink on charred papyrus, (2) script usage and (3) the computation on the data (model and modelling) in relation to ink residues.
In 2015 the founders of the challenge published their research on a charred scroll (parchment) from
En-Gedi [
For the purpose of this paper it is essential to know potential form and development of letters used in
the Herculaneum scrolls. They span from the fourth century BCE to 79 CE with Latin and Greek letters.
A detailed treatment is given by Cavallo, "Libri scritture scribi a Ercolano", Naples 1983 [
The original analysis pipeline (mesh model of scroll as a mass spring system driven by the gradient
criterion of second order symmetric tensor and salience measure of the intensity volume (which denotes
the location of animal skin surface, mapping voxels as textures to mesh model, flatten segments, merge
segments by hand)) is now dominated by AI-driven methods. The reason for that is threefold: first the
segmentation of sheets of material is dificult and local geometrical and statistical expressions of surface
orientation are not always possible depending on the preservation state (some papyrus sheets are
simply crushed or baked together); second the poor ink signal in the XPCT dataset needs to be boosted
and third it’s the AI time. We will focus on the work that was carried out to estimate ink residues in
lfattened parts of a papyrus scroll (segments) represented by the volume around of the papyrus sheet.
To solve the problem of estimating regions of an unknown material density characteristics in relation
to surrounding regions and classify them as written regions. Youssef Nader utilizes the training of
the a TimeSformer model and its application for inference. The approach is quite diferent from the
original intention of the TimeSformer [
The analysis of ancient carbonized scrolls, particularly those recovered from the Villa of the Papyri in
Herculaneum, requires a digital pipeline to prepare the data for ink detection and text reconstruction.
We leverage the preprocessed volumetric data provided by the Vesuvius Challenge [
The scrolls examined by the Vesuvius Challenge community have undergone a non-destructive scanning
procedure using XPCT, executed at the Diamond Light Source synchrotron facility. Unlike conventional
absorption-based X-ray imaging, XPCT is sensitive to minute changes in the refractive index, making it
ideal for detecting the subtle structural diferences within the carbonized papyrus and ink [
These raw scans, however, do not directly yield readable text. Due to the compressed and irregular
geometry of the scrolls – folded, coiled, and damaged over centuries – text is often embedded within
highly distorted 3D spaces, distributed across dozens of sub-millimeter depth layers. Thus, a critical
step involves virtual unwrapping, in which the topology of the scroll is modeled computationally and
then digitally flattened into a series of depth images suitable for further analysis. This unwrapping
pipeline builds on the prior work by Stabile et al. [
The scroll is ultimately converted into a stack of 65 axial slices – each representing a distinct depth
layer within the material [
One of the main components of the Vesuvius Challenge infrastructure is the Volume Cartographer pipeline, introduced by Parsons et al. [18], which automates the segmentation and mapping of scroll geometries onto 2D image stacks. In contrast to earlier manual or semi-automated methods, Volume Cartographer provides a reproducible, data-driven solution to the segmentation problem, producing aligned sheet maps from raw volumetric data.
At the core of Volume Cartographer is a hierarchical sheet detection algorithm that identifies and tracks distinct papyrus layers within the scroll volume. The system begins by isolating regions of coherent planar structure using a curvature-based filter across the volumetric scan. These candidate layers are then refined through a multi-stage fitting process that leverages surface normals and local geometry to produce high-fidelity surface meshes. These meshes are subsequently used to interpolate lfattened representations of each sheet via a deformation-minimizing transformation. The output is a stack of 2D orthogonal slices that correspond to the original layers but have been geometrically corrected to remove bending and torsion [18].
In our project, we build upon the pre-segmented and virtually unwrapped output generated by Volume
Cartographer. This includes the depth-normalized 65-slice representation of Scroll 5, from which we
extract regions of interest for ink detection. Importantly, the segmentation process already incorporates
denoising and artifact rejection, meaning that the layers we process have reduced geometric noise and
exhibit a relatively consistent topology [
While our contribution does not involve segmentation itself, the quality and accuracy of the input data are directly contingent on the success of Volume Cartographer. Misalignments, topological artifacts, or segmentation errors at this stage can propagate downstream, leading to false positives in ink detection, i.e. the misidentification of substrate textures as letterforms. Therefore, the robustness and generality of Volume Cartographer were essential prerequisites for the iterative modeling pipeline described in subsequent sections.
Moreover, the segmentation approach ofers an implicit form of contextual regularization. By aligning
layer surfaces according to the physical curvature of the scroll, Volume Cartographer ensures that
adjacent image slices correspond to adjacent material layers, preserving local spatial continuity. This
is particularly valuable for models like TimeSformer [
By building atop the outputs of XPCT scanning and the Volume Cartographer system, we are able to engage directly with virtual representations of ancient manuscripts, thus sidestepping the most fragile and destructive aspects of physical papyrology. These processed scroll images provide a reliable substrate upon which our ink inference techniques are deployed.
Our methodology aims to reliably detect ink traces from the segmented, flattened volume slices produced during preprocessing. Given the extreme visual similarity between carbon-based ink and the papyrus substrate, we adopt a human-in-the-loop strategy that combines expert-informed manual labeling, iterative model refinement and implementation techniques designed to maximize visibility and interpretability. This section illustrates the system’s implementation details, our image enhancement and labeling strategies and the iterative model training workflow.
All models were implemented using the PyTorch Lightning framework to enable modular and reproducible training. We used Weights & Biases [19] to track all experiments, logging metrics, hyperparameters, and qualitative outputs.
Training was conducted on a remote Linux server equipped with an NVIDIA Tesla V100 GPU (32GB VRAM). We adopted the 2023 grand prize-winning method from the Vesuvius Challenge, which utilizes a TimeSformer model architecture [17] using a sliding window approach. Each training input consisted of a 64 × 64 × 26 volumetric window, extracted with a stride of 32. The very small scale of this window size is visualized in Figure 2 as a yellow 64 by 64 pixel square, which takes up only a fraction of the space of the full letter. This configuration is chosen to preserve local structural context of the papyrus material while preventing the model from being able to learn to recognize entire characters, which would likely lead to overfitting and poor generalization.
To evaluate the model performance during training, the segment 20241028121111 (see Figure 3) was always held out as a validation set and never used for training. This segment overlapped substantially with a nearby labeled segment (20241028121112), minimizing overall data loss while still providing the trainer class with metrics for evaluation and learning rate scheduling.
To enhance the interpretability of scroll segments generated during virtual unwrapping, we began with predictions from a pretrained TimeSformer model [17]. These initial predictions served as a starting point for expert-guided manual annotation.
To create a reliable training set, we considered two alternative labeling strategies: • Trace-based labeling: This conservative strategy prioritized precision. We labeled only clearly visible, well-defined ink traces that matched expected Greek letterforms. Faint traces were included only if they structurally aligned with expected shapes and showed suficient contrast against the background. All labels were validated by a specialist of ancient Greek, referencing authoritative sources including [20] and comparative paleographic studies [21]. • Expansive labeling: This strategy sought to increase coverage by extrapolating partial traces into full letterforms when there was high confidence in the inferred structure. Ambiguous features – such as vertical strokes potentially being part of multiple characters like (iota), Π (pi), Ψ (psi), (tau), (kappa) – were excluded. This approach increased the quantity of labeled data, as gaps in incomplete letters were filled, as seen in Figure 4. Conversely, it also introduced the possibility of adding false positive labels, which are correct from a reader’s perspective, but might teach the model to hallucinate ink, where there is none, ultimately preventing our goal of visual clarity.
Testing the second approach for one iteration, the results were visually much noisier, which led to our choice of labeling using the Trace-based approach for the remainder of the process.
We employed an iterative model training approach to improve ink trace detection over time. Figure 5 illustrates the overall flow. The central idea was to refine the labels after each iteration based on model inference and expert analysis, thereby enhancing both label quality and model accuracy. Each iteration followed this general process: 1. Correct false positives by removing previously labeled letters that were no longer supported by model inference in the current iteration. 2. Identify and annotate new traces that plausibly match letterforms found in the [20], particularly those attributed to works by Philodemus (1st century B.C). 3. Count the number of readable letters in each segment to evaluate progress. 4. Retrain the TimeSformer [17] model from scratch on the revised label set, then run inference to initialize the next round.
This process allowed us to iteratively improve both the precision of the model and the clarity of the training data. The combination of structured annotation, expert validation, and feedback-guided training significantly reduced noise and false positives.
The iterative process concluded after four rounds, when the final iteration no longer showed noticeable improvements in visual clarity or letter count. This saturation point was considered the efective limit of our current approach.
The data used in this study originates from Scroll 5 of the Vesuvius Challenge dataset [18]. The scroll has been CT-scanned and virtually unwrapped using the state-of-the-art pipeline provided by the competition organizers (explained in Section 3.1).
Although the scroll contains 23 available segments, we worked with a subset of 15, which were used consistently throughout all iterations of the project. These 15 segments include: 20241028121111, 20241028121112, 20241030083650, 20241030152031, 20241105112301, 20241113070770, 20241113080880, 20241113090990, 20241025062040, 20241025145341, 20241025150211, 20241102160330, 20241108111522, 20241108120731, and 202411081207312. Of these, the initial round of manual labeling and model training used the first 8 segments (see Figure 5 for the workflow diagram).
Each segment contains 65 volumetric layers, corresponding to depth-wise slices through a single sheet of papyrus. We consistently used layers 17 through 42, which cover the central portion of each sheet. This range provided the highest ink visibility while avoiding distortion or noise from overlapping neighboring layers. Figure 6 illustrates how one sheet of papyrus appears as multiple layers in the CT data.
As a starting point, we applied the 2023 Vesuvius Challenge grand prize model [17], trained on Scroll 1, to all 15 segments of Scroll 5. The resulting predictions served as input for the first round of manual labeling and model training in our iterative pipeline.
We evaluated the progression of identifiable letters across three model iterations, using a benchmark of 169 letters manually labeled across 8 scroll segments. These letters were verified by an expert in ancient Greek and served as the baseline for subsequent inference and retraining cycles.
Only ink traces that could be confidently identified as full letters of the ancient Greek alphabet were included in the count. Ambiguous fragments – such as vertical strokes that could plausibly belong to multiple characters, like (iota), Π (pi), Ψ (psi), (tau), (kappa) – were deliberately excluded to maintain a high standard of certainty in the evaluation.
In the first iteration, inference was performed on all 15 segments, resulting in 286 identifiable letters – an increase of 117 over the initial benchmark. This improvement was due to the model’s ability to generalize beyond the initial labeled set and detect additional partial or faint characters. Iteration 2 yielded a more modest increase to 321 letters, while the final iteration raised the count to 368, marking the highest number of identified characters achieved during the project. Overall, this represents more than a twofold increase compared to the initial benchmark.
While gains diminished across iterations, Iteration 3 still contributed 47 newly recognized letters, indicating that the model continued to extract signal from previously ambiguous areas. However, it also became evident that the majority of readable content had been recovered: noise was largely suppressed, and regions previously confused for ink had been ruled out. This suggests that further iterations would likely not yield meaningful new information from the existing input data.
Performance varied across segments. Some, such as 20241025062040 and 20241108111522, initially had no identifiable letters but saw substantial improvement after refinement (gaining 42 and 22 new letters, respectively). In contrast, other segments with clearer early signals, like 20241030152031, showed slower but steady increases over time. These diferences reflect the physical variability in preservation (and the state of the estimation of the target volume of Volume Cartographer) and legibility across the scroll.
While the overall trend across iterations was positive, we did observe a few instances of regression, where the number of identifiable letters in certain segments temporarily decreased. These cases typically arose when regions initially labeled as letters were later judged, by the model or during manual verification, to be more likely noise. As the model improved, it became more selective, sometimes discarding partial or ambiguous traces that were previously accepted as valid characters. In other cases, slight changes in predicted shapes rendered a previously plausible letter unrecognizable. Such corrections, although reflected as numerical decreases, often represented a gain in labeling precision.
Additionally, across iterations, we observed that some characters underwent shape reinterpretation, which contributed to fluctuations in the number of identifiable letters. As shown in Figure 7, an example of such a transformation involved ink traces that were initially classified as Π (Pi) and X (Chi), which in this specific case were later reassigned as T (Tau), O (Omicron), and Σ (Sigma), the latter of which was commonly written in a form resembling the modern Latin letter ’C’ during the relevant historical period. While such transformations afected local letter counts within specific segments, they also highlight the model’s ability to refine its interpretation of ambiguous traces over time.
Figure 8 and Figure 9 illustrate the evolution of segments 20241108111522 and 20241113080880 respectively, which progressed from zero recognized letters to 22 by the third iteration and from 4 to 14. A full breakdown of results by segment and iteration is provided in Table 1.
While the number of identifiable letters is not a standardized metric, it ofers a practical proxy for model performance in this interdisciplinary context. Unlike traditional benchmarks, the goal here is not classification accuracy but the recovery of legible ancient text. These results highlight the efectiveness of iterative refinement for enhancing model reliability in low-signal, high-noise settings, such as carbonized papyrus scrolls.
This study explored an iterative labeling approach for ink detection in the carbonized Herculaneum Scroll 5. By leveraging a pretrained TimeSformer model, originally trained on Scroll 1, we systematically refined our dataset through multiple cycles of manual annotation and model retraining. This process resulted in a significant increase in the number of identifiable letters, demonstrating the efectiveness of progressive label refinement in challenging historical document analysis tasks.
Our findings highlight the importance of high-quality manual annotation in improving model performance, particularly in cases where ink traces are faint and dificult to distinguish from the substrate. One key finding was that a trace-based labeling strategy, which focused strictly on visible ink traces without reconstructing missing portions, produced clearer results than the approach that attempted to extrapolate incomplete letters. While the latter method increased the amount of labeled data, it introduced additional uncertainty and noise, reducing the clarity of detected text. Additionally, our results reinforce the value of expert verification in ensuring historical and linguistic accuracy, preventing misinterpretation of detected letterforms.
Challenges remain, particularly in distinguishing ink from the surrounding papyrus, but recent eforts have expanded the availability of labeled data. With our iterative approach, we have contributed to an improved dataset, which can now serve as a foundation for future model training. Building on this, future work should explore semi-supervised learning and active learning techniques to further optimize the annotation process while maintaining label accuracy. Additionally, enhancing preprocessing methods, such as contrast optimization and structure-preserving augmentation, may improve ink visibility and model robustness.
Beyond the technical aspects, collaboration between computer scientists, historians, and papyrologists will be essential for making meaningful progress in virtual unwrapping and text reconstruction. As deep learning continues to advance, these interdisciplinary eforts will play a critical role in recovering and interpreting texts from the Herculaneum scrolls and other fragile manuscripts, bringing lost historical knowledge back to light.
This work uses data provided by the Vesuvius Challenge. We gratefully acknowledge Vesuvius Challenge team and the Scroll Prize organizers for providing access to the scanned Herculaneum scroll datasets. These datasets, including high-resolution XPCT scans and segmentation outputs, were made available through https://scrollprize.org and are central to the analyses presented in this paper. All the data used in the preparation of this paper were obtained from the EduceLab-Scrolls dataset [18].
We thank the entire Vesuvius Challenge community for their contributions to open research and their continued eforts in making ancient history accessible through modern technology.
All relevant data used in this paper that exceeds the scope of the Vesuvius Challenge repository is openly accessible in our public GitHub repository https://github.com/FrankDeinzer/ki2025_data. Researchers are invited to reproduce or extend our pipeline.
During the preparation of this work, the authors used ChatGPT, Grammarly in order to: Grammar and spelling check, Paraphrase and reword. After using this services, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.