Style change detection aims to identify points within a document where authorship shifts occur, which is crucial for tasks like plagiarism detection, authorship verification, and writing support. This submission addresses the intrinsic style change detection task from PAN, which involves identifying sentence-level style changes in multi-author documents under varying topical constraints. We employ a RoBERTa-based model that captures subtle stylistic diferences between consecutive sentences. Our approach achieves F1 scores of 0.823, 0.766, and 0.667 on the easy, medium, and hard datasets, respectively, demonstrating robust performance across increasing levels of dificulty.
The objective of the style change detection task is to locate the places in a multi-author document
where authorship shifts. [
Over the years, the PAN shared task on multi-author writing style analysis has evolved significantly.
Earlier editions focused on identifying whether a document was authored by one or more individuals
(2018), estimating the number of contributing authors (2019) [
Previous work on Multi Author Writing Style analysis spans a wide range of methodologies. Multiple
eforts adopt a binary classification framework. The document is divided into text segments that are
compared to determine whether they are written by the same author or co-written by two diferent
authors. [
Zlatkova et al. explore ensemble learning through the concept of stacking LightGBM classifiers
trained over TF-IDF encoding. They also explore the use of Multi Layered Perceptrons for the same. [
Weerasinghe et al. explore the task on small and large datasets, with the logistic regression for the
former and neural networks for the latter. The authors use metrics such as ROC, AUC and F1-scores to
evaluate their models used in TIRA. [
Deibel et al. make use of MLP and Bidirectional LSTM architectures. They use the models in an
ensemble setup to capture both sequential and non-linear correlations. [
A notable approach is the one used by Shams Alshamasi et al. which diverges from the usual binary
classification approach to employ a clustering-based approach. The aim is to create clusters of segments
sharing the same author. They use algorithms such as K-Means and DBSCAN. [
A famous trend in this task is to use transformer architectures such as BERT and RoBERTa for binary
classification. [
To support the development and testing of style change detection models, three datasets corresponding to the three dificulty levels— easy, medium, and hard—are made available. All datasets contain annotated ground truth reflecting sentence-level style changes, with the exception of the last test segment.
Each data set is divided into three segments: the training set (70% of the data) is accompanied by ground truth and is used to train and develop models; the validation set (15%) also includes ground truth and serves to tune and evaluate model performance; the test set (15%) contains no ground truth and is reserved for the final evaluation of submitted systems. This structured split ensures fair benchmarking across all dificulty levels while supporting robust model development and optimization.
This section describes the methodology for implementation of our approach. We approached the problem as a binary classification task, predicting whether a particular sentence pair shares the same authorship or not. Our goal was to leverage the capabilities of pre-trained language models to achieve stable precision and recall results on both seen and unseen data. To achieve this goal it was necessary to prepare training data and conduct fine tuning in a manner to avoid model overfitting. The approach can be explained in 2 phases: 1) Data Preparation and 2) Model fine tuning.
To prepare the dataset for fine-tuning, we created sentence pairs from the input documents. Using the provided solution vector, we identified sentence pairs written by the same author (negative class) and by multiple authors (positive class). However, the resulting dataset was overwhelmingly unbalanced, leading to a bias toward same-authored pairs. To address this, we divided our documents into training and validation sets. From each training document, we sourced an equal number of positive and negative sentence pairs to ensure the training data was balanced. For the validation set, we included all possible sentence pairs to ensure the model is evaluated on data that closely resembles realistic inputs.
To prepare sentence pairs for our model we performed tokenization using pretrained tokenizers for the model architecture(RoBERTa in our case). For our truncation strategy we employed the maximum token length to be 256 tokens with the assumption that individual sentences would rarely cross the length restrictions. This was a practical adaptation that allowed us to balance between capturing suficient contextual information and maintaining computational eficiency.
The pretrained RoBERTa-base [
To avoid overfitting and improve generalisation we employed weight decay regularization [
ℒtotal = ℒCE + ∑︁ 2 where ℒCE is the standard cross-entropy loss, is the weight decay coeficient, and trainable weights of the model.
For accelerated training and eficient hardware usage we utilized fp16 for our training that allowed us to train weights on 16 bit floating point arithmetic instead of 32 bit standard precision. We relied on dynamic loss scaling to maintain training stability despite the lower precision. are the
The model utilizes a standard classification head to generate our final predictions. It consists of a Dropout Layer attached to the final hidden layers to counter overfitting, a linear layer and an activation layer where we utilized softmax for inference.
This architecture enables the model to learn the subtle stylistic diferences between sentence pairs and make accurate binary predictions on authorship change.
This section outlines the experimental setup used for training and evaluation, followed by the results
obtained on the oficial PAN 2025 datasets [
Balanced sentence pairs (equal positive and negative) were used for training, while all sentence pairs were used in validation to mimic real-world data distributions. The model was checkpointed after each epoch, and the best-performing checkpoint was used for final inference.
We evaluated the model’s ability to detect style changes across all three PAN-provided dificulty levels: easy, medium, and hard. The results are presented in Table 4. Evaluation metrics include precision, recall, and F1-score.
The model performs strongly on the easy and medium datasets, where topic variation provides subtle clues for authorship change. In the hard setting—where topical consistency is enforced—the model still maintains reasonable performance, demonstrating its ability to identify fine-grained stylistic diferences without relying on content-based signals.
In this paper, we focused on the PAN 2025 task of detecting style change at the sentence level in multiauthor writings. Our solution frames the task as a binary classification problem, using the RoBERTa-base model fine-tuned on balanced sentence pairs. We proposed data balancing, regularization, and precisionaware training strategies to enhance the model’s robustness and generalizability.
Our model scored F1 of 0.823, 0.766, and 0.667 on the easy, medium, and hard datasets respectively, improving upon naive baselines and providing competitive performance even under stringent topical limitations. These outcomes point to the model’s capacity for sensitive stylistic variations without relying substantially upon topic changes. Future directions could leverage more advanced architectures or contrastive learning strategies towards improved performance, particularly in topic-consistent settings.
We thank the PAN organizers for providing the datasets and evaluation framework. We also acknowledge the computational resources provided by Pune Institute of Computer Technology.
The author(s) have not employed any Generative AI tools. A. Panchenko, M. Potthast, A. Shelmanov, E. Stamatatos, B. Stein, Y. Wang, M. Wiegmann, and E. Zangerle, Overview of PAN 2025: Voight-Kampf Generative AI Detection, Multilingual Text Detoxification, Multi-Author Writing Style Analysis, and Generative Plagiarism Detection , CLEF 2025, Lecture Notes in Computer Science, Springer, Madrid, Spain, 2025. [17] E. Zangerle, M. Mayerl, M. Potthast, and B. Stein, Overview of the Multi-Author Writing Style Analysis Task at PAN 2025, Working Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, Madrid, Spain, 2025. [18] M. Fröbe, M. Wiegmann, N. Kolyada, B. Grahm, T. Elstner, F. Loebe, M. Hagen, B. Stein, and M. Potthast, Continuous Integration for Reproducible Shared Tasks with TIRA.io, in: J. Kamps, L. Goeuriot, F. Crestani, M. Maistro, H. Joho, B. Davis, C. Gurrin, U. Kruschwitz, A. Caputo (Eds.), Advances in Information Retrieval - 45th European Conference on Information Retrieval (ECIR 2023), Dublin, Ireland, April 2–6, 2023, Proceedings, Part III, Lecture Notes in Computer Science, vol. 13982, Springer, 2023, pp. 236–241.