This paper explores the identification of authorial style shifts in multi-author documents to facilitate further author identification. The model is proposed to employ contrastive learning techniques to analyze writing styles and optimized the sentence segment embedding output from the encoder of a pre-trained model. This optimization enables the encoder to produce more similar vector representations in space for sentences with similar styles while widening the distance between the embedding representations of paragraphs with diferent styles. By utilizing the contrastive learning-based encoder to generate sentence embeddings through an analysis of labeled data combined with paragraph sample pairs, we classified them using a fully connected layer. Experimental results demonstrate that we achieved F1 scores of 0.696, 0.717, and 0.503 on Task 1, Task 2, and Task 3 of the oficial test set, respectively.
In the realm of text analytics, discerning writing style shifts in multi-author documents is a complex yet
fascinating task. This analysis not only verifies textual integrity but also identifies potential plagiarism.
Recently, following the advancement of PAN series tasks with limited topic diversity, style analysis has
garnered greater attention. In this context (PAN 2024 [
In the realm of text style analysis, traditional methods customarily depend on manually extracted
features such as word frequency, part-of-speech tags, and sentence length statistics, or utilize models
like TF-IDF [
With the emergence of deep learning, models such as convolutional neural networks (CNN), recurrent
neural networks (RNN), long short-term memory networks (LSTM), and transformers have been
extensively utilized for text style analysis tasks. These models can autonomously learn intricate patterns
in text and efectively diferentiate between diferent text styles. For instance, RNN or LSTM’s ability to
process sequence data can capture context information in text [
Despite significant advancements in text style analysis by deep learning models, they still confront
certain challenges. Firstly, models typically necessitate substantial labeled data for training, which
may be infeasible in some domains. Secondly, deep learning models often lack interpretability [
In recent years, contrastive learning [
Previously, in style change detection tasks, Chen et al. [
PAN 24 furnishes tasks [13] with three diverse complexity tiers. It necessitates identifying all positional alterations in the stylistic conventions at the paragraphed level within an assigned text (i.e., scrutinize for stylistic discrepancies between two successive paras). The primary distinction of these tasks lies in the range of document subject matters: • Easy: The paragraphs of a document cover a variety of topics, allowing methods to make use of topic information to detect authorship changes. • Medium: The topical variety in a document is small (though still present) forcing the methods to focus more on style to efectively solve the detection task.
• Hard: All paragraphs in a document are on the same topic.
All documents are provided in English and may contain an arbitrary number of style changes.
However, style changes may only occur between paragraphs (i.e., a single paragraph is always authored
by a single author and contains no style changes). Each input problem is referenced by an ID (i.e., the
document that detects style changes), which is then used to identify the solution submitted for the
input problem. The ground truth data includes the number of authors and the binary labels of each pair
of consecutive paragraphs (1 for style changes, otherwise 0), but does not provide specific paragraph
author information [
Within this study, the contrastive learning method based on cosine phrase is used to train an encoder [
Figure 1 displays the model’s comprehensive framework, where pairs of paragraphs undergo identical pooling strategy encoders and generate vector representations in the space. We’ve completed the text embedding phase with writing style incorporated. Concurrently, a feature matrix (u, v, |u – v|) is used to through contrastive analysis. This enables final classification via a FCNN classifier.
Encoder
Pooling (u,v,|u-v|)
FCNN
Classifier v Paragraph 1
Paragraph 2
During the training phase of an encoder, each positive and negative instance pair is fed into the encoder for embedding. Ideally, the cosine distance between positive pairs should be less than that between negative pairs. That is, for any positive instances pair (, ) ∈ Ω pos and negative instances pair (, ) ∈ Ω neg, there are:
cos(i, j) > cos(k, l)
Where x represents the embedding representation of the paragraph . The work of Su et al., [14] [15]and
Sun et al. [16] suggests an efective solution to such problems, here is the equation [
∑︁ (,)∈Ωpos,(,)∈Ωneg (cos(k,l)− cos(i,j))) Where > 0 is a hyperparameter, which is taken as 20 in this experiment. The above equation is used to optimize the encoder, and the cosine distance of the encoder output instances is evaluated for correlation with the labels using the spearman metric, which assesses how well the relationship between two variables can be described using a monotonic function. (1) (2) Posi ve Pairs
Nega ve Pairs Embedding Encoder
(a) Author 1
Author 1 Posi ve Instances
Author 1
Author 2 Nega ve Instances
S1 S3
Cluster (b)
S2
Upon coding-encoder training completion, we freeze the parameters of the encoder, encode instances of the predefined paragraph pairs, extract the last layer of the model. We use the vectorial representations
Example Document He's at my place half the time and his fiance(e)'s place the other half of the time. He's been (homeless) couch surfing for several years and only recently got engaged to his other partner.
To the extent that they are there, it is with your consent. The state has passed laws making sure that vulnerable people (not saying he‘s one) don’t get abused (not saying you‘re abusing him) Group 2: Hostile territory, essentially. Mines, shelling, drones, just angry Ukrainians with guns (assuming these kids have no white flag above their heads, but even then, its not always… Lack of food and heating will not help them either. And if they are unlucky, they will have tshqeumadfsronmeaerbscyafpoirnmge.dEitshpeerctifhiceayl'llyl btoe…prevent Group 3: He was and still is self directing in the eyes of the US systems, but it would be easy to show that based on his memory test scores here and in Denmark that he was well beyond the point…
of these two matrices as paragraph embeddings (u, v), subtract them and take their absolute value, then
concatenate them with the original matrix to form a feature matrix (u, v, |u - v|) [
Firstly, it is imperative that we segment the oficial dataset to transform it into positive or negative
instances corresponding to each paragraph. In the oficial data set, there are no definitive authorship
information for each paragraph. This implies that if the sample transformation operation is based solely
on a dataset containing binary labels, both the size and accuracy of the training set will be insuficient.
(For instance, if two paragraphs written by the same author are mislabelled as negative instances while
others are deemed as positive instances, that may confuses the model.) To circumvent this issue, we
will employ the following strategy to generate positive and negative examples [
• Two-by-two combinations of negative instances between two adjacent but diferent groups.
Through this method shown in Figure 3, a significant volume of high-quality instance pairs can be obtained. Our method is exclusively utilized for the training set. For the validation set, we employ positive and negative instance pairs transformed from original labels only.
Secondly, we select the RoBERTa-base model as our pre-trained model and fed the prepared data to it. Our hyperparameters are set as follows: For the encoder, the batch size is set to 24, the maximum sequence length is 512, and the excess will be truncated. The initial learning rate is set to 1e-5, and trained in 10 epochs; For classifiers, the batch size is set to 64, the initial learning rate is set to 5e-5, and trained in 10 epochs.
The optimized result with the best F1 score of 0.8074 was accomplished by RoBERTa as pre-trained encoder model. The code and model were finally packaged in a docker file and submitted to Tira
platform for execution, leading to the final measure of the model’s performance. Table 1 provides the scores obtained by our model in the oficial test set.
This work is supported by the Social Science Foundation of Guangdong Province, China (No. GD24CZY02). [12] Y. Yan, R. Li, S. Wang, F. Zhang, W. Wu, W. Xu, Consert: A contrastive framework for selfsupervised sentence representation transfer, arXiv preprint arXiv:2105.11741 (2021). [13] E. Zangerle, M. Mayerl, M. Potthast, B. Stein, Pan24 multi-author writing style analysis (2024).
URL: https://zenodo.org/records/10677876. [14] X. Huang, H. Peng, D. Zou, Z. Liu, J. Li, K. Liu, J. Wu, J. Su, P. S. Yu, Cosent: Consistent sentence embedding via similarity ranking, IEEE/ACM Transactions on Audio, Speech, and Language Processing 32 (2024) 2800–2813. doi:10.1109/TASLP.2024.3402087. [15] J. Su, M. Zhu, A. Murtadha, S. Pan, B. Wen, Y. Liu, Zlpr: A novel loss for multi-label classification, arXiv preprint arXiv:2208.02955 (2022). [16] Y. Sun, C. Cheng, Y. Zhang, C. Zhang, L. Zheng, Z. Wang, Y. Wei, Circle loss: A unified perspective of pair similarity optimization, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6398–6407.