Authorship identification is essential to the detection of undesirable deception of others' content misuse or exposing the owners of some anonymous malicious content. While it is widely studied for natural languages, it is rarely considered for programming languages. Accordingly, a PAN@FIRE task, named Authorship Identification of SOurce COde (AI-SOCO), is proposed with the focus on the identification of source code authors. The dataset consists of crawled source codes submitted by the top 1,000 human users with 100 correct C++ submissions or more from the CodeForces online judge platform. The participating systems are asked to predict the author of a given source code from the predefined list of code authors. In total, 60 teams registered on the task's CodaLab page. Out of them, 14 teams submitted 94 runs. The results are surprisingly high with many teams and baselines breaking the 90% accuracy barrier. These systems used a wide range of models and techniques from pretrained word embeddings (especially, those that are tweaked to handle source code) to stylometric features.
After the success of the previous tasks on source code such as SOurce COde re-use (SOCO) [
SOurce COde (AI-SOCO)1 task, one of the tracks of the 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020).2 In the following sections, we define the task, discuss the dataset we introduced to the research community, present the available baselines and results, and highlight the achieved accomplishments. nEvelop-O LGOBE https://github.com/AliOsm/ (A. Fadel)
© 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings
AI-SOCO focuses on identifying the author who wrote a given piece of code out of a predefined set of authors. This facilitates solving issues related to cheating in academic, work and open source environments. The detection of cheating in academic communities is significant to properly assess the contribution of students or researchers. Moreover, in work environments, credit sometimes goes to people who did not deserve it. Such issues of plagiarism could arise in open source projects that are available on public platforms. Similarly, systems developed for the AI-SOCO task can be used in public or private online coding contests, whether done in coding interviews or in oficial coding training contests, to detect the cheating of applicants or contestants. Such systems can also play a significant role in detecting the source of anonymous malicious software.
The dataset is composed of source codes collected from the open submissions in the CodeForces3 platform. CodeForces is an online judge for hosting competitive programming contests, where each contest consists of multiple problems. The problems’ dificulty (aka index) ranges from A to Z, where A is the easiest and Z is the hardest. The majority of contests have 5 to 7 problems (A-G). So, the occurrence of problems with indexes bigger than G is very rare. Figure 1 shows an example of a CodeForces problem, while Figure 2 shows a sample of the possible solutions by three diferent CodeForces users.
A user can solve a problem by writing a solution for it using any of the available programming languages on the website, and then submitting the solution through the website. The solution is tested automatically against a set of test cases and its result can be correct (accepted) or incorrect (wrong answer, time limit exceeded, etc.).
For our dataset, we selected the top rated 1,000 users (according to the website’s rating system) and collected 100 source codes per user. This gives a total number of 100,000 source codes. All collected source codes are correct and written using the C++ programming language. For each user, all collected source codes are from unique problems and were submitted within the smallest possible time period to avoid the possibility of users changing their coding styles over time. The dataset was split randomly with 50% used for training, 25% used for development and the remaining 25% used for testing. The training, development and unlabeled testing datasets are available through our GitHub repository.4 The full version of the dataset (with testing dataset labels) is available through Zenodo.5 Table 1 shows some statistics about the dataset, while Figure 3 shows the solutions distribution over CodeForces indexes and Figure 4 shows the solutions distribution over C++ compiler versions.
For a multi-class classification problem like ours, many evaluation metrics are suggested in the literature. However, since our dataset is completely balanced, we are evaluating the performance of the systems with accuracy since it is a simple and easily interpreted metric.
In this section, we present the baseline approaches we developed for this task along with their results. All implementations mentioned in this section are available on our GitHub repository (See Footnote 4). Table 2 shows all baselines results. The pretrained and fine-tuned models are 4https://github.com/AliOsm/AI-SOCO 5https://zenodo.org/record/4059840 available through HuggingFace models hub.6
This baseline predicts a random author for each piece of code from the list of 1,000 authors. Its expected accuracy is 0.1%. We used this baseline to put a threshold for systems, whether they 6https://huggingface.co/models?search=ai-soco are better or worse than the randomness.
To build a code representation, this baseline converts each source code into a vector that
represents the count of the 100 printable characters. Then, it builds a logistic regression [
In this baseline, we tried to improve the code representation by vectorizing the source codes
using term frequency-inverse document frequency (TF-IDF) [
We pretrained a RoBERTa [
To compare the efect of increasing the number of attention heads vs. increasing the number of Transformer layers, we trained another model with 6 Transformer layers and 12 attention heads. This model follows C++ RoBERTa Tiny pretraining and fine-tuning procedures as well. It achieved 93.19% and 92.88% accuracy on development and testing datasets, respectively.
The competition was hosted on the CodaLab10 platform. Surprisingly, 60 teams registered for the competition, 14 of which made 94 runs on the Evaluation and Post-Evaluation phases. The participating teams’ accuracy results were higher than expected, ranging from 74.52% up to 95.11% on the testing dataset.
In this section, we discuss the details of the teams’ submissions that were disclosed by the time this paper is written. Table 3 shows the results of all submissions.
We start by discussing the submissions made by the deadline. 1. Team UoB [12]: The best approach in the competition was proposed by the Team UoB.
The authors proposed several approaches and the best one was built using byte-level ngrams. Each source code is encoded into a vector representing the 20,000 most commonly occurring byte-level n-grams in the training dataset, where = 6 . Each n-gram in the vector was represented by a binary count, with 1 indicating the n-gram was present and 0 indicating the n-gram was absent in the given source code. The training dataset ngram vector representations were then used to train a densely connected neural network classifier, which could then be used to predict authorship of the development and testing datasets. Their approach achieved 95.41% and 95.11% accuracy on the development and testing datasets, respectively. 10https://competitions.codalab.org/competitions/25148 2. Team Yang1094 [13]: According to the oficial results, the second best approach is Team Yang1094. Similar to some of the models of Team UoB [12], Team Yang1094 used an approach that relies on word-level and character-level n-gram features that are used to train logistic regression classifier. The character-level n-gram features (where = 2, … , 7 ) ifltered using term frequency-inverse document frequency (TF-IDF) gave the best results for this team with an accuracy of 94.28%. 3. Team Alexa [14]: Team Alexa proposed an ensemble of Naïve Bayes (NB) models and pretrained models. Specifically, their best model is a weighted combination of Multinomial NB (MultinomialNB), Bernoulli NB (BernoulliNB), and two versions of CodeBERTa. For the NB models, their features are the top 30,000 character-level n-grams (where = 1, … , 5 ) with TF-IDF, whereas the CodeBERTa versions difer only in the learning rate. The resulting accuracy is 93.36%. 4. Team LAST [15]: This team’s submissions are all based on the KNN procedure with = 1 . As for the preprocessing and feature extraction steps, they first used an indentationaware tokenization followed by extracting token-level n-grams (where = 1, … , 4 ) and skip-grams (1 for each 3-gram and 3 for each 4-gram). Then, they used a binary coding of the presence of the features and relevance frequency as a supervised term weighting scheme along with L2 normalization. The best accuracy reported for this team is 92.19%. 5. Team FSU_HLJIT [16]: Inspired by earlier work on plagiarism detection and microblog ifltering, this team approached the task at hand as a ranking problem. The best performing system from this team ranked the source codes according to the number of occurrences of the character level n-grams (where = 15 ). Despite its simplicity, this system got an accuracy of 91.57%. 6. Team UMUTeam [17]: This team’s top system consisted of a combination of char-level n-grams and other author’s traits features. The former were combinations between 1 and 8 length applying TF-IDF with Sublinear TF scaling. On the other hand, the author’s traits included (1) the average length of code blocks and comments, (2) regular expressions to detect languages other than English, such as Arabic, Indian, or Russian; as well as (3) characters employed in the creation of ASCII-art. In addition, since some source codes included contact data such as nicknames, URLs, or Twitter accounts, the UMUTeam also compiled a Bag of Words (BOW) composed of the tokens that followed the words name and author in the block comments. Once all the features were compiled, they were filtered by applying a feature selection consisting of discarding low-variance features and the models were trained using a Random Forest (RF) classifier. The result is an accuracy of 91.16%. 7. Team Abdalrhman: This team used word-level and character-level n-grams with some preprocessing and cleaning steps for the dataset. For the preprocessing step; they normalized numbers into a special token then cleaned all lines with length greater then 50 or less than 4 characters. As for extracting the features, they used word-level and character-level (n-grams from text inside word boundaries) using TF-IDF features, (where = 2, … , 6 ) for both word-level and character-level. Then the features are stacked from both vectorizers and fed into Linear Support Vector Classification (LinearSVC) which achieved an accuracy of 90.88%.
After the competition’s deadline, two teams made interesting submissions. Their details are as follows.
• Team Twist Bytes: This team did not disclose the details of its submitted systems, which achieved an accuracy of 94.4%. This result is the second best result reported for this task. • Team SSNCSE_NLP [21]: This team used character count vectorization and TF-IDF to extract features that are then fed into an RF classifier. The resulting accuracy is 85.73%.
There are some common characteristics among all participating teams. For example, they have all explored the use of diferent versions of n-grams. In most cases, n-gram features proved to be very powerful achieving the best (or near-best) results. Moreover, byte-level and character-level n-grams seem more promising than word-level n-grams. As for classification, diferent models showed strong performance individually and within ensembles. Actually, many teams have reported that ensembling only improves the results by a small margin. Finally, deep learning models and pretrained models showed good performance, but they do not emerge as clear winners like the case for many authorship analysis tasks [22, 23]. This applies to general models, such as RoBERTa, as well as models customized for handling source code, such as CodeBerta.
Looking at the predictions made by the participating systems, one can see certain patterns and trends. Appendix A shows five “easy” samples that each participating system managed to predict correctly, whereas Appendix B shows five “hard” samples that none of the participating system managed to predict correctly. Easy samples tend to exhibit easily detectable unique patterns, such as the use of t y p e d e f and # d e f i n e , whereas hard examples tend to be short or use patterns that are common among many authors. We performed a simple intersection between the participating systems’ predictions and discovered that 17,501 test samples out the 25,000 test samples (i.e., 70% of the test set) were correctly predicted by all systems. On the other hand, only 589 test samples (i.e., 2.36% of the test set) were never predicted correctly by any system. This indicates that the problem might not be as dificult as we initially expected.
To support this conjecture, we studied the participating systems’ predictions at the user level. We found out that our test set, has 133 “very easy” users out of 1,000 users. These users have such an easily identifiable coding styles that all participating systems managed to predict all of their testing cases correctly. One might think that having 13.3% very easy users in the test set is not that bad. However, if we consider slightly less easier users, i.e., ones with prediction rates of 90% or more across all participating systems, we end up with 603 users. Going to lower prediction rates (e.g., 80% rate of 70% rate) covers a vast majority of the users (83.1% and 93.2%, respectively).
On the other hand, very few users are dificult to identify. For example, the most dificult user, User 579, had a prediction rate of only 35.14% across all participating systems. Other dificult users with prediction rates lower than 50% include User 198 (prediction rate 36%), User 998 (prediction rate 38.29%), User 945 (prediction rate 40.57%), User 563 (prediction rate 41.14%), User 780 (prediction rate 44%), and User 205 (prediction rate 48.57%). Nonetheless, the number of hard users is rather low, which justifies the relatively high accuracy levels achieved by all participating systems. Figure 5 shows the percentage of correctly predicted samples per user.
Finally, instead of looking at this issue from the sample-level or the user-level, we look at it from the problem-level. It turns out that, for most problems, all participating systems managed to correctly identify the code authors. For very few problems, none of the participating systems managed to correctly identify any of the code authors for these problems. Figure 5 shows the percentage of systems that predicted samples of problems correctly.
The work of the last author was partially funded by the Spanish MICINN under the research project MISMISFAKEnHATE on MISinformation and MIScommunication in social media: FAKE news and HATE speech (PGC2018-096212-B-C31); as well as by the European Cooperation in Science and Technology under the COST Action 17124 DigForAsp. unsupervised multitask learners, OpenAI blog 1 (2019) 9. [12] A. Crosby, H. T. Madabushi, UoB at AI-SOCO 2020: Approaches to Source Code Classification and the Surprising Power of n-grams, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [13] Y. Yang, L. Kong, Z. Han, Y. Han, H. Qi, N-gram-based Authorship Identification of Source Code, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [14] M. Bni Younes, N. Al-Khdour, Team Alexa at Authorship Identification of SOurce COde (AI-SOCO), in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [15] Y. Bestgen, Boosting a KNN Classifier by Improving Feature Extraction for Authorship Identification of Source Code, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [16] Z. Han, T. Li, X. Wang, Y. Xu, M. Wu, Z. Li, Z. Wu, Y. Han, Ranking-based and Classificationbased Approaches for Code Author Identification, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [17] J. A. García-Díaz, R. Valencia-García, UMUTeam at AI-SOCO’2020: Source Code Authorship Identification based on Character N-Grams and Author’s Traits, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [18] A. V. Mandalam, Abhishek, Embedding-based Authorship Identification of Source Code, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [19] C. Suman, A. Raj, S. Saha, P. Bhattacharyya, Source Code Authorship Attribution Using Stacked Classifier, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [20] P. Sriiesaranusorn, S. Wattanakriengkrai, T. Son, T. Tanaka, C. Wiraatmaja, T. Ishio, R. G.
Kula, Kode_Stylers: Author Identification through Naturalness of Code: An Ensemble Approach, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [21] N. N. A. Balaji, B. Bharathi, SSNCSE_NLP@Authorship Identification of SOurce COde (AI-SOCO) 2020, in: Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020), CEUR Workshop Proceedings, CEUR-WS.org, 2020. [22] W. Daelemans, M. Kestemont, E. Manjavacas, M. Potthast, F. Rangel, P. Rosso, G. Specht, E. Stamatatos, B. Stein, M. Tschuggnall, et al., Overview of pan 2019: Bots and gender profiling, celebrity profiling, cross-domain authorship attribution and style change detection, in: International Conference of the Cross-Language Evaluation Forum for European Languages, Springer, 2019, pp. 402–416. [23] M. N. AlRashdan, M. Abdullah, M. Al-Ayyoub, Y. Jararweh, Authorship analysis of english and spanish tweets, Proceedings of the Association for Information Science and Technology 57 (2020) e261.
Listing 1: Easy Sample 1 | Problem ID: 99992 | User ID: 814
Listing 2: Easy Sample 2 | Problem ID: 99452 | User ID: 742 1 # i n c l u d e < b i t s / s t d c + + . h> 2 3 u s i n g namespace s t d ; 4 5 / / u s i n g l l = i n t 6 4 _ t ; 6 u s i n g l l = l o n g l o n g ; 7 u s i n g u l l = u i n t 6 4 _ t ; 8 u s i n g i 3 2 = i n t 3 2 _ t ; 9 u s i n g u32 = u i n t 3 2 _ t ; 10 u s i n g i 6 4 = i n t 6 4 _ t ; 11 u s i n g u64 = u i n t 6 4 _ t ; 12 u s i n g p i i = p a i r < i n t , i n t > ; 13 u s i n g p l l = p a i r < l l , l l > ; 14 u s i n g l d = d o u b l e ; 15 16 # d e f i n e X f i r s t 17 # d e f i n e Y second i n t i s i z e , o s i z e ; c h a r i b u f [ BUFSIZE + 1 0 ] , obuf [ BUFSIZE + 1 0 ] ; c h a r ∗ i s , ∗ i t , ∗ os = obuf , ∗ o t = obuf + BUFSIZE ; } f o r ( ; i s d i g i t ( ch ) ; x = x ∗ 10 + ch − ’ 0 ’ , ch = g e t c h a r ( ) ) ; r e t u r n f ? −x : x ;
w r i t e ( STDOUT_FILENO , obuf , os − o b u f ) ; 180 f o r ( a u t o x : a ) p r i n t f ( ”%d ” , x ) ; p u t s ( ” ” ) ; 181 } 182 183 i n t 6 4 _ t power ( i n t 6 4 _ t a , i n t 6 4 _ t b , i n t 6 4 _ t p ) 184 { 185 i f ( ! b ) r e t u r n 1 ; 186 i n t 6 4 _ t t = power ( a , b >> 1 , p ) ; 187 t = t ∗ t % p ; 188 i f ( b & 1 ) t = t ∗ a % p ; 189 r e t u r n t ; 190 } 191 192 / / mt19937_64 rd ( chrono : : s t e a d y _ c l o c k : : now ( ) . t i m e _ s i n c e _ e p o c h ( ) . count ( ) ) ; 193 mt19937 rd ( chrono : : s t e a d y _ c l o c k : : now ( ) . t i m e _ s i n c e _ e p o c h ( ) . count ( ) ) ; 194 195 u s i n g namespace i o ; 196 197 t e m p l a t e < c l a s s T> 198 i n l i n e v o i d f r e s h m i n ( T& a , c o n s t T& b ) 199 { 200 i f ( a > b ) a = b ; 201 } 202 203 t e m p l a t e < c l a s s T> 204 i n l i n e v o i d freshmax ( T& a , c o n s t T& b ) 205 { 206 i f ( a < b ) a = b ; 207 } 208 209 c o n s t l l B = 3 1 ; 210 / / c o n s t l l MOD = 9 9 8 2 4 4 3 5 3 ; 211 c o n s t i n t INF = 1 0 0 0 0 0 0 0 1 0 ; 212 / / c o n s t l l I N F l l = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 LL ; 213 c o n s t i n t MAXN = 4 0 0 0 1 0 ; 214 215 i n t dx [ ] = { −1 , 1 , 0 , 0 , −1 , −1 , 1 , 1 } ; 216 i n t dy [ ] = { 0 , 0 , −1 , 1 , −1 , 1 , −1 , 1 } ; 217 218 l d d e t ( l d x1 , l d y1 , l d x2 , l d y2 , l d x3 , l d y3 ) 219 { 220 r e t u r n x1 ∗ y2 − x2 ∗ y1 + x2 ∗ y3 − x3 ∗ y2 + x3 ∗ y1 − x1 ∗ y3 ; 221 } 222 223 i n t n ; 224 v e c t o r < i n t > v [MAXN] ; 225 l l a [MAXN] ; 226 227 v o i d s o l v e ( ) 228 { 229 n = i n p ( ) ; 230 f o r ( i n t i = 1 ; i <= n ; ++ i ) a [ i ] = 1LL << i ; 231 l l sum = a [ n ] ; 232 f o r ( i n t i = 1 ; i < n / 2 ; ++ i ) sum += a [ i ] ; 233 f o r ( i n t i = n / 2 ; i < n ; ++ i ) sum −= a [ i ] ; sum = abs ( sum ) ; w r i t e l n ( sum ) ; f o r ( i n t T = inp ( ) ; T − −; ) s o l v e ( ) ;
Listing 3: Easy Sample 3 | Problem ID: 98902 | User ID: 721 1 # i n c l u d e < b i t s / s t d c + + . h> 2 # i n c l u d e < e x t / pb_ds / a s s o c _ c o n t a i n e r . hpp> 3 # i n c l u d e < e x t / pb_ds / t r e e _ p o l i c y . hpp> 4 5 # pragma GCC o p t i m i z e ( −O3 ) 6 # pragma GCC o p t i m i z e ( 0 f a s t ) 7 # pragma GCC o p t i m i z e ( ” u n r o l l − l o o p s ” ) 8 9 # d e f i n e f i f i r s t 10 # d e f i n e s e second 11 # d e f i n e s q r ( x ) ( x ) ∗ ( x ) 12 # d e f i n e p_b push_back 13 # d e f i n e m_p make_pair 14 # d e f i n e p l l p a i r < l l , l l > 15 # d e f i n e a l l ( v ) v . b e g i n ( ) , v . end ( ) 16 # d e f i n e pw ( x ) ( 1 l l << x ) 17 18 u s i n g namespace s t d ; 19 u s i n g namespace __gnu_pbds ; 20 t y p e d e f l o n g l o n g l l ; 21 t y p e d e f l o n g d o u b l e l d ; 22 c o n s t l l MAXN = 1 1 2 3 4 5 6 ; 23 c o n s t l l N = 1 e6 ; 24 c o n s t l l MOD = 1 e9 + 7 ; 25 26 t e m p l a t e <typename T> u s i n g o r d e r e d _ s e t = t r e e <T , n u l l _ t y p e , l e s s <T> , r b _ t r e e _ t a g , t r e e _ o r d e r _ s t a t i s t i c s _ n o d e _ u p d a t e > ;
Listing 4: Easy Sample 4 | Problem ID: 98374 | User ID: 930 1 # i n c l u d e < b i t s / s t d c + + . h> 2 # d e f i n e pb push_back 3 # d e f i n e f a s t i o s _ b a s e : : s y n c _ w i t h _ s t d i o ( 0 ) ; c i n . t i e ( 0 ) ; c o u t . t i e ( 0 ) ; 4 # d e f i n e i n t l o n g l o n g 5 # d e f i n e p i i p a i r < i n t , i n t > 6 # d e f i n e a l l ( x ) x . b e g i n ( ) , x . end ( ) 7 # d e f i n e l b lower_bound 8 # d e f i n e l d l o n g d o u b l e 9 u s i n g namespace s t d ; 10 c o n s t i n t N = 3 e5 + 1 ; 11 v e c t o r < p i i > a d j [N ] ; 12 i n t dp [N ] ; 13 i n t a [N ] ; 14 i n t ans = 0 ; 15 v o i d d f s ( i n t s r c , i n t par ) { 16 dp [ s r c ] = a [ s r c ] ; 17 m u l t i s e t < i n t , g r e a t e r < i n t >> s t ; 18 f o r ( a u t o i t : a d j [ s r c ] ) { 19 i f ( i t . f i r s t ! = par ) { 20 d f s ( i t . f i r s t , s r c ) ; 21 dp [ s r c ] = max ( dp [ s r c ] , a [ s r c ] − i t . second + dp [ i t . f i r s t ] ) ; s t . i n s e r t ( dp [ i t . f i r s t ] − i t . second ) ;
Listing 5: Easy Sample 5 | Problem ID: 97797 | User ID: 665 1 # i n c l u d e < b i t s / s t d c + + . h> 2 # i n c l u d e < e x t / pb_ds / a s s o c _ c o n t a i n e r . hpp> 3 # i n c l u d e < e x t / pb_ds / t r e e _ p o l i c y . hpp> 4 u s i n g namespace s t d ; 5 u s i n g namespace __gnu_pbds ; 6 7 t e m p l a t e <typename T> 8 u s i n g o r d e r e d _ s e t = t r e e <T , n u l l _ t y p e , l e s s <T> , r b _ t r e e _ t a g ,
t r e e _ o r d e r _ s t a t i s t i c s _ n o d e _ u p d a t e > ; 9 # d e f i n e l l l o n g l o n g i n t 10 # d e f i n e v i v e c t o r < i n t > 11 # d e f i n e v l v e c t o r < l l > 12 # d e f i n e p i i p a i r < i n t , i n t > 13 # d e f i n e p l l p a i r < l l , l l > 14 # d e f i n e pdd p a i r < double , d o u b l e > 15 # d e f i n e v i i v e c t o r < p i i > 16 # d e f i n e v l l v e c t o r < p l l > 17 # d e f i n e vd v e c t o r < d o u b l e > 18 # d e f i n e vb v e c t o r < b o o l > 19 # d e f i n e e l ” \ n ” 20 # d e f i n e u l l u n s i g n e d l o n g l o n g i n t 21 # d e f i n e l d l o n g d o u b l e 84 85 86 87 } 88 r e t u r n os << ” } ” ; 89 } 90 t e m p l a t e <typename T> 91 i n l i n e s t d : : ostream &o p e r a t o r << ( s t d : : ostream & os , c o n s t s t d : : u n o r d e r e d _ s e t < T>& v ) { r e t u r n os << s t d : : s e t <T >( v . b e g i n ( ) , v . end ( ) ) ; 92 93 } 94 t e m p l a t e <typename T1 , typename T2> 95 i n l i n e s t d : : ostream &o p e r a t o r << ( s t d : : ostream & os , c o n s t s t d : : unordered_map < T1 , T2>& v ) { r e t u r n os << s t d : : map<T1 , T2 >( v . b e g i n ( ) , v . end ( ) ) ; f o r ( typename s t d : : s e t <T > : : c o n s t _ i t e r a t o r i i = v . b e g i n ( ) ; i i ! = v . end ( ) ; ++ i i ) { i f ( ! f i r s t ) os << ” , ” ; os << ∗ i i ; f i r s t = f a l s e ;
B. Hard Samples 1 # i n c l u d e < b i t s / s t d c + + . h> 2 u s i n g namespace s t d ; 3 4 i n t main ( ) { 5 i n t a , b , c ; 6 c i n >> a >> b >> c ; 7 c o u t << min ( a + 2 , min ( b + 1 , c ) ) ∗ 3 − 3 ; 8 r e t u r n 0 ; 9 }
Listing 6: Hard Sample 1 | Problem ID: 43 | User ID: 428
Listing 7: Hard Sample 2 | Problem ID: 15564 | User ID: 289 c i n >> n ; v . r e s i z e ( n + 1 ) ; v [ s t ] . pb ( 0 ) ; s t ++; f o r ( i n t i = 0 ; i < n ; i ++) { } f o r ( i n t i = 0 ; i < s t ; i ++) { i n t c = ( l + r ) / 2 ; i f ( v [ c ] . back ( ) >= x ) l = c + 1 ; e l s e r = c ; f o r ( i n t j = 0 ; j < v [ i ] . s i z e ( ) ; j ++) i f ( v [ i ] [ j ] ! = 0 ) c o u t << v [ i ] [ j ] << ’ ’ ; c o u t << ” \ n ” ;
Listing 8: Hard Sample 3 | Problem ID: 31849 | User ID: 148 1 # i n c l u d e < i o s t r e a m > 2 # i n c l u d e < v e c t o r > 3 # i n c l u d e < a l g o r i t h m > 4 # i n c l u d e <cmath > 5 # i n c l u d e <map> 6 # i n c l u d e < s e t > 7 # i n c l u d e < sstream > 8 9 u s i n g namespace s t d ; 10 11 map< p a i r < i n t , i n t > , i n t > l o l ; 12 13 i n t gcd ( i n t a , i n t b ) 14 { 15 i f ( ! b ) r e t u r n a ; 16 r e t u r n gcd ( b , a%b ) ; 17 } 18 19 p a i r < i n t , i n t > kek [ 2 0 0 5 0 0 ] ; 20 21 i n t main ( ) 22 { 23 i n t n ; 24 c i n >> n ; 25 f o r ( i n t i = 0 ; i <n ; i ++) 26 { 27 s t r i n g s ;
Listing 9: Hard Sample 4 | Problem ID: 49881 | User ID: 71 1 # i n c l u d e < b i t s / s t d c + + . h> 2 u s i n g namespace s t d ; 3 4 i n t i n p u t ( ) { 5 c h a r c ; c i n >> c ; 6 r e t u r n c == ’ 1 ’ ; i o s _ b a s e : : s y n c _ w i t h _ s t d i o ( 0 ) ; c i n . t i e ( 0 ) ; c o u t . t i e ( 0 ) ; cout < <2000 < < e n d l ; f o r ( i n t i = 1 0 0 0 ; i > = 1 ; i − −)
cout << i << ” ” <<1<< ” ” << i << ” ” <<2<< e n d l ; f o r ( i n t i = 1 ; i < = 1 0 0 0 ; i ++)
cout << i << ” ” <<1<< ” ” << i << ” ” <<2<< e n d l ; r e t u r n 0 ;
Listing 10: Hard Sample 5 | Problem ID: 65320 | User ID: 21 7 } 8 9 voProblem i d p r i n t _ a n s ( v e c t o r < i n t > r , v e c t o r < i n t > c ) { | User ID : 10 i n t ans = 0 ; 11 f o r ( i n t i : r ) ans += i ; 12 f o r ( i n t i : c ) ans += i ; 13 14 c o u t << ans << ’ \ n ’ ; 15 f o r ( i n t i = 0 ; i < r . s i z e ( ) ; i ++) { 16 i f ( r [ i ] ) c o u t << ” row ” << i << ’ \ n ’ ; 17 i f ( c [ i ] ) c o u t << ” c o l ” << i << ’ \ n ’ ; 18 } 19 } 20 21 i n t main ( ) { 22 i o s : : s y n c _ w i t h _ s t d i o ( f a l s e ) ; c i n . t i e ( 0 ) ; 23 24 i n t n ; c i n >> n ; 25 i n t a [ n ] [ n ] , c [ n ] ; 26 f o r ( i n t i = 0 ; i < n ; i ++) { 27 f o r ( i n t j = 0 ; j < n ; j ++) { 28 a [ i ] [ j ] = i n p u t ( ) ; 29 } 30 } 31 f o r ( i n t i = 0 ; i < n ; i ++) { 32 f o r ( i n t j = 0 ; j < n ; j ++) { 33 a [ i ] [ j ] ^= i n p u t ( ) ; 34 } 35 } 36 f o r ( i n t i = 0 ; i < n ; i ++) { 37 c [ i ] = i n p u t ( ) ; 38 } 39 v e c t o r < i n t > tog_row ( n ) , t o g _ c o l ( n ) ; 40 f o r ( i n t i = 0 ; i < n ; i ++) { 41 f o r ( i n t j = 0 ; j < n ; j ++) { 42 i f ( c [ i ] + c [ j ] == 0 ) { 43 i f ( a [ i ] [ j ] ) { 44 c o u t << −1; e x i t ( 0 ) ; 45 } 46 } e l s e i f ( c [ i ] + c [ j ] == 1 ) { 47 i f ( a [ i ] [ j ] ) { 48 i f ( c [ j ] == 1 ) { 49 i f ( ! tog_row [ i ] ) { 50 tog_row [ i ] = 1 ; 51 } 52 } e l s e { 53 i f ( ! t o g _ c o l [ j ] ) { 54 t o g _ c o l [ j ] = 1 ; 55 } 56 } 57 } 58 } 59 } 60 } f o r ( i n t i = 0 ; i < n ; i ++) { f o r ( i n t j = 0 ; j < n ; j ++) { i f ( tog_row [ i ] | | t o g _ c o l [ j ] ) { i f ( tog_row [ i ] && c [ j ] ) a [ i ] [ j ] ^= 1 ; i f ( t o g _ c o l [ j ] && c [ i ] ) a [ i ] [ j ] ^= 1 ; i f ( s p a c e 1 == s a m p l e _ s p a c e ) c o n t i n u e ; i f ( s p a c e 2 == s a m p l e _ s p a c e ) { tog_row [ i ] = 1 ; f o r ( i n t j = 0 ; j < n ; j ++) {
i f ( c [ j ] ) a [ i ] [ j ] ^= 1 ; } f o r ( i n t i = 0 ; i < n ; i ++) { i f ( s a m p l e _ s p a c e [ i ] == ’ 1 ’ ) {
t o g _ c o l [ i ] = 1 ; r e t u r n 0 ;