English. Drug forums and online chat rooms constitute a relevant source of information for drug use, whose content can serve as reliable sources of information for national agencies with a high number of discussions taking place on various topics. We aimed at investigating whether forum posts could provide useful information as regards to both the early appearance and the monitoring of drug names. A Drug Name Recognition system was used to extract drug terms from the cryptomarket forum of Silk Road 2 thanks to a Conditional Random Fields model. Results of our analysis showed that our model enabled us to discover the presence of 232 new drug names compared to the presence of 106 traditional drug names, which reflect the importance of internet traces as being robust and exploitable with respect to crime phenomena. Italiano. I forum sulle droghe costituiscono una fonte di informazione rilevante per quanto riguarda l'uso di droghe, poiché il loro contenuto può essere utilizzato dalle agenzie nazionali visto l'alto numero di discussioni che si svolgono su vari argomenti. Il nostro obiettivo è stato quello di verificare se i post dei forum potessero fornire informazioni di rilievo per quanto riguarda sia la comparsa precoce sia il monitoraggio dei nomi delle droghe. È stato utilizzato un 'Conditional Random Field model' per estrarre i nomi di droga dal forum del cryptomarket di Silk Road 2. I risultati della nostra analisi hanno dimostrato che il nostro modello ha permesso di scoprire la presenza di 232 nuovi nomi di droghe rispetto alla presenza di 106 nomi di droghe tradizionali, il che riflette l'importanza delle tracce trovate su internet come robuste e sfruttabili rispetto ai fenomeni criminali.
choactive Substances) as a replacement of well-known drugs, whose efects have been known for years and whose trading is strictly forbidden” [8, p. 2]. NPS are In order to efectively monitor these forums, being able to defined as “substances of abuse, either in a pure form or recognize drug names is key, as it is considered a critical a preparation, that are not controlled by the 1961 Single step for drug information extraction [14]. Therefore, the Convention on Narcotic Drugs or the 1971 Convention task of automatic DNR has been defined as actively seekon Psychotropic Substances, but which may pose a pub- ing to recognize drug mentions in texts as well as to adelic health threat. The term “new” does not necessarily quately classify them into (pre-defined) categories [ 15]. refer to new inventions — several NPS were first synthe- Automatic DNR has heretofore mostly been conducted sized decades ago — but to substances that have recently in relation to pharmacovigilance (see for instance [16]) become available on the market” [9, p. 2]. As they are and goes hence one step further than the simple name among the first to be interested in new trends, researchers extraction, as it represents “the science and activities thus started investigating the massive use of online fo- concerned with the detection, assessment, understandrums. These online forums therefore possibly represent ing and prevention of adverse efects of drugs or any a novel approach of harm reduction for drug users and, other drug-related problems”, such as DDIs (drug-drug among others, an “entry point for drug support services” interactions) [17]. [7, p. 1]. A major challenge in forum analysis can how- DNR is a particularly challenging task due to several ever be pinpointed, as “unlike regular blogs, they include reasons, among which the following [15]: posts from numerous authors with vastly varying levels • The way individuals name drugs may greatly vary of activity, writing styles and skills, as well as proficiency (e.g. ‘coke’, ‘snow’ or ‘white’ can all be used to in the area to which the forum is devoted” [10, p. 787]. talk about cocaine);
In that context, the use of NLP (Natural Language Pro- • There are frequent occurrences of both abbrevicessing) techniques has to be pinpointed, as they can ations and acronyms, which make it dificult for help provide insights into the appearance of new drugs scientists to identify the exact drug users refer on the market. Indeed, several studies concentrated on to (e.g. O.C. stands for both Oxycodone and oral the automatic extraction of drug terms from online drug contraceptive); forums (see for example [11] or [12]), while other stud- • New drug names are constantly used among the ies noted that CRF (Conditional Random Fields) showed drug community (e.g. Clarity is a relatively new good performance results as regards the recognition of term to talk about MDMA); drug terms [13], thanks to the use of specific linguistic • Drug names may sometimes contain a series of features (e.g., POS (Part-of-Speech) tagging). Moreover, symbols that are mixed up with common words to the best of our knowledge, no study explored the use (e.g. 3.4-Methylenedioxy-Methamphetamine to of a CRF model for DNR (Drug Name Recognition) in a refer to MDMA); cryptomarket forum.
The aim of the current study is thus to determine • A few drug names sometimes correspond to nonwhether methods from the field of NLP and of computa- continuous strings of text, also called multi-word tional forensic linguistics can be applied for drug-term expressions (e.g. Synthetic marijuana). discovery, and more particularly, whether CRF can be used as a model for a DNR system to uncover novel drug terms from the cryptomarket forum of Silk Road 2. The ifrst objective is to classify terms that are considered as completely new in regards to a database of well-known drugs, those that are variants of already-known drugs and those that are variants of new drug terms. A second objective is to help identify new drug terms and thus strengthen the monitoring of existing NPS earlywarning systems. It also aims at understanding how the contribution of data that was extracted from a particular discussion forum, namely Silk Road 2, can be used to monitor the appearance of NPS.
The vast majority of studies conducting DNR research usually concentrate on the biomedical sector and, more particularly, on both biomedical articles [14] and medical documents [18]. These studies were generally conducted using either machine learning approaches, such as CRF and RI (Random Indexing) or using neural approaches, such as LSTM (Long Short-Term Memory). A great deal of research was equally carried out as regards social media [13], which also usually employed NLP techniques, such as word embeddings (see for example the use of Word2Vec in [13]). To the best of our knowledge, only two studies were however conducted with respect to the darknet (see [12] and [19]). As a result, it can be put forward that very few research pertaining on emerging drug terms in forums as well as on cryptomarkets have been conducted heretofore.
Making use of a list of drug names and after a preprocessing phase, Kaati et al. Kaati et al. [12] constructed context vectors using RI VSM. Then, they returned the of the CRF model and model accuracy, qualitative analywords which had context vectors similar to those of the sis of the extracted drug names. analyzed drug terms as a list of potential candidates of “new drugs” [20, p. 1]. Their RI approach yielded a pre- 3.1. Data collection, preprocessing and cision rate between 0.70 and 0.80 without more precise information as regards the recall nor the F1 score of their semi-automatic annotation model. Al-Nabki et al. Al Nabki et al. [19] developed The data used originates from a huge archive which was DarkNER, a NER (Named Entity Recognition) that was collected from 2013 to 2015 by Gwern Branwen, a freecrafted from neural networks, which concentrated on lance writer and researcher [23]. In this study, we used identifying six categories of named entities (i.e., location, data extracted from the forum of Silk Road 2, which was person, products, corporation, group, and creative-work) scraped on 19th April 2014. It contains 308.3 Mo, 29.041 from onion domains on TOR. Their model was trained on texts and it amounts to 38.422.770 tokens. the W-NUT-2017 dataset and tested on manually tagged In order to train our CFR model on accurate data (i.e. samples of TOR hidden services [19]. Among others, on data related to drugs), a filtering approach was used their NER model based on Bi-LSTM (Bidirectional Long to only retain the files in which drug names appeared. It Short-Term Memory) enabled researchers to extract drug should be highlighted that the selected files thus mention names. Their model yielded a high precision but also a at least one drug once. For that purpose, a python method very low recall, which could be linked to the presence of was developed to only keep the files which included sperare terms in their training data. It is however important cific terms (i.e., all the drug terms that appeared in the to emphasize that both these studies did not enable to UNODC conventions; the latter making up our dictionary distinguish NPS from other drugs. of drug names). The filtered corpus contains 10.269 files and amounts to 30.305.889 tokens. The whole corpus was 3. Methodology tokenized using NLTK’s tokenizer and each token was then POS-tagged using Spacy’s POS tagger, which was The CRF-DNR model used in this research is part of the trained for the English language [24]. various NLP techniques on which computational forensic To make an accurate distinction between both new linguistics has relied. Forensic linguistics “is an interdisci- and traditional drug, we focused on the definition of NPS plinary field of applied/descriptive linguistics which com- which was provided by the UNODC (i.e., United Nations prises the study, analysis and measurement of language Ofice on Drugs and Crime). In this project, the new in the context of crime, judicial procedures or disputes drugs hence correspond to the NPS as considered by the in law” [21]. In that particular context, computational UNODC, namely the drugs that are not controlled either forensic linguistics represents a relatively young field of by the 1961 Single Convention on Narcotic Drugs or the study, which is a sub-branch of computational linguistics 1971 Convention on Psychotropic Substances. Each drug that thus combines forensic science, computer science enclosed in both conventions will thus be considered as a and linguistics and which is concerned with the interac- traditional drug, while all the street names associated to tions between computers and human language in a legal these drugs will also be considered as traditional drugs context, in order to inform on criminal phenomena. It [9]. has shown various advantages in analyses of naturally Based on our dictionary of drug names, our corpus was occurring data conducted in the legal context, such as its automatically pre-annotated following the IOB2 format ability to quantify each finding, which results in scientists so as to reduce the amount of time needed to annotate being able to provide degrees of certainty to the Court the dataset. This format implies that each word must thanks to statistical models [22]. Moreover, alongside be annotated with a tag (B, I, or O). It allows to encode quantitative analyses, qualitative analyses were also con- the scope of multi-word named entities: for instance, a ducted in this research to characterize the diferent drug given drug name starts with the (B for beginning) tag terms that were extracted from our data so as to provide and its following components are tagged as (I for inside). detailed insights that can be used by forensic scientists as Non-drug words are tagged as outside (O) [25]. Another well as to enhance how forensic linguistics can help pro- feature was added to this standard format in NLP, in vide detailed and qualitative results. Our research hence order to characterized the drug as being “OLD” or “NEW” included the following phases: data collection, data filter- thanks to a distinction made in our dictionary between ing approach, content extraction, preprocessing through drugs that were enclosed in the UNODC conventions both the tokenization and the POS-tagging of the corpus, prior to 2014 (i.e., “OLD”) and drugs that were however automatic pre-annotation as well as manual disambigua- found in the dictionary, but enclosed in the conventions tion and manual annotation of the “old” and “new” drugs, after 2014 (i.e., “NEW”). This annotation layer helped features selection for the CRF-DNR model, development provide a dataset of quality which contains elements that have heretofore never been annotated within a forum and drug dataset (i.e., the distinction between “OLD” and “NEW” drug in the context of NPS).
It is important to highlight that all “B+OLD”, “B+NEW” as well as “O” tags were all manually checked after the automatic annotation step, in order to find 1) new drug names (i.e., drug names that are not enclosed in the UNODC conventions); 2) new variants of already known drug names (i.e., variants that were not enclosed in our dictionary); 3) variants of new drug names.
3.2. Extraction method Our best model, which included both the use of the dictioFor this research, we made use of CRF, a sequential clas- nary and the word embeddings, yielded a precision rate sification model that was proposed by Laferty in 2001 of 0.96, a recall of 0.85 and a F1 score of 0.90. We should Laferty et al. [26]. We opted for the use of the CRF model, notice that our model outperforms the results of our semias it is relatively easy to implement, it takes into account automatic annotation (0.90 vs. 0.88), which constituted the context of words, but also because it provides the our baseline. Hence, the quality of the corpus annotation opportunity for incorporating arbitrary overlapping fea- was also verified thanks to the use of specific metrics tures. Moreover, many successful approaches to DNR (i.e., recall, precision, and the F1 score). The performance that made use of NLP techniques, such as the CRF were results of the automatic annotation were the following: trained with specific linguistic features. After having a recall of 0.93, a precision of 0.88 and an F1 measure of read the literature, we noticed that the following fea- 0.90. Our results also outperform those found in [14]. It tures were usually used for the extraction of drug terms is however important to clarify that a LSTM-CRF model in the biomedical field [ 27], namely word embeddings, [27] also implemented for DNR showed a better perforcharacter embeddings, prefix of the token, sufix of the mance than our model, which highlights the limit of the token, POS, current token, start or end of sentence, ini- latter but also that adding a LSTM layer to our CRF could tial capital letter, all-lowercase letter, all-uppercase letter, be interesting (see Table 1 for a summary of the diverse all-letters, all-digits, if it contains digits, if it is part of a results). Other improvements could be to include both dictionary, if it contains punctuation. We however be- active learning and iterative corrector to our model, as lieve that it could also be interesting to add the length of it can help optimize the annotation using fewer training the token as a feature, as certain drug names are repre- data and by prioritizing which data should be labelled for sented as acronyms (e.g., LSD) or are particularly long the training dataset, so as to yield better annotated data. (e.g., alpha-Pyrrolidinopentiophenone). We also decided We also conducted a qualitative analysis of our drug reto add the following traits for each token-previous (i.e., sults. We observed that hallucinogens represent the most each token that precedes the current analyzed token) frequent category, followed by amphetamines, cannabis, and each token-next (i.e., each token that follows the coca and cocaine, opium and opiates, central nervous current analyzed token), namely initial capital letter, all- system depressants, opioids and synthetic cannabinoids. lowercase letter, all-uppercase letter, all-letters, all-digits, Comparing the use of traditional denomination of drugs if it contains digits, if it contains punctuation, if it is in with their street names, we observed that some drug catthe dictionary, token-length. Our feature selection thus egories are more often referred to by their traditional contains 40 linguistic features. names (i.e., opium and opiates and Central Nervous Sys
For this research, we subdivided our corpus into three tem depressants). On the contrary, other drug categories diferent datasets: 50% of the entire dataset was used to (i.e., cannabis, synthetic cannabinoid, opioids, coca and train the model, 25% to test the model and 25% to select cocaine, amphetamines and hallucinogens) show a higher the best hyperparameters. We made use of CRFSuite number of occurrences as regards their street names. from scikit learn [28] in order to develop our CRF model. These results are particularly significant considering the We then ran our CRF on the basis of the stochastic gra- drug categories of cannabis (with 89.3% of occurrences dient descent optimization algorithm with a minimum for street names), opium and opiates (with 94.8% of occurfrequency of 0.1, 100 possible iterations, a 10-fold cross- rences for traditional drug terms), opioids (with 83.84% of validation and a fixed learning rate of 0.1 to optimize occurrences for street names), amphetamines (with 91.6% our parameters, as similar methods have heretofore been of occurrences for street names). Generally speaking, it used for the optimization of the model [29]. can be observed that street names make up for the vast majority of drug term occurrences (69.1% vs. 30.9%).
Our model enabled us to discover the presence of 232 new drug names, i.e., (1) names of new drugs, that is to say drugs that do not appear in the UNODC conven- many random new drug terms. tions, (2) variant names of traditional drugs but also (3) Our analysis enabled us to grasp the number of occuracronyms of traditional and non traditional drugs). In to- rences of specific drug categories as well as of drugs that tal, 76 new drug names (32.8% of the total of new drugs), are enclosed in the UNODC conventions. It was observed 129 variant names of traditional drugs (55.6% of the total that some drug categories have a higher number of occurof new drugs) and 27 new acronyms of drugs (11.6% of rences as regards their traditional drug names (i.e., opium the total of new drugs) were found, against the presence and opiates and Central Nervous System depressants). (more or less frequent) of 106 traditional drug names as On the contrary, other drug categories (i.e., cannabis, well as their street names. As seen above, 2279 occur- synthetic cannabinoid, opioids, coca and cocaine, amrences of traditional names and their street names were phetamines and hallucinogens) show a higher number uncovered, while 788 occurrences of new drug names of occurrences as regards their street names. Generally were also detected, which amount to a total of 3067 oc- speaking, it could be observed that street names make currences, i.e., 74.3% for already known drug names and up for the vast majority of drug term occurrences. 25.7% for new drug names. It is thus important to notice Our model also enabled us to discover the presence of that although they are considered as “new drug names”, 232 new drug names (i.e., names of new drugs, that is to they make up for a certain proportion of the total number say drugs that do not appear in the UNODC conventions, of drug names. Moreover, there are also more types in variant names of traditional drugs but also acronyms of the category of new drug names than in the category of traditional and non traditional drugs). Hence, 76 new traditional drugs (258 vs. 101, that is to say 69.9% and drug names (32.8% of the total of new drugs), 129 variant 31.1%, respectively). names of traditional drugs (55.6% of the total of new drugs) and 27 new acronyms of drugs (11.6% of the total of new drugs) were found, against the presence (more or less 5. Conclusion frequent) of 106 traditional drug names as well as their street names. Moreover, 2279 occurrences of traditional In order to assist states in both their identification as well names and their street names were uncovered, while as their reporting of NPS, the UNODC decided to estab- 788 occurrences of new drug names were also detected. lished the so-called Early Warning Advisory (EWA). The It is hence important to notice that although they are latter serves as a repository full of information on known considered as “new drug names”, they make up for a NPS in order to improve the international understanding certain proportion of the total number of drug names. of NPS distribution and efects and thus to better under- Moreover, there are also more types in the category of stand particular health threats posed by the NPS. The new drug names than in the category of traditional drugs latter specifically extracted both data and information (258 vs. 101, that is to say 69.9% and 31.1%, respectively). that were found on the Internet. This is the reason why With respect to the other two DNR studies (i.e. [12] we decided to extract data from forum posts from the and [19]) that focused on forum posts, it can be observed cryptomarket of Silk Road 2, as they contain user gener- that the vast majority of the terms found in this research ated content that is diferent from simple product lists were not uncovered in these previous studies. It is thus that can be normally found on cryptomarkets. We thus important to emphasize the fact that emerging drug terms aimed at analyzing whether forum posts could provide can be both extracted and monitored thanks to online useful information as regards the early appearance of resources, such as forum posts. It should be noted that drug names. The purpose of this research was also to it is possible to rely on the various information that is developed a CRF-DNR model in order to analyze whether available on these forums when wishing to grasp new both the use of NLP techniques, such as the CRF model, drug terms. Online forums are thus promising sources and of specific linguistic features could help extract (new) for the early detection of drugs, suggesting thus that the drug terms. use of an automated system could help national agencies
For the purpose of this study, we decided to semi- to identify new drugs. automatically annotate our corpus, which enabled us Our approach however has limitations that can be to have access to an annotated corpus and thus to train worked on. It is important to notice that we only made our CRF model. It is important to emphasize that this use of data from one cryptomarket forum, namely Silk task would be particularly time-consuming should it be Road 2. Even if it is considered as a major cryptomarket, done completely manually, as new posts on (cryptomar- it is not representative of all cryptomarket forums. This ket) forums continuously appear; the latter resulting in analysis could thus be improved by using data gathered the never-ending task of manually annotating data and from other cryptomarket online forums. It could also thus new drug terms. Another advantage linked to our be interesting to analyze other online sources, such as method is the fact that the model makes use of data from websites, cryptomarket shops as well as data found in an already established list rather than by just looking at other languages but also to analyze other online sources, such as websites, cryptomarket shops. Another limitation through the analysis of digital, physical and chemiis linked to the fact that this study made use of posts that cal data, Forensic Science International 267 (2016) were launched on a specific date (i.e. 2014-04-19) and 173–182. doi:10.1016/j.forsciint.2016.08. that usually went on for several weeks, thereby giving 032. us a relatively static snapshot of the language used on [6] J. Aldridge, D. Décary-Hétu, Cryptomarkets: The this specific forum at that particular time. We could thus Darknet As An Online Drug Market Innovation, equally focus on data extracted from other periods of Technical Report, NESTA, 2015. time. An area of future research would be to perform a [7] M. J. Barratt, Discussing illicit drugs in public interstudy by conducting DNR over time, that is to say over net forums: Visibility, stigma, and pseudonymity, various months and years. This kind of study could help in: M. Foth (Ed.), CT ’11: Proceedings of the 5th Ingain insight on the rise and fall of specific drug terms. ternational Conference on Communities and Tech
Moreover, an obvious shortcoming that is linked to our nologies, Paparazzi Press, Brisbane, Australia, 2011, model is the fact that it performs poorly at identifying p. 159–168. doi:10.1145/2103354.2103376. terms that are common in the language but which also [8] J. Buxton, T. Bingham, The Rise and Challenge of have a very specific use in drug-related settings (e.g. shit). Dark Net Drug Markets, Technical Report, Global Hence, 11.34% of the semi-automatic annotation were Drug Policy Observatory, 2015. considered as false positives, which means that 11.34% [9] UNODC, NPS Leaflet: New Psychoactive Subof the terms that were annotated as drug terms were stances, Leaflet, UNODC, 2020. not drug terms but referred to other meanings. This rep- [10] F. Del Vigna, M. Avvenuti, C. Bacciu, P. Deluca, resents an important shortfall, as drug terms are often M. Petrocchi, A. Marchetti, M. Tesconi, Spotting represented as already known and common words. One the difusion of new psychoactive substances over possible step to tackle this issue would be to add a further the internet, in: International Symposium on Intelgrammatical and semantic layer into the model in order ligent Data Analysis, 2016. doi:10.48550/arXiv. to disambiguate homographs (e.g., Word to Gaussian Mix- 1605.03817. ture (w2gm)). It is thus important to emphasize that our [11] P. Deluca, Z. Davey, O. Corazza, L. Di Furia, model could be improved by using both active learning M. Farre, L. Holmefjord Flesland, M. Mannonen, and iterative corrector, as it can help optimize the annota- A. Majava, T. Peltoniemi, M. Pasinetti, C. Pezzolesi, tion using fewer training data and by prioritizing which N. Scherbaum, H. Siemann, A. Skutle, M. Torrens, data should be labelled for the training dataset, so as to P. van der Kreeft, E. Iversen, F. Schifano, Identifyyield better annotated data. Another improvement could ing emerging trends in recreational drug use; outbe to add a Bi-LSTM layer to our CRF model so as to take comes from the psychonaut web mapping project, both context and longer relationships into account. Progress in Neuro-Psychopharmacology and Biological Psychiatry 39 (2012) 221–226. doi:10.1016/ j.pnpbp.2012.07.011.
References [12] L. Kaati, F. Johansson, E. Forsman, Semantic technologies for detecting names of new drugs on dark[1] F. Caudevilla, The internet and drug markets, vol- nets, in: IEEE International Conference on Cyberume 21, EMCDA, Lisbon, 2016, pp. 69–76. doi:10. crime and Computer Forensic (ICCCF), 2016, pp. [2] 2K8.1K0r/u3it2h4o6f,0J8..Aldridge, D. Décary-Hétu, M. Sim, [13] 1S–.7S.imdopis:1o0n.,1N1.0A9d/aImCCs,CFC..2B0r1u6g.m7a7n4, 0T4.2C6o.nners, E. Dujso, S. Hooren, Internet-facilitated drugs trade: Detecting novel and emerging drug terms using An analysis of the size, scope and the role of the natural language processing: A social media corpus Netherlands, RAND Corporation, Santa Monica, study, JMIR Public Health Surveillance 4 (2018). [3] 2E0u1r6o.pdool,i:H1o0w.7i2ll4eg9a/lRdRr1u6gs07su.stain organised crime [14] Sd.oLi:i1u0,.B2. 1T9a6n/gp,Qub.lCihcehne,Xal.Wtha.n7g7, 2X6..Fan, Feature in the eu, Business Fundamentals, Europol, 2017. engineering for drug name recognition in biomed[4] J. Broséus, D. Rhumorbarbe, C. Mireault, V. Ouel- ical texts: feature conjunction and feature seleclette, F. Crispino, D. Décary-Hétu, Studying il- tion, Computational and Mathematical Methods in licit drug traficking on darknet markets: Structure and organisation from a canadian perspec- [15] SM. eLdiuic, iBn.eT(a2n0g1,5Q)..dCohie:1n0, .X1.1W5a5n/g2, 0D1r5u/g9n1a3m4e8r9e.cogtive, Forensic Science International 264 (2016) 7–14. nition: Approaches and resources, Information 6 [5] dDo. iR:1h0u.m10o1rb6a/rjb.e,fLo.rSstcaieihnlit,.J.2B0r1o6s.é0u2s,.Q04.5R.ossy, [16] (O2.0C15o)ra7z9z0a–,8S1. 0A. sdsoi,i:G1.0T.r3in3c9a0s/,Pi.nSfiom6o0n4a0to7,9J0.C.orkP. Esseiva, Buying drugs on a darknet market: A ery, P. Deluca, Z. Davey, P. van der Kreeft Torrens, better deal? studying the online illicit drug market D. Zummo, F. Schifano, Novel drugs, novel solu