The paper considers the solution to the problem of extracting information from short lines of pharmacological orientation in Russian language. As an example, pharmacy lines are used, from which you need to extract the full name of the drug, manufacturer, form of issue, dosage, number of pieces in a package and some other parameters. To extract this information, a conditional random field (CRF) algorithm was used. There was also created a method for preliminary standardization of the strings to bring string tokens to a single form. More than seven thousand pharmacy lines were marked for the experiments and 2 CRF models were trained - with and without preliminary standardization of the lines. For the model with standardization, the following results were obtained: accuracy for diferent data sets is 0.95 (on the validation set) and 0.89 (on the test set). For the model without standardization, the accuracy is 0.95 (on the validation set) and 0.87 (on the test set).
Extracting information from texts is relevant as it is used to solve a number of problems. The main goal of the tasks of extracting information from texts is to convert unstructured text data to some structured form (for example, a table or a semantic graph) for further processing of the received data.
Text analysis mainly consists of the following steps: – vectorization of text; – application of various methods (for example, machine learning) for their further processing, depending on the problem being solved.
Text vectorization is converting words to normal form and then converting
them to vector form. For this, methods of tokenization, morphological analysis
and vectorization are used. To convert words to an imperfect form, the libraries
Mystem [
At the next stage, depending on the task, regular expressions, rules, additional dictionaries, machine learning methods, etc. are applied.
In this paper, we consider the problem of extracting information from short pharmacy lines containing information about goods sold, for their further comparison with a predetermined reference book of medicinal products. Such solutions can be applied in various fields of activity and companies. For example, a marketing agency can use this information to assess the pharmaceutical market. Large companies with many warehouses and stores can use this kind of solution to automate the accounting of their products.
A feature of the texts used in this work is their small length and high density of entities that need to be recognized. For example, a pharmacy line contains information about the name of the drug, manufacturer, batch number, taste, if available, form of release, dosage, etc. Also, the texts contain many words that were not previously known (for example, new names of drugs or manufacturers), a minimum of grammar and many abbreviations. These features severely limit the application of the most commonly used approaches and algorithms. 2
The task of extracting information from texts is a Named Entity Recognition task (NER). A named entity is an n-gram in text for which a class is defined. The task of recognizing named entities is to select continuous fragments of text and classify them.
At the entrance, a pharmacy line in Russian is given approximately of the following type: "АСКОРБИНОВАЯ К-ТА ГЛЕНВИТОЛ КЛУБНИКА №10 ТАБ.ЖЕВ. КРУТКА". It is necessary to first recognize the name of the drug, manufacturer, lot number and other parameters in this line, then link them to the reference name of the drug, manufacturer, lot number, etc. for further search for this string in the directory.
Let us list the problems that complicate the solution of this problem, which are to be solved: – Abbreviations of some words ("к-та" instead of "кислота"). – Producers recorded in diferent languages ("биодерма лаборатория" and "BIODERMA LABORATORIES"). – Words that have multiple meanings depending on the context (the word «мед» as a taste or an abbreviation for the word «медицинский»). 3
An Overview of Named Entity Recognition Methods Initially, the NER problem was solved without machine learning at all - using rule-based systems (for example, regular expressions). This solution stops working normally as soon as any ambiguities of the natural language come into play, but even in our task it can be used to determine the batch number, since a limited number of ways of recording it can be distinguished in the data. This solution gives us an f1-score of about 0.96 on one dataset and 0.93 on the other.
In [
Progress in solving the NER problem has become the methods of classical
supervised machine learning. In addition, entity dictionaries were actively used,
which did not solve the ambiguity problem, but improved the quality. Among the
algorithms that were actively used then were Support Vector Machine (SVM, [
If we are talking about modern algorithms, then the problem of recognizing
named entities is solved usually by neural network algorithms using Bi-LSTM
+ CRF (long short-term memory + conditional random fields [
In the article [
Separately, I would like to mention that short texts difer significantly from
long ones, and standard methods for recognizing named entities will work poorly
for them. This is exactly what is shown in the article [
[
Training CRF Model to Extract Entities from Pharmacy Strings The training was carried out on 6000 marked lines, which were combined into a table. Each row of the table contains the pharmacy line itself, as well as all the parameters that need to be extracted from it. The piece of the data is in the table 1. The output is a trained CRF model capable of predicting an ordered sequence of classes corresponding to these tokens for any ordered sequence of tokens. Let’s consider the presented steps of the algorithm in more detail. 4.1
String Standardization By standardizing a string in this task, we mean bringing the string tokens to a single form. The method that standardizes strings does the following conversions: – Removes extra characters (quotes, brackets, commas) – Brings tokens in cyrillic to a single form, uses a dictionary of substitutions for this. At this step, the most frequent errors in the spelling of tokens are "corrected", the ending is brought to a pre-selected form and abbreviations are replaced with full words – In fractions, replaces a comma with a dot – Removes extra spaces and add spaces where needed.
Example string before standardization ’ВАКСИГРИП СУСП.В/М И П/К 0,5МЛ/ДОЗА ШПР. №1’ and after it ’ВАКСИГРИП СУСПЕНЗИИ ВНУТРИМЫШЕЧНОГО ВВЕДЕНИЯ И ПОДКОЖНОГО 0.5 МЛ ДОЗА ШПР №1’
The application of standardization in this task has several goals: – This approach allows you to improve the accuracy of the model and learn better on a small sample (or a smaller sample to achieve similar quality, if we consider an approach with and without standardization). – Since we isolate and classify tokens to further search for the closest drug or product in a directory consisting of all possible options, the second goal of standardization is to use ordinary equality instead of using metrics to compare the proximity of tokens. This allows you to use filtering by those ifelds that are unambiguously standardized in our country. 4.2
Converting a String to the Form Required to Use CRF Initially, the data is a table of almost 6,000 labeled rows. Each row of the table contains the pharmacy row itself, as well as all the parameters that need to be extracted from it (see Table 1).
To train the CRF model, it is necessary to present the data in the form of a table, each row of which contains one token, the number of the pharmacy line from which this token was taken, as well as the class corresponding to this token (see Fig. 1).
Description of possible classes: – FORM_QN - number of pieces in a package – FULL_NAME - full name of the drug – MV - volume – NM_D - dosage – NM_F - form of issue – PROD - manufacturer – O - does not belong to any of the above classes
Not all the parameters listed here are required to appear in every line. As features of the word were used: the word itself in lower case, the last 2 characters of this word, the length of the word and a flag about whether this token is a number or not. And also the same features for two neighboring tokens. 4.4
Teaching the CRF Model to Predict the Class for a Word To train the model and conduct experiments, the entire data set was divided into training and test samples (the size of the test sample is 20% of the entire data set).
The CRF (Conditional Random Fields) method was chosen as a classification method, because it allows you to independently form a set of features by which you can vectorize words and texts and is popular for the NER problem, as it is intended for marking sequences. Using word embedding and other standard vectorization methods is not suitable for this task. New drugs appear, all words are specific, and the existing methods and pre-trained models were trained in a common vocabulary.
A random field is a multidimensional random variable V, where each component is a one-dimensional random variable. For convenience, we will assume that ∀i Vi are discrete and the set of their values is finite. We denote the implementation of a multidimensional random variable V as v ∈ Ω, where Ω is the set of all possible configurations. A random field can be represented as a graph, in which the vertices are the components of the multidimensional random variable V, the edges are the dependencies between them. A random field is called Markov if 2 Markovian conditions are satisfied: 1. ∀v ∈ Ω P (V = v) > 0 2. P (Vi = vi|Vj = vj , j ∈ A\ {i}) = P (Vi = vi|Vj = vj , j ∈ δi) where δi - set of neighbors of the vertex Vi.
A conditional random field is a Markov random field, in which the set of random variables is divided into 2 disjoint subsets - X and Y - the set of observable and hidden variables. The prediction task is to optimally reconstruct the values of y, provided that we know the observables x. That is, the optimization task is to maximize the conditional probability p (y | x): y∗ = argmaxyp(y|x). Calculation of the model p * (y | x) is solved as an optimization problem with given constraints (the diference between the observation and its estimate must be minimal and the condition Px p(y|x) = 1 for all x). According to the Hammersley-Cliford theorem (which connects Markov random fields and the Gibbs distribution), we need to maximize
Q ψ c(x,y) p(y|x) = Pc∈C(QG) ψ c(x,y′ ) ,
y′ ∈y c∈C(G) where the factor functions ψ c are usually the exponent of a linear combination of functions from features with weights that need to be determined during
K training ψ c = exp( P fk(xc, yc)θ k). This method belongs to the probabilistic k=1 methods of classical machine learning. Its implementation has good speed, which is very important when processing large amounts of information.
More details about the CRF method can be found in [
For the experiments, 2 samples were used. The first sample contains 6,000 pharmacy lines and is randomly divided into training and validation at a ratio of 80%/20%. The second sample is an additional 1000 lines taken from another dataset, which contains a significant proportion of the unknown drug for the model, since they were absent in the training sample. This sample was used for the test.
The two resulting models (with and without string standardization) were tested on validation and test datasets. In the tables 2, 3, 4 and 5, you can see the results of the experiments.
Vectorization of tokens by n-grams and further comparison of vectors using cosine distance were used as a baseline. The resulting average accuracy for further comparison was 0.65.
The first thing you may notice is better quality of both models compared to the baseline.
The model shows the worst results on the test data (especially for MV and NM_D). This can be explained by the fact that the data in the test set contain a large number of completely new drugs for the model and have some diferences from the data on which the training and validation was carried out. For example, dosages and volumes without specifying units of measurement are more common in the test set.
You can also notice that on the validation set string standardization does not improve the prediction quality, but on the test set, there are noticeable improvements for volume, dosage and form of release - the classes on which the standardization method has the most significant influence. The diference with validation can be explained by the fact that the data in the test set have more typos and abbreviations that need to be corrected through standardization, so the consequences of standardization are more noticeable.
In all experiments the model predicts full name of the drug NM_FULL best of all, the worst predictable classes are dosage NM_D and volume MV. Dificulties with dosage and volume may occur because they are too similar and easy to confuse. Using the CRF method, it was possible to obtain a model showing good results in the recognition of named entities in short texts of pharmacological topics. Accuracy for the validation data is 0.95, for the test data it is 0.89. Deterioration of results can be explained by the emergence of new drugs that are absent in the training sample, and by some diferences in the data structure - for example, the frequent absence of units of measure for volume and dosages. In the future, it is planned to improve the quality by using combinations of diferent approaches to build a model for the classification of words and by expanding the set of features for vectorization of tokens.