This paper discusses the use of the transfer learning method in a text scene detection task. The transfer learning is an e ective method in image analysis tasks, in particular, classi cation and object detection. Nevertheless, the application of this approach in combination with various methods for detecting text scenes almost is not described. The experiment is conducted to transfer knowledge about text detection to the detection for price tags using Fully Convolutional Network. COCO-Text, based on the MS COCO dataset, ImageNet dataset and dataset made up of a large number of images of the prices of various stores obtained during monitoring are taken as base datasets for the transfer learning. The target dataset is compiled from photographs of price monitoring with price tags and prices marked on them. The results of the experiment show how the application of transfer learning a ects the training speed of a FCN in the task of detection price tags and prices for each of the base datasets.
Monitoring of prices allows for e ective pricing based on the prices of competitors. At the moment, many retail chains conduct the price monitoring process in the following way: low-quali ed personnel visits competitors' shops, photographs the goods and price tags of the monitored goods, and writes down information about the goods and prices to the database, and then other employees verify the correctness of the information entering into the database. Every day more than one million photographs are received for monitoring, and therefore this process does not allow to quickly analyze the prices of competitors and conduct pricing based on this information.
The task of price tag recognition aims to signi cantly speed up the process
of monitoring the prices of competitors by reducing the number of photos being
checked. The whole task is performed in two stages: the localization of the price
tag and the price and the recognition of the name of the product and its price
[
The collection and labelling of the dataset for the detection of price tags and prices is quite complex and expensive, and therefore this article proposes the use of the method of transfer learning to solve the problem of nding price tags.
In the transfer learning, two approaches can be distinguished: the transfer
learning from a similar task and the transfer learning from a similar domain
[
There are two approaches to nding text in an image. The rst approach involves
methods based on region proposals. State-of-the-art region proposals methods
such as R-CNN in its pure form are not suitable for text searching because the
design of the anchor box is not suitable for a large aspect ratio of text strings
[
The second approach includes methods based on segmentation, such as
TextBlock FCN [
Moreover, hybrid methods are emerging that combat the shortcomings of
both approaches [
In recent years, the transfer learning method has been applied to most image
analysis tasks, such as the classi cation [
For the experiment, the Fully Convolutional Network method was chosen. This
method solves the problem of semantic segmentation. The key feature of this
method is that the architectures implementing this method do not contain any
fully connected layers (see Fig. 1) [
At the input, such a neural network receives an image and an output
image is created with the number of channels equal to the number of predicted
classes, where each channel is a binary mask showing where the object of the
corresponding class is on the image. In the original article [
To convert VGG16 architecture into FCN, the following operations were performed: fully connected layers, atten layer and last max pooling layer have been removed; convolutional layer with a kernel size of 1x1 , the number of lters equal to 128 and the relu activation function, upsampling layer increases the output of the last convolutional layer 16 times and convolution layer with a kernel size of 3x3, a number of lters equal to 2, and a sigmoid activation function have been added. The Adam optimizer was used with the parameters: learning rate is 10 3; 1 is 0.9, 2 is 0.999, is 10 7:
In this article, in addition to VGG16, the use of the MobileNet [
The transfer learning was carried out as follows: the weights of the target network were initialized by the weights of the networks trained on other tasks, then the target network was trained on the target dataset.
The training was terminated by the early stopping method: the training stopped when there was no improvement in the metric on the validation set for ve epochs.
The following data augmentations were used: image rotation from -10 to 10 degrees, shift from 0 to 0.1 in any direction, zoom from 0.9 to 1.1. 4
4.1
The transfer learning was conducted by the following datasets.
1. ImageNet. Dataset used in the annual competition for pattern recognition. At the moment it contains 14,197,122 images containing 21,841 categories.
2. COCO-Text. On this dataset, the problem of text search was solved. It is made up of pictures included in the MS COCO dataset of photos containing text. It contains 63,686 images with 145,859 text instances.
3. Dataset collected from the price monitoring pictures. This dataset was used to solve the problem of classi cation of products from the photo. It contains about 500,000 photos of goods divided into 150 classes. 4.2
The target dataset consists of 10,642 price monitoring photographs of more than 10 stores, on which there is one of six types of goods and a price tag. For each photo, the areas in which the price tag and price are located are labeled. In the photo, there may be more than one price tag and more than one price on the price tag and in such cases a price tag that corresponds to the product in the photo was chosen, and the price was selected based on the expert's opinion (see Fig. 2). 5
5.1
The Jaccard coe cient metric, also known as Intersection over Union, was chosen, which is calculated as the intersection of the predicted and true areas of the object's location by the image divided by their union. As shown in Table 1, using the transfer learning, it was possible to increase the speed of training and it was not possible to improve the quality of models. The greatest increase in the speed of training was due to the transfer learning based on one subject area. The transfer learning based on the same task did not give a gain in speed in comparison with the transfer learning based on the ImageNet dataset. Based on the results of the experiment, it can be concluded that the use of the transfer learning method makes it possible to make learning process several times faster. The best approach was the approach of transferring learning from the tasks of same subject area. In our experiment, one epoch of training took 5 minutes and we managed to reduce the training of models from 3.5 hours to 45 minutes. Acceleration of training models will allow us to more e ectively test models with di erent parameters.
In this paper, the application of the transfer learning to the Fully Convolutional Network method, which is based on segmentation, was considered. In the future, it is planned to explore the e ectiveness of the transfer of learning for methods based on region proposals. In addition, in the future, the domain's dataset will grow to more than a million images divided into more than 1000 classes, which, perhaps, will further accelerate the training process for nding price tags and prices.