=Paper=
{{Paper
|id=Vol-2485/paper57
|storemode=property
|title=Automated Algorithm for Determining the Interplanar Distances of the Crystal Structure of a Substance from Transmission Electron Microscopy Images
|pdfUrl=https://ceur-ws.org/Vol-2485/paper57.pdf
|volume=Vol-2485
|authors=Stepan Nebaba,Alexander Pak,Alena Zakharova
}}
==Automated Algorithm for Determining the Interplanar Distances of the Crystal Structure of a Substance from Transmission Electron Microscopy Images==
https://ceur-ws.org/Vol-2485/paper57.pdf
Automated Algorithm for Determining the Interplanar Distances of the
Crystal Structure of a Substance from Transmission Electron
Microscopy Images
S.G. Nebaba1, A.Ya. Pak2, A.A. Zakharova3
stepan-lfx@mail.ru|ayapak@tpu.ru|zaa@tu-bryansk.ru
1
Keldysh Institute of Applied Mathematics RAS, Moscow, Russia;
2
Tomsk Polytechnic University, Tomsk, Russia;
3
Bryansk State Technical University, Bryansk, Russia
A problem of automated image processing of transmission electron microscopy and its application value is considered in this
paper. An automated algorithm for estimating the interplanar distances of the crystal structure of a substance from transmission
electron microscopy images is proposed. The software implementation of the algorithm was developed and tested on several raster
images, and the evaluation results were compared with the results obtained using the specialized software named Gatan Microscopy
Suite v.1.8. The high degree of coincidence of the results showed the viability of the proposed approach and the prospects of its further
development in the area of transmission electron microscopy images processing.
Keywords: computer vision, image processing, image analysis, transmission electron microscopy.
development of software systems for image analysis of electron
1. Introduction microscopes with the possibility of mass use and
implementation.
Computer vision methods and algorithms associated with
processing and analysis of raster images are widely used in 2. Automated algorithm for estimating the width
various fields of science [1,4,5,7,8], including the field of
processing of images, obtained by electron microscopes [6].
of the layers of the material in a raster image
One of the fundamental problems in these areas is the The key idea of the proposed algorithm is based on
automation of materials composition and structure evaluation selection, normalization, and evaluation of a part of the raster
according to their images. Effective solution of this problem image obtained using an electron microscope, which has a
simplifies the tasks of non-destructive quality control of regular layered structure.
materials and products, their identification and determination of The algorithm can be described with the following sequence
their properties and appearance [6]. of actions:
The development of information technology and computing 1. The user selects the area of the image that needs to be
devices contributes to rapid progress in solving such type of analyzed. It is enough to select 4 points, each of which is
problems. However, to date, existing methods of identification inside a regular structure in the image.
and evaluation of micro-objects in the raster image do not have 2. The program rotates the image at a certain angle through
sufficient versatility, which would allow them to be easily affine transformations in order to build a regular structure
automated. In addition, these techniques, methods and strictly perpendicular to the x-axis and simplify further
algorithms are often part of proprietary software, which is calculations [3].
closely associated with electron microscopy equipment and is 3. The image is binarized with a given threshold, which allows
protected by copyright of manufacturers of this equipment [2]. to split the pixels, defining the layers through white pixels,
The cost of such equipment can reach hundreds of millions of and the gaps between the layers through black pixels [8].
rubles, which is unacceptable for a sufficiently significant part 4. The sequence of sums of white pixels along the x-axis is
of researchers and scientific organizations. calculated, which shows the pixel density.
At the same time, systems for analyzing such images can be 5. After calculation of the first derivative of the sequence
widely demanded by many scientific organizations as well as by obtained in Section 4, a sequence of values characterizing
enterprises in the manufacturing sector, where they can be the number of transitions between layers can be obtained.
applied for the on-line monitoring and analysis of the Positive values of the derivative characterize the proximity
composition and structure of materials and products from them. to the center of the layer, negative ones characterize the
Cheap analogues of existing software for critical areas of proximity to the center of the gap between the layers.
enterprise activities can stimulate the development of 6. After determination of the extreme right and extreme left
technologies for the synthesis, analysis and production of micro local maxima of the sequence, as well as the first local
and nanomaterials. Finally, improving the characteristics of minimum, the threshold for the intersection of the x-axis
materials for various purposes and products based on them can can be set and, after counting the number of such
have a positive effect on the level of safety in many areas of intersections between the maxima, the average evaluation of
human activity as well as promote the development of new the layer width, which is measured in the number of points
technologies. of the original image, can be calculated.
The practical application of specialized software for image 7. The width of the layer, which is estimated by the physical
processing obtained with an electron microscope can be seen in quantity, can be obtained with the knowledge about the
[6] and many others papers. This fact indicates that the field of ratio between image points and real physical quantities.
knowledge is developing in the modern world. In general, the formula for calculating the average layer
Thus, the considered interdisciplinary problem seems width in the image can be represented as follows:
relevant and promising. There is a need in the systematization πΏ
πΏπ = π , (1)
of the existing methods and algorithms for automated πβπΏπ
evaluation and image processing and in the development of where Ls is the width of one layer, measured by a physical
software for evaluating and analyzing electron microscopy quantity (for example, nm),
images on their basis. In the future, this can contribute to the
Copyright Β© 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
LN is the length of the selected area with a regular structure
of the material in the image, in pixels,
n is the number of layers in the selected area, calculated
using the proposed algorithm,
Li is the ratio of the image size in pixels with real values
(for example, pixels / nm).
The proposed algorithm is quite simple in terms of
implementation and does not require large computational
resources, but in suggested form it requires a number of manual
actions from the user: in particular, such actions as selecting the
area of the analyzed structure in the image, checking the
correctness of the rotation angle, choosing the binarization
threshold and image ratios with real physical quantities are
predominantly manual at this moment. Almost all of these
limitations can be easily overcome by choosing an appropriate
way for parameters selection and preparation of images for
analysis, but the initial choice of the analysis area is always left
to the software user.
The described algorithm has been tested on several high Fig. 3. Graphs of pixel density in binarized and rotated images.
resolution TEM-images (Transmission electron microscopy
images) taken in the direct resolution mode, when crystalline Figure 4 shows the derivatives of the sequences, which
planes are visible in the composition of the crystalline material represent the number of "layers" (atomic planes) in the image.
directly in the image (Fig. 1).
3. Software implementation of the algorithm and
testing on real data
The software implementation of the proposed algorithm
was performed in the Microsoft Visual Studio development
environment using the open source library of computer vision
algorithms and image processing openCV.
Figure 1 presents examples of the original images, with
which the proposed algorithm was tested.
Fig. 4. Derivatives of the pixel density sequences.
Fig. 1. Examples of TEM-images shoot in direct resolution Table 1 presents parameters of the algorithm with which the
mode. tests were carried out, as well as the final result of the
evaluation of the average layer width in the image, calculated
Figure 2 shows the results of rotation and binarization of by formula (1).
selected areas of the images. Table 1. The parameters of the algorithm and the result of
evaluation the width of the layer in the image
Parameter name Image 1 Image 2
Binarization threshold 5/255 100/255
Angle of rotation 62 41
Range of points (by X- 763-1216 505-799
axis)
Li, pixels / nm 74 19
The number of layers in 25 40
the range, n
Fig. 2. Examples of images, which were binarized and rotated Estimation of the 0.2449 0.3868
in the direction of the viewed planes of the crystal structure average interplanar
perpendicular to the x-axis. distance, nm
Figure 3 shows the sequences characterizing the density The obtained results were compared with the results of
distribution of white pixels of images along the x-axis. calculations of specialized software supplied as part of an
analytical complex based on a JEOL JEM 2100F transmission
electron microscope (Gatan Microscopy Suite v.1.8) [2].
4. Comparison of the results of the proposed The value of the interplanar distance d = 0.3870 nm Β± 0.012
algorithm with the results of specialized nm was determined as a result of measurements. Lower image
software quality, as well as a smaller zoom coefficient can be the reasons
of the larger deviation in this case.
Image 1 processing with the Gatan Microscopy Suite v.1.8 As a result of comparing the values obtained by the
software (Fig. 5) made it possible to determine the interplanar developed algorithm with the values of specialized software
spacing using the Process FFT function as part of an integrated used as reference, it can be concluded that there are
assessment of the entire particle (image 1), which shows a insignificant variances in the obtained data. Differences of data,
single crystal, and the resulting interplanar spacing, d = 0.2468 which were by the developed algorithm, from the "reference"
nm, was obtained. At the same time, the Profile function made data are about 0.005 nm. Also it should be noted that the
5 measurements of interplanar distances in different parts of the specialized software package Gatan Microscopy Suite v.1.8
crystal. According to the results of these measurements, the uses in its work a βrawβ image file with total data volume
value d = 0.2498 nm Β± 0.0033 nm was determined. equals 16.9 MB, saved directly during the operation of the
electron microscope, and usually accessible to operators of
electron microscopes. At the same time, the developed
algorithm analyzes the standard graphics file with JPEG
extension and a data volume less than 1 MB, which, as a rule, is
the best quality that generally available to persons conducting
image analysis. Thus, the ability to work directly with the
public JPEG file is shown in this work.
5. Conclusion
The paper reviewed the existing approach to processing and
analyzing images of the crystal structure of substances using
specialized software for analysis of transmission electron
microscopy data.
The analysis of the problem of evaluation the average width
of the interplanar distance in the image is carried out; image
processing methods suitable for the task of automating the
calculation of the interplanar spacing of the crystal structure of
a substance, which is shown in images taken in the direct
resolution mode, are highlighted.
An algorithm that allows to calculate in a partially
Fig. 5. The result of image 1 processing with the Gatan automatic mode the interplanar distance of the crystal structure
Microscopy Suite v.1.8 software. of a substance is proposed. The proposed algorithm was tested
on several images, the numerical results were compared with
It is impossible to conduct an integral evaluation using the the results obtained with the same images using specialized
Process FFT function for image 2 due to the presence of several software that designed for analysis of images obtained with
types of crystalline objects in the image. However, the Profile using a transmission electron microscope.
function makes it possible to select the same part of the The use and further development of the proposed image
analyzed image area 2, which was analyzed above by the processing algorithm will make it possible to solve the problem
author's algorithm. Figure 6 shows the software window during of evaluation of the average value of the interplanar distance in
image processing. the image without the involvement of specialized software. It
makes sense to improve the proposed algorithm both in the
direction of better automatization of the evaluation process and
accuracy increasing. In addition, in the future, it seems
appropriate to consider the possibility of analyzing local areas
in the image in order to search for anomalies in the crystal cell,
or, in other words, its defects.
6. Acknowledgments
This work has been supported the Ministry of Education
and Science of the Russian Federation by the Grant No.
2.1642.2017/4.6.
7. References
[1] Alhadidi B. Mammogram Breast Cancer Image Detection
Using Image Processing Functions / B. Alhadidi, M.H.
Zu'bi, H.N. Suleiman. // Information Technology Journal.
2007. Vol. 6. β. 2. P. 217β221.
[2] Gatan Microscopy Suite Software [Electronic Source].
URL: https://www.gatan.com/products/tem-analysis/gatan-
microscopy-suite-software. (Last accessed: 11.06.2019).
Fig. 6. The result of image 2 processing with the Gatan [3] Gonzalez R.C. Digital Image Processing (3rd Edition) /
Microscopy Suite v.1.8 software. R.C. Gonzalez, R.E. Woods // Prentice-Hall, Inc., Upper
Saddle River, NJ, USA, 2006. P. 976.
[4] Leutenegger S. BRISK: Binary Robust invariant scalable
keypoints / S. Leutenegger, M.Chli, R.Y. Siegwart //
Proceedings of the 2011 International Conference on
Computer Vision (ICCV '11). 6 November 2011. P. 2548-
2555.
[5] Nebaba S.G. An Algorithm for Building Deformable 3d
Human Face Models and Justification of its Applicability
for Recognition Systems / S.G. Nebaba, A.A. Zakharova //
SPIIRAS Proceedings. 2017. Vol. 52. P. 157-179.
[6] Pak A. Synthesis of ultrafine cubic tungsten carbide in a
discharge plasma jet / A. Pak, A. Sivkov, I. Shanenkov, I.
Rahmatullin, K. Shatrova // International Journal of
Refractory Metals and Hard Materials. 2015. Vol. 48. P.
51-55. ISSN 0263-4368.
https://doi.org/10.1016/j.ijrmhm.2014.07.025.
[7] Schettini R. Underwater Image Processing: State of the Art
of Restoration and Image Enhancement Methods / R.
Schettini, S. Corchs // EURASIP Journal on Advances in
Signal Processing. 2010.
[8] Sokratis V. A Hybrid Binarization Technique for
Document Images / V. Sokratis, E. Kavallieratou, R.
Paredes, K. Sotiropoulos // In: Biba M., Xhafa F. (eds)
Learning Structure and Schemas from Documents. Studies
in Computational Intelligence. 2011. Vol. 375. Springer,
Berlin, Heidelberg.