=Paper=
{{Paper
|id=Vol-2763/CPT2020_paper_s7-7
|storemode=property
|title=Patterns detection in diffraction images of transmission electron microscopy
|pdfUrl=https://ceur-ws.org/Vol-2763/CPT2020_paper_s7-7.pdf
|volume=Vol-2763
|authors=Nebaba Stepan Gennadyevich,Pak Alexander Yakovlevich
}}
==Patterns detection in diffraction images of transmission electron microscopy==
https://ceur-ws.org/Vol-2763/CPT2020_paper_s7-7.pdf
Patterns detection in diffraction images of transmission electron
microscopy
S.G. Nebaba, A.Ya. Pak
stepan-lfx@mail.ru|ayapak@tpu.ru
Tomsk Polytechnic University, Tomsk, Russia
Specialized software that supports existing approaches to processing images of the crystal structure of materials for analyzing
transmission electron microscopy images have a lot of different digital image processing methods, but major part of it are weakly
automated. In some tasks automated algorithms of image processing have been developed, e.g. in task of estimation of the width of a
layer of material from a raster image. The paper considers the problem of automated processing of diffraction images obtained by
transmission electron microscopy. A number of modifications, such as Watershed algorithm, binarization and Fast Fourier Transform,
are proposed for existing image processing algorithms. These modifications can help automate the processing of the diffraction pattern
of a material sample from an image of transmission electron microscopy. The given examples of image processing of particular cases
of diffraction patterns have shown the prospects for the development of algorithm based on combination of the proposed modifications
of considered algorithms. Adaptive binarization with Watershed segmentation would be useful in automated distance estimation in
transmission electron microscopy images.
Keywords: computer vision, image processing, image analysis, transmission electron microscopy, crystalline diffraction pattern.
pattern. The TEM method can be applied to solve
1. Introduction problems such as:
Methods and algorithms of computer vision associated • characterization of the structure of the sample in
with the processing and analysis of raster images are volume and on the surface;
widely used in various fields of science [1-5], including in • determination of the qualitative phase composition of
the field of processing images obtained using electron the sample;
microscopes [6]. One of the fundamental problems in these • determination of orientational relations between the
areas is the automation of assessing the composition and elements of the structure of the sample.
structure of materials from their images. An effective The development of information technology and
solution of these problems simplifies the tasks of non- computing devices contributes to progress in solving such
destructive quality control of materials and products, their problems. However, nowadays, existing methods for
identification and determination of their properties and identifying and evaluating microobjects from a raster
appearance [6]. image do not have sufficient universality that would make
Transmission electron microscopy (TEM) involves the it easy to automate them. In addition, these methods and
study of thin images using a beam of electrons passing algorithms are often part of proprietary software that is
through them and interacting with them. The electrons that closely associated with electronic microscopy equipment
pass through the sample are focused on the imaging and is protected by the copyright of the manufacturers of
device: a fluorescent screen, a photographic plate or a this equipment [7]. The cost of such equipment can be
camera sensor. Using TEM, it is possible to study objects high, which is unacceptable for a fairly significant part of
even at the atomic level. At relatively low magnifications, researchers and scientific organizations.
the contrast on the TEM arises due to the absorption of At the same time, systems for analyzing such images
electrons by the material of the test sample. At high can be widely demanded by many scientific organizations
magnifications, the complex interaction of waves forms an as well as enterprises in the manufacturing sector, making
image that requires a more complex interpretation. it possible to carry out operational control and analysis of
Modern TEMs have operating modes that allow one to the composition and structure of materials and products.
study the elemental composition of samples, crystal Cheap analogues of existing software for critical areas of
orientation, phase shift of electrons, etc. enterprise activity can stimulate the development of
The TEM method is used to assess the structure of the technologies for the synthesis, analysis and production of
material, both in the volume of the sample and in its micro- and nanomaterials. Ultimately, improving the
surface region. TEM is one of the most highly informative characteristics of materials for various purposes and
research methods used in materials science, solid state products based on them can both have a positive impact on
physics, biology, and other sciences. the level of safety in many areas of human activity, and
The ability to observe the electron diffraction pattern contribute to the development of new technologies.
(or, in other words, to observe cross sections of the In [6] and many others papers, the practical use of
reciprocal lattice of the sample) simultaneously with the specialized software for processing images obtained using
image of the microstructure of the sample makes it an electron microscope, can be seen. That indicates the
possible to obtain valuable information on the symmetry development of this field of knowledge in the modern
of the crystal lattice and structural defects of the material world.
under study. Comparison of microphotography with Thus, the interdisciplinary topic under consideration
electron diffraction patterns makes it possible to correlate seems relevant and promising. There is a need to
the microstructure elements with a particular crystalline or systematize existing methods and algorithms for the
amorphous phase identified on the electron diffraction automated evaluation and processing of images and to
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY
4.0)
develop software for the evaluation and analysis of images As a result of comparing the values obtained in the
of electron microscopy based on them. In the future, this framework of the developed algorithm with the values
could contribute to the development of software systems obtained by specialized software and used as reference,
for analyzing images of electron microscopes with the insignificant discrepancies of the order of 0.005 nm were
possibility of mass application and implementation. obtained, which indicates the successful development of
an analogue to existing algorithms.
2. Automation of the algorithm for estimating
the width of a layer of material from a raster 3. Determination of parameters of the crystal
image lattice of materials
Earlier in [8], an automated algorithm was proposed Within the framework of increasing the level of
for estimating the width of material layers from a raster automation of TEM algorithms, determining of the crystal
image. The key idea of it is to select, normalize, and lattice parameters of material samples was chosen as the
evaluate an area with a regular layered structure of a raster next task.
image obtained using an electron microscope. Also, affine The phase composition and orientation of the crystals
transformations were used in the algorithm in order to can be determined from the diffraction patterns of the
build a regular structure strictly perpendicular to the samples.
abscissa axis and simplify further calculations [9], Diffraction contrast is formed as a result of elastic
binarization [4] and the calculation of derivatives of the scattering of transmitted electrons by regularly located
obtained one-dimensional sequences. atoms in the crystal lattice.
With the known relation between the image points and Contrast is used to determine the crystallinity of the
the actual physical quantities, it is possible to obtain the film and the lattice parameter (figure 3).
layer width estimated by the physical quantity.
The proposed algorithm is quite simple from the point
of view of implementation, and does not require large
computational resources.
The described algorithm was tested on several images
shoot in direct resolution mode; crystalline planes in the
composition of the crystalline material are visible in such
images (figure 1).
Figure 1. Examples of images shoot in direct resolution mode
Figure 3. Example of a diffraction pattern of a silicon lattice
The results were compared with the results of
calculations of specialized software (Gatan Microscopy Specialized software processes these images in a semi-
Suite v.1.8) [7]. An example of image processing using automatic mode, requiring the selection of starting points
this software is presented in figure 2. for calculations. An example of such processing is
presented in Figure 4 [6].
Figure 2. Example of image processing result in Gatan Figure 4. An example of processing a diffraction pattern by
Microscopy Suite v.1.8 specialized software
As can be seen from figure 4, the software function 4. Evaluation of the effectiveness of image
consists of determination of brightness in a circular radius processing algorithms in the task of
from the center of the diffraction pattern, as well as automating the determination of crystal
calculation of the number of points on concentric circles lattice parameters
and the distance between these circles, which characterize
There are a number of algorithms that can be useful in
the material sample. The problem with this approach is in
automating the processing of diffraction patterns. A
the necessity of accurate manual selection of the center of
number of well-known image processing algorithms and
the image and the distance to one of the concentric circles,
the possibilities of its applying to this task were
consisting of bright points.
considered.
In addition, not all images of the diffraction pattern
For images with a sufficiently high contrast, the
have the same high quality, which allows to determine
determination of distances can be carried out by
important characteristics. Images with excessive
transferring the entire image to the frequency domain, for
brightness of the central area can be the most frequent
example, using a variation of the Fourier transform (DFT).
example of low quality images (Figure 5). The situation
However, in case of irregular brightness or if the image
with reflection of points is also often can be found (Figure
does not fit the conditions for such a conversion, it will be
6). These specific cases make difficulties in developing a
impossible to determine anything from it.
universal automatic algorithm that would be able to parse
The Watershed method of automatic image
all particular cases with the same efficiency.
segmentation [10,11], available in the OpenCV library of
computer vision algorithms, was considered as one of the
possible directions in the automation of determining the
number and location of bright points in the image of the
diffraction pattern. An example of his work for such
images is presented in Figure 7.
Figure 7. An example of image segmentation using the
Watershed algorithm
The advantages of using this algorithm are in
automatic selection of all areas with large differences in
the brightness of the image and in determination of their
size and position. The disadvantage of this algorithm is
that it is very sensitive to the choice of the brightness
threshold and to its irregular distribution in the processed
Figure 5. An example of a diffraction pattern with high- image. Figure 7 shows that, despite the successful
brightness area in the center of an image
selection of points remote from the center, the entire area
in the center almost merges into one continuous segment.
Given the specifics of the images of the diffraction
pattern of materials, it can be assumed that a modification
of this algorithm that dynamically changes the sensitivity
threshold from the center of the image to its borders can
become quite effective for the task of evaluating these
images.
Another, simpler automation option may be the
calculation of a binary image according to a similar
principle, with a dynamic sensitivity threshold, taking into
account the decrease in brightness of images from the
center to the borders (Figure 8).
Figure 6. An example of a diffraction pattern with reflection of
points Figure 8. Example of a binary image of a diffraction pattern
Thus, it can be assumed that for the task of determining D., Shanenkova, Y. // Surface and Coatings
the characteristics of the diffraction pattern, the most Technology. 2011. Vol. 291. P. 1-6.
effective direction will be a complex algorithm combining [7] Gatan Microscopy Suite Software [Electronic
binarization and segmentation to automate the search for Source]. URL: https://www.gatan.com/products/tem-
the image center and bright points distant from it with the analysis/gatan-microscopy-suite-software. (Last
algorithm for calculating the brightness of concentric accessed: 11.06.2019).
circles traditional for specialized software with previously [8] Nebaba S.G., Pak A.Y., Zakharova A.A. Automated
known positions of the center and distance to these circles. Algorithm for Determining the Interplanar Distances
Further work in automation of the processing of of the Crystal Structure of a Substance from
images of diffraction patterns will be carried out in the Transmission Electron Microscopy Images // CEUR
direction of developing specific modifications of the Workshop Proceedings. 2019. Vol. 2485. pp. 248-
considered algorithms. 251. DOI: 10.30987/graphicon-2019-2-248-251
[9] Gonzalez R.C. Digital Image Processing (3rd Edition)
5. Results / R.C. Gonzalez, R.E. Woods // Prentice-Hall, Inc.,
The review of the existing approach to processing Upper Saddle River, NJ, USA, 2006. P. 976.
images of the crystal structure of materials using [10] Zaripova A.D., Zaripov D.K., Usachev A.E.
specialized software for analyzing TEM data is carried out. Visualization of high-voltage insulators defects on
The analysis of the problem of estimating the infrared images using computer vision methods //
parameters of the crystal structure from the image of the Scientific Visualization. 2019. Vol. 11 (2). pp. 88-98.
diffraction pattern of a material sample is carried out; [11] Khvostikov A.V., Krylov A.S., Mikhailov I.A.,
image processing methods suitable for automating this Malkov P.G. Trainable active contour model for
task are identified. A number of possible modifications of histological image segmentation // Scientific
existing algorithms are proposed, which allow to process Visualization. 2019. Vol. 11 (3). pp. 80-91.
such images in a partially automatic mode.
About the authors
The development of these modifications will make it
able to solve the problem of estimating the parameters of Nebaba Stepan G., Ph.D., engineer of Division for
the crystal structure without involving specialized Automation and Robotics of Tomsk Polytechnic University. E-
software. mail: stepan-lfx@mail.ru.
Pak Alexander Ya., Ph.D., associate professor of Division for
Acknowledgments Automation and Robotics of Tomsk Polytechnic University. E-
mail: ayapak@tpu.ru.
The work was supported by RFBR, Grant № 18-41-
700001.
References:
[1] Leutenegger S. BRISK: Binary Robust invariant
scalable keypoints / S. Leutenegger, M. Chli, R.Y.
Siegwart // Proceedings of the 2011 International
Conference on Computer Vision (ICCV '11). 6
November 2011. P. 2548-2555.
[2] Nebaba S.G. An Algorithm for Building Deformable
3d Human Face Models and Justification of its
Applicability for Recognition Systems / S.G. Nebaba,
A.A. Zakharova // SPIIRAS Proceedings. 2017. Vol.
52. P. 157-179.
[3] Schettini R. Underwater Image Processing: State of
the Art of Restoration and Image Enhancement
Methods / R. Schettini, S. Corchs // EURASIP Journal
on Advances in Signal Processing. 2010.
[4] Sokratis V. A Hybrid Binarization Technique for
Document Images / V. Sokratis, E. Kavallieratou, R.
Paredes, K. Sotiropoulos // In: Biba M., Xhafa F.
(eds) Learning Structure and Schemas from
Documents. Studies in Computational Intelligence.
2011. Vol. 375. Springer, Berlin, Heidelberg.
[5] Alhadidi B. Mammogram Breast Cancer Image
Detection Using Image Processing Functions / B.
Alhadidi, M.H. Zu'bi, H.N. Suleiman. // Information
Technology Journal. 2007. Vol. 6. №. 2. P. 217–221.
[6] Sivkov, A. Deposition of a TiC/Ti coating with a
strong substrate adhesion using a high-speed plasma
jet / Sivkov, A., Shanenkov, I., Pak, A., Gerasimov,