Specialized software that supports existing approaches to processing images of the crystal structure of materials for analyzing transmission electron microscopy images have a lot of different digital image processing methods, but major part of it are weakly automated. Automatic algorithm is able to make the crystal structure analysis more fast and effective process. The paper considers the problem of automated processing of SAED images of transmission electron microscopy. Proposed automated image processing algorithm based on methods of adaptive binarization and Watershed segmentation allows one to determine the distances on the diffraction pattern of a material sample on the image of transmission electron microscopy. The proposed algorithm has been tested on several SAED images, distances were calculated in automatic mode and compared with the results of semi-automatic measurement in Digital Micrograph GMS 1.8 software. The analysis of the results showed high agreement in considered cases, which let us assume that proposed algorithm has good development prospects.
Methods and algorithms of computer vision associated with the processing and
analysis of raster images are widely used in various fields of science [
Transmission electron microscopy (TEM) is used to assess the structure of the material, both in the volume of the sample and in its surface region. TEM is one of the most highly informative research methods used in materials science, solid state physics, biology, and other sciences.
The ability to observe the electron diffraction pattern simultaneously with the image of the microstructure of the sample makes it possible to obtain valuable information on the symmetry of the crystal lattice and structural defects of the material under study. Comparison of microphotography with electron diffraction patterns makes it possible to correlate the microstructure elements with a particular crystalline or amorphous phase identified on the electron diffraction pattern.
The development of information technology and computing devices contributes to
progress in solving such problems. However, nowadays, existing methods for
identifying and evaluating microobjects from a raster image do not have sufficient
universality that would make it easy to automate them. In addition, these methods and
algorithms are often part of proprietary software that is closely associated with
electronic microscopy equipment and is protected by the copyright of the
manufacturers of this equipment [
At the same time, systems for analyzing such images can be widely demanded by many scientific organizations as well as enterprises in the manufacturing sector, making it possible to carry out operational control and analysis of the composition and structure of materials and products. Cheap analogues of existing software for critical areas of enterprise activity can stimulate the development of technologies for the synthesis, analysis and production of micro- and nanomaterials. Ultimately, improving the characteristics of materials for various purposes and products based on them can both have a positive impact on the level of safety in many areas of human activity, and contribute to the development of new technologies.
Earlier in [
As the next task in the framework of automation and analysis of TEM results, the most important is the determination of the crystal lattice parameters of material samples from the electron diffraction patterns (Selected Area Electron Diffraction, SAED). The task of decoding SAED is to determine the radii at which the diffraction maximum is located relative to the image of the central electron beam (see Fig. 1) and to recalculate the found values to the corresponding sizes of the interplanar spacings of the crystal lattice on a known scale. An example of such an image processed in a specialized software package Digital Micrograph GMS 1.8 from Gatan can be seen in Fig. 1.
Specialized software processes these pictures in a semi-automatic mode, requiring the selection of starting points for calculations. In particular, by pressing a mouse button the operator is invited to select two diffraction maxima, which, according to the operator’s opinion, are located on the same diameter of the circle concentric to the reflection of the central beam. Through this operation, the system sets a reference point. Next, the operator, by clicking the mouse button, sets the points corresponding in his opinion to the presence of a diffraction maxima at them. The system determines the distance from the geometric center of the central beam to each designated point and calculates the corresponding values of interplanar distances characteristic of the analyzed crystalline material.
With this approach, the software function is to determine the intensity of brightness in a circular radius from the center of the diffraction pattern, as well as to calculate the number of points on concentric circles and the distance between these circles, which characterize the material sample. This approach requires manual choosing the center of the image, as well as the centers of all the bright points in the image.
In addition, not all SAED images are of the same high quality in terms of image segmentation techniques. As an example of a “poor quality” image, an image with an excessive brightness of the central area of the image is shown on Fig. 2, it is difficult to identify diffraction maxima, which is typical for samples with nanoscale objects or objects characterized by short-range order. Also a typical case is SAED image of single-crystal shooting, on which diffraction maxima of the first, second, and other orders can be identified. It can also be possible to trace symmetry in the arrangement of diffraction maxima corresponding to symmetry in the crystal structure. This symmetry may be combined with reflections typical of the graphite matrix in which the crystal is located, as can be seen from the presence of ring reflections (Fig. 3). These specific cases make it difficult to automate the image processing, which would be equally effective in most special cases.
A number of well-known image processing algorithms [
As a result, a complex algorithm was developed. That algorithm includes the
Watershed automatic image segmentation method [
Given the specifics of the SAED images of materials, it was suggested that an algorithm that dynamically changes the sensitivity threshold from the center of the image to its edges can be quite effective for the task of evaluating these images. 2
Automated algorithm for evaluation of distances on SAED images As a result of a series of experiments, a sequence of methods and algorithms for image processing and analysis was selected, which made it possible to obtain a stable solution of problem of finding distances on the considered examples of SAED images
in automatic mode.
An integrated automated algorithm for evaluating distances in a SAED images (see Fig. 4) can be described by the following sequence of steps: 1. Bilateral filtering of the original image (window size is 50 points and is determined depending on the linear dimensions of the image), which allows reducing highfrequency noise with minimal loss of significant information. 2. Calculation of a binary image of the result of bilateral filtering (brightness threshold value equals 20 on the scale [0, 255]). A low brightness threshold makes it possible to approximately estimate the center of the diffraction in SAED image with the bright spot of illumination. The center of the diffraction in SAED image often does not completely coincide with the center of the image. 3. The Hough transform for detecting circles is applied to a binary image. The circle search parameters are selected for the reason that binarization with a low threshold leaves enough information only to determine the center, but not for small bright spherical objects in the image. To avoid false detection of small circles, the parameters are determined based on the linear dimensions of the image (iW is the image width in pixels, iH is the image height in pixels): the maximum circle radius (MaxR) is defined as 1/3 of the width of the original image (iW/3), and minimum (MinR) - 1/10 of its width (iW/10). The minimum distance between the centers of the circles is set equal to the width of the image (MinDist = iW), which excludes the detection of circles other than the central spot in the binary image. The scale factor of the image is taken equal to 6 (k = 6). 4. Steps 2 and 3 are repeated to specify the center of the circle (Rcc) with more stringent parameters. A binary image with a doubled brightness threshold is calculated (in the considered examples – 40 on the scale [0, 255]). 5. The Hough transform is performed with the following parameters: the radii become dependent on the radius of the circle found at the first iteration (Rfc): MaxR = Rfc + iW/10, MinR = Rfc/5; k = 4. As a result of this sequence of steps, it is possible to determine the point-center of the diffraction with higher accuracy. In the example images, the discrepancy in the definition of the center compared to the manual method was no more than 3 pixels. 6. Adaptive binarization of the initial image processed by the bilateral filter. This algorithm takes into account the relative brightness of the local areas of the image and does not set a strict binarization threshold, which is suitable for calculating the distances to all significant bright objects in the image. The following parameters were chosen as universal for SAED images: the block size is iW/40, the threshold number for each block is 255, and the subtracted coefficient is C = -1. As a result of this operation, all significant objects are highlighted on the image with a slight influence of the noise components of the image. 7. Selection of object outlines by the Watershed algorithm on an image processed by adaptive binarization. 8. The point-center (ci) is calculated for each contour through the central moments (mij) [10]. If the point-center lies outside the contour, this object is excluded from further consideration as being falsely defined. 9. The calculation of the distances between the point-center of diffraction in the SAED image and the point-centers of each of the contours (ri). The contours obtained in step 6 can be processed in various ways (histogram construction, relative and absolute brightness analysis, symmetry analysis), but the simplest way is to directly calculate the distances. 10. The set of found distances to objects in the image is converted to the units of change used (in the examples under consideration, Angstrom, the scale of measurement is 2 1/nm (Sn), the size of the scale in the image is 128 pixels (Sp = 128)) according to the
Patterns Detection in SAED Images of Transmission Electron Microscopy 7 following formula:
= 2 0∗∗ , (1) where dci is the diameter of the circle built with the center at the point of the diffraction in the SAED image through the center-point from the i-th object in the image. Thus, knowing that dci = 2*rci formula (1) can also be represented as: = 1 0∗∗ . (2) 11. Distances are sorted by increasing values, close values are defined as duplicates with a tolerance of 0.05 Angstrom, which is acceptable in most calculations. Based on these data, one can get an idea of the number of points lying at one or another distance from the diffraction center in the SAED image. 12. Assuming the assumption that distances that are repeated 2 or more times are significant, we get a set of several key distances characterizing the chemical composition of the substance in the image. 3
Testing the developed algorithm and comparing the results The developed algorithm has been tested on several SAED images (see Fig. 2 (a), 3 (b), 5 (c,d)). The software implementation of the proposed algorithm was performed in the Microsoft Visual Studio development environment using the open source library of computer vision algorithms and image processing openCV.
c)
d)
Fig. 5.Examples of SAED images for testing the proposed algorithm In Fig. 6-7 examples of binarization and the search for a center of diffraction in SAED images are shown.
Fig. 8 shows examples of adaptive image binarization. d) Fig.6. Binarization and diffraction center search c) d) Fig. 7. Binarization and search for a center of diffraction in SAED images with refined parameters
Further, the Watershed algorithm was applied to these images, the result of which can be seen in Fig. 9. d)
Fig. 9.Watershed algorithm processing examples
Results d)
Fig. 10. Distance measurement results on SAED Images Analysis the problem of evaluation of the crystal structure parameters from the SAED image of a material sample is carried out in the paper. Image processing methods suitable for automating this task is identified.
An automated image processing algorithm that makes it able to determine distances on a SAED image of a material sample is proposed.
The algorithm has been tested on several SAED images. An analysis of the results showed that the proposed algorithm is promising. In particular, the advantages of the developed method can be seen when there are many diffraction maxima on SAED, the analysis of which is usually characterized by a subjective assessment of the operator who choosing points.
The development of this algorithm will make it able to automate an evaluation of parameters of the crystal structure without involving specialized software.