=Paper=
{{Paper
|id=Vol-3742/paper6
|storemode=property
|title=Development of an algorithm for identification of damage types on the surface of sheet metal
|pdfUrl=https://ceur-ws.org/Vol-3742/paper6.pdf
|volume=Vol-3742
|authors=Yurii Palianytsia,Iaroslav Lytvynenko,Abdellah Menou,Grigorii Shymchuk,Andrii Dubchak
|dblpUrl=https://dblp.org/rec/conf/citi2/PalianytsiaLMSD24
}}
==Development of an algorithm for identification of damage types on the surface of sheet metal==
https://ceur-ws.org/Vol-3742/paper6.pdf
Development of an algorithm for identification of damage
types on the surface of sheet metal
Yurii Palianytsia1,∗,†, Iaroslav Lytvynenko1,†, Abdellah Menou2,†, Grigorii
Shymchuk1,† and Andrii Dubchak1,†
1 Ternopil Ivan Puluj National Technical University, 56, Ruska Street, Ternopil, 46001, Ukraine
2 ASTI LAB, SAP+D School, Mohammed VI Polytechnic University, Benguérir, Morocco
Abstract
The article developed an algorithm for identifying types of damage on the surface of sheet
metal. Given that contour detection of elements based on classical methods, including surface
scratches, is already part of modern digital microscopic systems, our approach simplifies the
system by transitioning from image analysis or object detection to signal analysis. We achieve
this by transforming coordinates to highlight the contours of surface scratches on metal,
potentially revealing new informative features about their shape.
In this study, by conducting Exploratory Data Analysis, we propose a novel approach for the
classification of images of surface scratches using point-estimate based signal analysis instead
of traditional image analysis techniques commonly utilized in contemporary digital microscopy
systems due to contour detection being inherent in them. Our strategy offers significant benefits
over standard procedures, including smaller datasets required during model training, simpler
modeling processes, and potential identification of unanticipated informative characteristics
related to scratch geometry. To attain these objectives, we implement a sequence of
preprocessing steps followed by assorted mathematical functions intended to extract pertinent
details regarding the scratches from the initial pictures.
Keywords
Algorithm, metal surface scratches, classification, coordinate transformation, methods of
identification, selection of contours, centroid, software, algorithmic support. 1
1. Introduction
One of the important elements of modern metallurgical production is quality control of
rolled metal, which allows to ensure high quality of products under the conditions of
CITI’2024: 2nd International Workshop on Computer Information Technologies in Industry 4.0, June 12–14, 2024,
Ternopil, Ukraine
∗ Corresponding author.
† These authors contributed equally.
iaroslav.lytvynenko@gmail.com (I. Lytvynenko); palanizayb@tntu.edu.ua (Y. Palianytsia);
abdellahmenou1@gmail.com. (A. Menou); gorych@gmail.com (G. Shymchuk); andriydubchak09@gmail.com
(A. Dubchak)
0000-0001-7311-4103 (I. Lytvynenko); 0000-0002-8710-953X (Y. Palianytsia); 0000-0002-0606-6620 (A.
Menou); 0000-0003-2362-7386 (G. Shymchuk. ); 0009-0001-9742-8446 (A. Dubchak)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
continuous operation of units. Detection of metal surface defects is implemented taking
into account machine vision. Cameras that provide high-speed video recording are used to
monitor surface defects. The analytical part of the defectometric system detects and
classifies existing defects and evaluates their admissibility from the point of view of
current standards (for example, GOST 21014-88). The system photographs the surface of
metal strips using high-speed cameras. The software recognizes possible surface defects
and highlights them on individual images. Detected defects are tied to the length of the
strip, this allows the VTK specialist to view the entire list of detected defects in each
individual roll. New types of defects that can be detected in the production process are
entered into a special defect classifier of the measurement system and are memorized.
The main task of diagnostic systems in metallurgical production is to detect defects in
time, which allows to determine the cause of their formation, which makes it possible to
adjust the operation of the unit, eliminate the cause of the defect (correct the technological
process) and/or reject finished products.
Analyzing the known studies, we can come to the conclusion that there is currently a
certain system of methods, models and means of detecting defects in the production of
rolled products, methodological principles of their use have been developed, which allow
solving a wide range of tasks. At the same time, the intensification of rolling causes the
emergence of new types of defects and the need to improve the accuracy of diagnosing the
known ones, which necessitates the development of new parameters and algorithms.
2. Analysis of recent research
Recent studies suggest that new perspectives have emerged in the field of classifying
surface damage on sheet metal. Rather than relying on traditional image analysis methods
commonly used in contemporary digital microscopy systems for contour detection, we
propose a novel approach based on signal analysis. This transition simplifies the system
by transforming coordinates to highlight the contours of surface scratches on metal,
potentially revealing new informative features about their shape.
The endeavor to achieve unparalleled accuracy in the classification of microscopic
images of surface scratches epitomizes the unwavering commitment to advancing the
frontiers of material science and enhancing the rigor of quality control processes. This
sophisticated domain is anchored at the intersection of cutting-edge computational
models, sophisticated machine learning algorithms, and intricate image processing
techniques. Together, they forge a comprehensive framework that underpins the detailed
analysis and recognition of surface anomalies. These imperfections, though typically
imperceptible to unaided human vision, can significantly impact the functional integrity
and aesthetic value of materials.
In the realm of material science, the meticulous examination of surface scratches is not
merely a technical challenge; it represents a critical step in safeguarding the structural
integrity of materials subjected to rigorous use. By leveraging the power of advanced
algorithms and computational resources, researchers and practitioners are able to dissect
and interpret the complex patterns of scratches, translating microscopic aberrations into
actionable data. This data is pivotal in informing the manufacturing process, guiding the
refinement of materials, and ensuring that the final products meet the highest standards of
quality and durability.
Furthermore, the fusion of machine learning algorithms with image processing
techniques has catalyzed a transformative shift in how surface scratches are classified.
Through the iterative process of training and model optimization, these algorithms can
learn to identify and categorize a vast array of scratch characteristics with remarkable
precision. The integration of these technologies enables a dynamic and adaptive approach
to scratch analysis, accommodating the ever-evolving demands of material science and
quality assurance.
In essence, the classification of microscopic images of surface scratches is more than a
technical chalange that drives innovation and excellence in material science. As
researchers continue to push the boundaries of what is possible, the classification systems
will become increasingly sophisticated, offering deeper insights into the minute details
that define the quality of materials. This ongoing quest not only enhances our
understanding of material properties but also underscores the importance of precision
and attention to detail in the broader context of scientific inquiry and industrial
application.
The pursuit of precision in the classification of microscopic images of surface scratches
is not only a testament to the dedication to material science and quality control but also a
reflection of the broader scientific endeavor to understand and manipulate the
microscopic world. This field, which intricately intertwines advanced computational
models, machine learning algorithms, and image processing techniques, stands as a
beacon of interdisciplinary innovation, driving forward our ability to discern and
categorize the minutest of surface irregularities.
In the context of scientific research, the classification of microscopic images of surface
scratches serves a dual purpose. Firstly, it is an essential component of quality control,
ensuring that materials meet the stringent standards required for high-performance
applications. Secondly, it contributes to the fundamental understanding of material
behavior under various conditions, informing the development of new materials and
treatments that are more resistant to wear and degradation.
The realm of deep learning has not remained untouched by the quest for flawless
classification. A comprehensive review of image-based surface defect detection using deep
learning highlights the unique challenges and rapid advancements in this field. It
underscores the potential of deep learning algorithms to revolutionize defect detection,
offering insights into future research directions that could further refine these
computational techniques [4].
The scarcity of defect data poses a significant hurdle in training robust models.
Addressing this, Xiaopin Zhong and associates provide an overview of image generation
techniques for industrial surface defects. Their work explores the synthesis of defect
images through traditional and deep learning-based methods, establishing benchmarks
that could serve as the foundation for models capable of learning from a rich tapestry of
artificially generated yet realistic defect images [5].
At the forefront of this scientific endeavor is the application of transfer learning
methods within convolutional neural networks (CNNs). Jing Zhang and colleagues have
pioneered the use of pre-trained CNN models, applying fine-tuning strategies to classify
microscopic laser engraving surface defect images with remarkable success. Their
approach, which eschews the need for handcrafted features, not only enhances robustness
but also achieves an impressive accuracy, demonstrating the power of deep learning in
extracting and processing complex image features [6].
Complementing the computational prowess of CNNs is the innovative use of laser
scattering techniques. Mohammad A. Younes has shed light on the potential of these
techniques for the on-line detection and classification of surface defects. The scattered
field produced by laser interaction with surface scratches offers a unique signature that
can be harnessed for real-time inspection, providing a non-contact method that is both
efficient and effective [7].
The development of specialized neural network architectures has also made significant
strides. Wei Li and collaborators introduced WearNet, a new lightweight deep neural
network tailored for automatic scratch detection. WearNet's design allows for a smaller
model size and faster detection speed without compromising on accuracy, making it a
valuable tool for industrial applications where time and computational resources are of
the essence [8].
The role of image segmentation in scratch classification cannot be overstated, and here,
the U-Net architecture has emerged as a key player. A comprehensive survey by Jian Wu et
al. has chronicled the evolution of U-Net and its variants, highlighting the critical
importance of segmentation in the overall classification process. The adaptability and
effectiveness of U-Net in various scenarios underscore its significance in the field [9].
The current landscape of microscopic image classification is characterized by a
convergence of diverse techniques. From the self-learning capabilities of deep
convolutional networks to the precision of laser diagnostics and the innovation in network
architectures, the field is witnessing a synergy that is pushing the boundaries of what is
possible. Researchers and practitioners are continually refining these methods, ensuring
that the classification of surface scratches remains not just a scientific endeavor but also a
practical tool for quality assurance in manufacturing and beyond [10-12].
3. Main part
In this study, we propose an algorithm for identifying types of damage on metal
surface. Rather than relying on classical image analysis or object detection methods, we
simplify the system by transitioning to signal analysis. Our approach involves
transforming coordinates to highlight the contours of surface scratches on metal,
potentially revealing new informative features about their shape.
Since contour detection based on classical well-established methods, including surface
scratch detection, is already an integral part of modern digital microscopic systems, our
approach does not require significant system complexity through complex or hardware-
specific algorithms.
The essence of our method lies in transitioning from image analysis or object detection
to point-estimates-based signal analysis, which potentially contains new informative
features about scratch shapes.
Our innovative approach includes the following steps:
Step 1: Image Preprocessing
- Convert RGB images to gray scale to simplify edge detection and decrease the number
of channels required for processing.
- Apply Gaussian filtering to minimize noise levels and improve edge detection
accuracy. This technique uses a weighted average blurring effect on neighboring pixels to
smoothen out minor variations in brightness levels, allowing clearer identification of
sharp transitions in light intensity indicative of possible fractures or structural anomalies.
- Normalize the resultant matrix such that each value lies within the range of 0 to 1.
This facilitates comparisons between different images as normalized data represents
relative brightness rather than absolute luminance.
Step 2: Binary Image using Otsu's Thresholding Technique
Step 3: Remove unwanted pixels from binary images utilizing morphology functions to
eliminate areas with less than 0.00015 times the total area's worth of pixels.
Step 4: Perform Morphological Closing on Grayscale Images. The Morphological Close
Operation combines a dilation and erosion process with identical structuring elements
applied to both procedures.
Step 5: Fill any empty spaces estimated based on the boundary perimeter.
Step 6: Identify the borders and centers of mass (centroids) of each object.
Step 7: Calculate distances from each centroid's location to its border in all directions
(i.e., 360 degrees).
Step 8: Extract time-invariant features known as Power Spectral Density (PSD).
Step 9: Conduct further data exploration through Exploratory Data Analysis (EDA).
For simplification, in this work we will consider several types of damage, in particular,
we will consider three types, conditionally dividing them into: attrition, line, scratch.
Figure 1 shows three types of damage.
attrition line scratch
03ca4d04c-segm.jpg 00d639396-segm.jpg 00af2671f-segm.jpg
а) b) c)
Figure 1: Example of identified pseudocolored metal surface damage
а) b) c)
Figure 2: Example of converting contours of damage types
Below is a high-level flowchart illustrating the steps of the proposed algorithm for
identifying metal surface damage based on contour analysis of surface scratches:
Figure 3: Proposed algorithm flow chart
Let's consider the algorithm in more detail step by step.
Step 1: Image Preprocessing
Grayscale Conversion
The initial phase of image preprocessing involves the conversion of RGB (Red, Green,
Blue) images to grayscale. This is a critical step because it simplifies the subsequent edge
detection process by reducing the complexity of the image data. In a grayscale image, each
pixel represents a shade of gray, corresponding to the luminance of the original colors.
The transformation from RGB to grayscale can be mathematically represented as:
𝑌 = 0.299𝑅 + 0.587𝐺 + 0.114𝐵 (1)
where Yis the luminance component, and R, G, B are the red, green, and blue color
components, respectively. This formula is derived from the human eye’s sensitivity to
different colors, with green being the most sensitive and blue the least.
Gaussian Filtering
After the conversion to grayscale, Gaussian filtering is applied to the image. Gaussian
filtering is a smoothing technique that reduces noise in an image. It is named after the
Gaussian (normal) distribution, which is used to create a convolution matrix (kernel)
applied to each pixel and its neighbors in the image. The Gaussian function is given by:
𝑥2 +𝑦2
1 −
𝐺(𝑥, 𝑦) = 2𝜋𝜎2 𝑒 2𝜎2 (2)
where x and y are the distances from the origin in the horizontal and vertical axes,
respectively, and σ is the standard deviation of the Gaussian distribution. The effect of this
filtering is to blur the image, which helps in reducing the impact of noise on edge
detection.
Normalization
The final step in the preprocessing stage is normalization. Normalization adjusts the
range of pixel intensity values. The purpose of normalization is to bring the intensity
values within a standard range, 0 to 1. This is important for comparing images captured
under different conditions and lighting. The normalization can be expressed as:
𝐼−𝐼𝑚𝑖𝑛
𝐼𝑛𝑜𝑟𝑚 = (3)
𝐼𝑚𝑎𝑥 −𝐼𝑚𝑖𝑛
where Inorm is the normalized intensity, I is the original intensity, Imin and Imax are the
minimum and maximum intensities in the image, respectively.
By performing these steps, the image is prepared for further analysis, such as
thresholding and feature extraction, which are crucial for the classification of surface
scratches. The preprocessing not only simplifies the data but also enhances the features
that are essential for accurate classification.
Step 2: Binaration (thresholding) Image using Otsu’s Thresholding Technique
Thresholding is a fundamental technique in image processing, particularly useful for
segmenting images into different regions. Among other (Global Thresholding, Adaptive
Thresholding, Entropy-based Thresholding, Multiple Thresholding) we applied, Otsu’s
Method is outstanding, which become stand-of-the-art method.
The key idea is to find the threshold value ( t ) that minimizes the weighted within-class
variance, which is equivalent to maximizing the between-class variance. This can be
mathematically represented as:
σ2B (t) = ω0 (t)ω1 (t)[μ0 (t) − μ1 (t)]2 (4)
where: σ2B (t) is the between-class variance; ω0 (t) and ω1 (t) are the probabilities of the
two classes separated by the threshold t ; μ0 (t) and μ1 (t) are the class means.
The algorithm exhaustively searches for the threshold that maximizes 𝜎𝐵2 (𝑡).
The implementation of Otsu’s method involves computing the histogram of the
grayscale image and then calculating the probability of each intensity level. The
cumulative probability and cumulative mean are also computed, followed by the between-
class variance for each intensity level. The optimal threshold is the intensity level that
maximizes the between-class variance.
The advantages of using Otsu’s method for creating binary images are: - Automatic
threshold selection: It does not require manual intervention, making it suitable for
automated systems. - Robustness: It is relatively unaffected by the overall brightness of
the image, which is beneficial when processing images with varying illumination
conditions. - Efficiency: It can be implemented efficiently, which is critical when dealing
with large datasets or real-time applications.
The resulting binary image after applying Otsu’s thresholding is a simplified
representation where the pixels are assigned a value of 0 or 1. This binary representation
is crucial for the subsequent steps in our approach, as it lays the groundwork for
morphological operations and feature extraction.
Step 3, 4, 5: Morphology Image Processing
In step 3 of our approach, we utilize morphology functions to remove unwanted pixels
from binary images. Morphology refers to the study of geometric structures and their
properties, particularly as they relate to mathematical morphology, which deals with the
spatial structure of objects by examining how they change under various transformations.
In this context, morphological operations are applied to binary images to modify their
shape, size, or position without affecting their logical content.
Morphological closing operation, also called reconstruction or hitting, is one of these
operations. It involves combining dilatation and erosion processes with identical
structuring elements applied to both operations. A structuring element is a small mask or
template used to manipulate an image at specific locations by shifting it over the original
image.
Dilatation is a morphological transformation that enlarges the objects in an image
while preserving their shape. It fills holes inside the foreground region (the white portion)
of an image with background values (black), thereby thickening the boundaries of those
regions. Erosion, conversely, reduces the size of objects in an image by removing pixels
near the edges. It shrinks the boundaries of objects, leaving them thinner but more
distinct.
Closing is the combination of dilation followed immediately by erosion. It helps to fill
narrow gaps or thin lines that may arise during segmentation, thus improving connectivity
among adjacent objects. By applying a close operator to a binary image, we smooth out its
boundary and enhance the contrast between objects and the background, making it easier
to analyze the resulting pattern.
Here, we apply the morphological closing operation to grayscale images instead of
binary ones because some scratches might have weak contrast against the substrate,
leading to low signal-to-noise ratios. In such cases, the gray level intensities of scratches
might fall below the threshold set during binarization, causing false negatives. Applying
closing first increases the pixel intensity in shallow areas around the scratch boundaries,
effectively boosting the contrast before converting back into binary format.
Aside from increasing the contrast and filling narrow gaps, another advantage of
implementing morphological closing is its ability to reduce salt-and-pepper noise, which
occurs randomly across the image. Salt-and-pepper noise appears as isolated black and
white specks scattered throughout an otherwise homogeneous background. These spots
often cause problems during segmentation, especially if the noise level exceeds the
contrast difference between the objects and the background. Morphological closing can
mitigate salt-and-pepper noise by merging nearby black and white specks together,
creating larger, more cohesive patches.
Employing morphological closing techniques enhances image quality by eliminating
unwanted pixel artifacts arising from noise and other irregularities. As a result, we achieve
improved segmentation outcomes and more reliable feature extractions during
exploratory data analysis. Therefore, incorporating morphological closing into our
innovative approach serves as a crucial stage that amplifies the effectiveness of
subsequent stages, making our proposed framework superior to traditional methods
based solely on classical well-established methods for contour detection.
Morphological closing is defined as the dilation of an image A by a structuring element
B, followed by erosion of the resulting image by the same structuring element. This can be
represented as:
Closing(A, B) = Erosion(Dilation(A, B), B) (5)
The dilation of A by B, denoted A ⊕ B, is defined as:
A ⊕ B = ⋃ Ab (6)
b∈B
where Ab is the translation of A by b. In simpler terms, it’s the set of all displacements
of B that intersect with A.
The erosion of A by B , denoted A ⊖ B, is defined as:
A ⊖ B = ⋂ A−b (7)
b∈B
where A−b is the translation of A by −b. It’s the set of all points x such that B, translated
by x, is contained in A.
For grayscale images, the operations are slightly different. The dilation of a grayscale
image f by a structuring element B is given by:
(f ⊕ B)(x) = max{f(x − b)} (8)
b∈B
And the erosion is given by:
(f ⊖ B)(x) = min{f(x + b)} (9)
b∈B
Thus, the closing of a grayscale image f by B can be represented as:
Closing(f, B) = (f ⊕ B) ⊖ B (10)
This operation enhances the contrast and fills in the gaps, as described, making it easier
to segment and analyze the scratches in our images.
Step 6, 7: Identification of Borders and Centers of Mass (Centroids)
Step 6, 7 of our approach, focusing on the identification of borders and centers of mass
(centroids) of each object in the context of surface scratch analysis.
The identification of borders and centers of mass, or centroids, is a pivotal step in the
analysis of microscopic images of surface scratches. This step is crucial for understanding
the geometric properties of the scratches and for subsequent feature extraction processes.
Centroid Identification
The centroid of an object in an image is the geometric center, and it’s a fundamental
characteristic used in various analyses and in most cases is already already an integral
part of modern digital microscopic systems. Mathematically, the centroid C of a shape is
the arithmetic mean position of all the points in the shape. For a discrete set of points
{xi , yi } representing the object in a binary image, the centroid coordinates (Cx , Cy ) are
calculated as:
∑n
i=1 xi ∑n
i=1 yi
Cx = n
, Cy = n
(11)
where n is the number of pixels in the object.
In the context of surface scratches, identifying the centroid allows us to analyze the
scratch’s orientation, length, and other morphological features relative to its central point.
Border Detection
Border detection involves identifying the outermost edges of objects within an image.
This is typically achieved through edge detection algorithms that look for discontinuities
in pixel intensity. For surface scratches, accurate border detection is essential for
determining the exact shape and size of the scratches.
Once the borders are detected, we can represent the boundary of a scratch using a set
of contour points {(xj , yj )}. These points are ordered and form a closed loop around the
object.
Centroid to Border Distance Measurement
With the centroids and borders identified, we can measure the distance from the
centroid to the border in all directions for 360 degrees. This radial distance function R(θ)
can be expressed as:
2 2
R(θ) = √(xj − Cx ) + (yj − Cy ) (12)
where (xj , yj ) are the coordinates of the contour points closest to vector oriented in n-th
integer degree direction, and θ is the angle between the centroid-to-border line and a
reference axis, typically the horizontal axis of the image.
This radial distance function provides a profile of the scratch shape and is particularly
useful for characterizing irregularities and asymmetries in the scratch morphology.
Feature Extraction from Centroid and Border Information
The information obtained from the centroids and borders serves as a basis for
extracting various features that describe the scratches. These features can include:
Area: Calculated by integrating the radial distance function over the entire contour.
Perimeter: Sum of the distances between consecutive contour points.
Circularity: A measure of how close the shape is to a perfect circle, calculated using the
area and perimeter.
Aspect Ratio: The ratio of the scratch’s length to its width, providing insights into its
elongation.
Orientation: The angle between the major axis of the scratch and a reference axis.
These features are instrumental in classifying scratches and understanding their
impact on the material’s surface properties.
The identification of borders and centroids is a foundational step in the classification of
microscopic images of surface scratches. It enables the extraction of meaningful features
as well as “time-invariant” features that are critical for the accurate characterization of
scratches. By transitioning from traditional image analysis to signal analysis, we can
uncover new informative features that enhance our understanding of scratch shapes and
their implications.
Step 6 and 7 is essential point what highlights the importance of precise geometric
analysis in the broader context of surface scratch classification and sets the stage for
further data analysis and ML model training.
4. Discussion of obtained results
The developed algorithm provides the following advantages:
The ability to do without large databases, but to use classic time-tested methods.
After feature extraction, my method eliminates the need for complex AI-based
approaches and everything comes down to simple AI-based methods.
The transition from image analysis or Object Detection to the analysis of signals that
potentially contain new, previously unknown, informative signs about the shape of
scratches.
1. No need for large databases as traditional methods can still be used.
2. After feature extraction, simpler AI-based approaches become sufficient since
complicated AI-based approaches are no longer necessary.
3. Transitioning from analyzing images or Object Detection to analyzing signals that
may hold newly discovered informative features regarding scratch shapes.
This shift towards signal analysis has several benefits, including:
a) Reduction in data requirements as traditional methods may suffice with fewer
examples needed for training models.
b) Simplified modeling processes due to less complex AI-based approaches required
after feature extraction.
c) Potential discovery of novel informative features related to scratch shapes through
signal analysis.
In our recent study, we propose a novel approach for the analysis of scratch
morphology using machine learning techniques. Unlike traditional image processing
methods such as contour detection and segmentation, which have been integrated into
most electronic microscope systems, our approach focuses on extracting information from
the extracted contours themselves.
By focusing on the extracted contours, we transition from image processing to signal
analysis, a shift that holds the promise of uncovering novel insights into the nature of
surface scratches. This perspective is particularly advantageous for scratches that exhibit
complex shapes or orientations [13], as the signal representation can capture intricate
details that may be challenging to discern from visual inspection alone.
One of the key strengths of our approach lies in its robustness to rotational variations.
By employing time-invariant features, such as the Power Spectral Density (PSD), our
algorithm becomes insensitive to the orientation of the scratches.
Furthermore, our approach is remarkably adaptable and hardware-agnostic. Unlike AI-
based techniques that often necessitate large datasets and specialized hardware, our
method is inherently ML-based and relies on classical algorithms that have been refined
over decades. This not only reduces the risk of overfitting but also ensures that our
algorithm can be readily integrated into existing microscopy systems without extensive
modifications.
However, it should be noted that while our current research has shown promising
results, further investigation is required to establish the method's generalizability across
diverse datasets and different types of materials. Nonetheless, given its efficiency,
simplicity, and versatility, we believe that our proposed technique represents a promising
avenue for future developments in the field of nanotechnology and material science.
Conclusions
In this paper, we present a novel approach for the classification of microscopic images
of surface scratches through exploratory data analysis (EDA), focusing specifically on
point-estimates-based signal analysis instead of traditional image analysis techniques
commonly employed in modern digital microscopy systems, since contour detection is
already an integral part of it. Our method provides several advantages over conventional
approaches, namely requiring smaller datasets during model training, enabling more
straightforward modeling processes, and potentially identifying previously undiscovered
informative features relating to scratch shape.
To achieve these goals, we have implemented a series of preprocessing stages followed
by various mathematical operations aimed at extracting relevant information about the
scratches from the input images. These stages include converting color images into
grayscale representations, applying Gaussian filters to reduce noise levels, normalizing
pixel values, thresholding, removing isolated regions, performing morphological closing,
filling empty gaps, calculating centroids, finding their respective boundaries, measuring
distances from those locations to adjacent edges, and finally computing power spectral
densities (PSDs). We then employ EDA techniques to analyze these PSD values to identify
patterns and relationships among them, ultimately leading to potentially improved scratch
classification capabilities.
Additionally, let us show the distances (minimum, median, maximum) between
centroid coordinates of clusters in the first three principal component analysis (PCA)
components: minimum 0.0028, median 0.019, maximum 0.0281. These measures provide
further insight into the distribution of the dataset in the reduced dimensional space
obtained via PCA transformation.
In future research, we plan to further develop the concept of treating scratch contours
as signals. This entails exploring additional time-domain and spectral-domain features,
such as Root Mean Square (RMS) Energy, Peak Amplitude, Crest Factor, Silence Ratio,
Temporal Centroid, Log Attack Time, Spectral Centroid, Spectral Roll-off, Spectral Flux,
Chroma Features, Harmonics-to-Noise Ratio, and Entropy of Energy. By incorporating
these features, we aim to enhance the discriminatory power of our classification system
and improve its ability to handle a diverse range of scratch patterns and complexities.
Furthermore, we will continue to investigate the potential of signal analysis for scratch
characterization, including the development of novel algorithms tailored to this unique
perspective. We believe that by treating scratches as signals, we can not only improve
classification accuracy but also gain deeper insights into the nature and behavior of
surface imperfections. This knowledge will have far-reaching implications for material
science, manufacturing, and quality assurance, ultimately leading to the development of
more durable and reliable materials.
References
[1] Hao Ding, Bo Xia, "YOLOv5s-DNF: A lighter and real-time method for detecting
surface defects in steel", 2023 4th International Conference on Computer Vision,
Image and Deep Learning (CVIDL), pp.564-569, 2023.
[2] Alaa Aldein M. S. Ibrahim, Jules-Raymond Tapamo, "A Survey of Vision-Based
Methods for Surface Defects’ Detection and Classification in Steel Products",
Informatics, vol.11, no.2, pp.25, 2024.
[3] Maruschak, P.; Konovalenko, I.; Osadtsa, Y.; Medvid, V.; Shovkun, O.; Baran, D.;
Kozbur, H.; Mykhailyshyn, R. Surface Illumination as a Factor Influencing the
Efficacy of Defect Recognition on a Rolled Metal Surface Using a Deep Neural
Network. Appl. Sci. 2024, 14, 2591. https://doi.org/10.3390/app14062591
[4] An Overview of Image Generation of Industrial Surface Defects. Xiaopin Zhong,
Junwei Zhu, Weixiang Liu, Chongxin Hu, Yuanlong Deng and Zongze Wu
Sensors 2023, 23(19), 8160; https://doi.org/10.3390/s23198160.
[5] Bhatt, P. M., Malhan, R. K., Rajendran, P., Shah, B. C., Thakar, S., Yoon, Y. J., and
Gupta, S. K. (February 9, 2021). "Image-Based Surface Defect Detection Using Deep
Learning: A Review." ASME. J. Comput. Inf. Sci. Eng. August 2021; 21(4):
040801. https://doi.org/10.1115/1.4049535.
[6] Zhang, J.; Li, Z.; Hao, R.; Wang, X.; Du, X.; Yan, B.; Ni, G.; Liu, J.; Liu, L.; Liu, Y.
Classification of Microscopic Laser Engraving Surface Defect Images Based on
Transfer Learning Method. Electronics 2021, 10, 1993.
https://doi.org/10.3390/electronics10161993
[7] Younes, M.A. Measurement and inspection of micro-features and micro-surface
scratches Alexandria University Alexandria Engineering Journal, Vol. 43 (2004),
No. 5, 545-552.
[8] Li, W., Zhang, L., Wu, C. et al. A new lightweight deep neural network for surface
scratch detection. Int J Adv Manuf Technol 2022, 123, 1999–2015.
https://doi.org/10.1007/s00170-022-10335-8
[9] Wu, J.; Liu, W.; Li, C.; Jiang, T.; Mohammad Shariful, I.; Yao, Y.; Sun, H.; Li, X.; Li, X.;
Huang, X. & Grzegorzek M. A state-of-the-art survey of U-Net in microscopic image
analysis: from simple usage to structure mortification. Neural Comput & Applic
2024, 36, 3317-3346. https://doi.org/10.1007/s00521-023-09284-4
[10] Zhang, J.; Li, Z.; Hao, R.; Wang, X.; Du, X.; Yan, B.; Ni, G.; Liu, J.; Liu, L.; Liu, Y.
Classification of Microscopic Laser Engraving Surface Defect Images Based on
Transfer Learning Method. Electronics 2021, 10, 1993. https://doi.org/10.3390/
electronics10161993
[11] Gao, W.; Wang, L.; Coffey, J.K.; Wu, H.; Daver, F. Finite Element Modelling
and Experimental Validation of Scratches on Textured Polymer
Surfaces. Polymers 2021, 13, 1022. https://doi.org/10.3390/polym13071022
[12] Konovalenko, I.; Maruschak, P.; Brevus, V. Steel Surface Defect Detection
Using an Ensemble of Deep Residual Neural Networks, Journal of Computing and
Information Science in Engineering, 2022, 22(1), 014501,
https://doi.org/10.1115/1.4051435
[13] Konovalenko, I.; Maruschak, P.; Kozbur, H.; Brezinová, J.; Brezina, J.;
Guzanová, A. Defectoscopic and Geometric Features of Defects That Occur in Sheet
Metal and Their Description Based on Statistical Analysis. Metals 2021, 11, 1851.
https://doi.org/10.3390/met11111851