Improving the quality of deepfake images is an important condition for their effective detection, since artifacts, noise, and blur can hide key signs of forgery. Optimizing visual quality allows you to preserve critical textural and structural features, which increases the accuracy of detection algorithms. OpenForensics was chosen as the dataset. The training sample was divided into training, testing, and validation samples. The sample size is 500 images In this paper, a web framework is proposed to improve the quality of images of human faces with the ability to generate dataset parameters, which allows for a universal approach to forming training parameters. Improving image quality will help to better detect deepfake images. The accuracy of improving image quality using neural networks is 91%.
1. Introduction
The aim of the research is to develop, implement and evaluate the effectiveness of a framework that allows you to improve the quality of deepfake images through adaptive reconstruction of structural and textural characteristics using convolutional neural networks.
To develop a comprehensive system, the following tasks must be implemented: - develop the main framework modules for generating dataset parameters and improving image quality;
- develop and implement a convolutional network architecture with recurrent connections to improve the quality of deepfake images;
- conduct research and analyze the results by metrics.
The influence of scientists on the development and analysis of image quality enhancement technologies using neural networks is fundamental and multifaceted. Thanks to scientific research, this field has rapidly evolved from simple filtering methods to complex deep architectures capable of restoring images with extreme accuracy. In work [1], to achieve the goal of the study, a dataset of 1063 ultrasound images was created by artificially degrading the quality of the original high-quality images. The images were evaluated by doctors, and the obtained scores were averaged to form labels. After data cleaning, 478 images were selected for training and testing models. To build a quality assessment system, a deep convolutional neural network and a residual network were used, and a transfer learning strategy was also applied, which allowed to improve learning under conditions of limited data volume. The test results demonstrated that the proposed CNN approach is effective and suitable for quantitative assessment of the quality of ultrasound images.
In [
The study [
The paper [
The datasets selected are OpenForensics [
The training sample was divided into training, testing, and validation samples. The sample size is 500 images.
One of the key aspects is the process of generating a dataset, so our task was to develop an approach for generating this dataset, for example images.
We use the JSON format as a way to store parameters and transfer data over the Internet via a separate API. JSON is a common format for transferring data over the network between individual servers.
The main page consists of a block displaying all studies (i.e. datasets) and a button for creating a new dataset (Figure 2). The main elements are: - Path to the folder with images; - Image height; - Image width; - Image type; - Crop; - Single image selection;
The path to the image provides the ability to enter the URL to the folder with files or to the archive with files
The height and width of the images is an important aspect, which is one of the key when choosing the architecture of the neural network, so bringing all the images in the sample to the same format allows classification without errors and exceptions.
The "Crop" operation is often used when it is necessary to divide a large image into smaller portions according to the requirements of the neural network architecture and the capabilities of the execution environment (computer, cloud storage).
The software module is implemented using the PHP programming language and the Laravel framework/
The module is based on the use of the MVC pattern, which allows you to clearly separate the controller, model and view of the project.
A feature of this project is the presence of the FaceDetector class, which is responsible for processing and highlighting faces in the image, for example, for the task of searching for deepfakes
The config.get method has been developed, which allows you to get specific parameter values. This approach also simplifies the process of training a neural network and retraining by different scientists, since in this case the training sample will be of the same type, which simplifies the setup process.
Web-based frameworks are structured software platforms that provide a set of tools, libraries, and templates to facilitate the development of web applications. They define a standard approach to building applications, reduce the amount of routine code, and facilitate rapid and unified creation of functionality. The structure of the modules of a web-based deepfake detection framework is shown in Figure 4.
Input image processing module is responsible for processing the input image, including specifying the path to the image folder and other parameters.
Determining input image parameters is intended for determining and storing the input image parameters in the database.
Architecture and CNN model storage module is intended for storing the convolutional neural network architecture configuration files and the models themselves in the form of files. Then you can use ready-made models for a specific type of task.
Dataset parameter generation module is implemented as a web page with fields for specifying dataset parameters, implemented using the Laravel framework.
Automatic image quality improvement module is responsible for directly improving the quality of images using convolutional neural networks.
Face detection module is intended for highlighting faces in images, which allows you to select only the necessary area in the image and reduce the number of calculations.
Thanks to frameworks, web development becomes more productive, safer and organized, which makes them an integral part of modern software engineering.
To generate possible poor-quality images, blurring, shadow correction, and salt-and-pepper noise algorithms were used. An example of a noisy image is shown in Figure 5. Noise is a key tool in improving the robustness of computer vision models. Different types of noise affect different aspects of an image geometry, color, brightness, or texture. Noise analysis and modeling allow us to assess the robustness of algorithms, improve the quality of preprocessing, and build more adaptive systems.
Image enhancement is one of the key tasks in computer vision, covering detail restoration, noise removal, resolution enhancement, and artifact correction.
Convolutional neural networks (CNNs) have proven to be a powerful tool for solving these tasks due to their ability to automatically extract and generalize relevant spatial features. Over the past decade, a number of architectures have been proposed that are specifically adapted to processing low-quality or corrupted images, including autoencoders, U-Nets, Residual Networks, GAN-based solutions, etc. Convolutional neural networks fundamentally differ from traditional (classical) image processing algorithms in terms of their approach to data analysis, flexibility, learning ability, and adaptation to complex tasks. Traditional algorithms require manual tuning of filters and features (e.g., edge detectors, gradient histograms, color histograms, etc.), while CNNs learn to extract relevant features from data by training on a large number of images.
The proposed architecture of the convolutional neural network is shown in Figure 6. Total params: 134,067 Trainable params: 134,067 Non-trainable params: 0
The image sample is divided into two parts: test and training. A feature of Unet networks and convolutional networks in general is the presence of repeated blocks. Therefore, we will present them as a set of operations: = {〈 〉; 〈 〉; 〈 〉; 〈 〉} where 〈C〉-set of convolution functions 〈B〉-batch normalization operation; 〈A〉-set of activation functions 〈Add〉-set of layers of the recurrent block Let us consider in more detail the key blocks separately: Convolution block
Formally, the convolution process can be represented as follows: 2 ∑
2 =− 2 =− 2 , × ℎ =
∑ ( − , − )∙ ℎ( , ) GELU allows small negative values when the input is less than zero, providing a richer gradient for backpropagation. (1) (2) (3) where 2 is half the filter height, 2 is half the filter length, x is the pixel column position, y is the pixel row position, , is the input image, h is the convolution kernel.
GELU Activation Function Formula for finding the activation function; ( )= 0.5 (1 + ℎ (√ ( + 0.044715 3))) 2
The performance assessment of convolutional neural networks (CNN), particularly in classification tasks, is based on quantitative metrics that reflect the accuracy of the model's decision-making.
Training parameters: Number of epochs: 45 steps_per_epoch=3 Accuracy The Accuracy metric is the basic metric for balanced classes.
Accuracy =
TP + TN TP + TN + FP + FN (4) where TP is true positive, TN is true negative, FP is false positive, FN is false negative.
Accuracy is one of the simplest and most commonly used metrics for evaluating the quality of a classification model. It shows what proportion of predictions the model made were correct. The value of Accuracy ranges from 0 to 1 (or from 0% to 100%). The network performance estimates based on the loss criterion are shown in Figure 8.
An important element for integrating deep learning methods into cloud services is the use of
MlOps [
Loss is a key metric in the training process of neural networks, measuring how much the model's predictions differ from the true labels (correct answers). Unlike metrics such as accuracy or precision, which are used to evaluate the finished model, loss is used during training to update the network's weights.
Based on the analytical approach and principles of developing software modules based on web systems, the main framework modules have been developed to generate dataset parameters and improve image quality.
Artificially noisy images have been generated for experiments. A convolutional network architecture with recurrent connections has been implemented to improve the quality of deepfake images.
The accuracy of improving image quality using a neural network is 91%.
The authors have not employed any Generative AI tools. based medical ultrasound image quality assessment." Complexity 2021, no. 1 (2021): 9938367.