Introduction

In world practice, computer vision technologies are widely used to process remote sensing images. To identify objects in remote sensing images, it is necessary to solve the tasks of detecting, recognizing, assigning accurate bounding boxes or masks for small, randomly oriented objects, separating them from the background, and providing object class labels [1,2].

Currently, a large number of models based on convolutional neural networks have been developed to improve the accuracy of object detection and recognition. In the process of recognizing and locating an object, the neural network model uses a rectangular bounding box to detect it, and then classifies and distinguishes between the object itself or the background within it [3,4]. Most cases of object detection from a perspective parallel to the Earth are parallel to the image coordinate axis with a small aspect ratio. As a result, a rectangular bounding box can better cover objects and contain less background [5]. However, in the case of observing military objects with a large aspect ratio and disordered direction in images acquired remotely from an observation angle perpendicular to the Earth [6], it is not possible to accurately surround the object with a rectangular bounding box alone [7]. In the field of computer image processing, a detector is a model for detecting and recognizing objects. To solve the problem of detecting simple objects, one-stage and two-stage detectors are used. One-stage detectors include: SSD, YOLO, RetinaNet, R³Det, RSDet, RIDet, FCOS, CSL, DCL, GWD, KLD, KFioU, and two-stage detectors include Fast R-CNN, Faster R-CNN, Mask R-CNN, Cascade R-CNN, RRPN, R²CNN, SCRDet, SCRDet++ [8].

Classical object detection is the detection of a simple object in an image using a horizontal bounding box. Nowadays, many high-performance methods for detecting simple objects, such as the two-stage model described by Fast R-CNN [9] and Faster R-CNN [10], focus on accuracy and reduce the amount of computation to improve detection speed. To solve the problem of changing the scale of an object in an image, the pyramidal feature network (FPN) method was proposed.

Since most approaches are based on the assumption that objects are located along horizontal lines in the image, the detector uses a rectangular bounding box parallel to the coordinate axis to detect and locate the object in the image. Then it classifies the object or background directly within this frame [3,4]. As a result, the task of detecting randomly rotated objects with a large aspect ratio arises, which increases the bounding box and, as a result, leads to overloading of the detector during classification, and in the case of detecting randomly rotated, densely spaced objects, the overlapping frames process complex scenes and make it difficult to distinguish a single object [11] (figure 1). To solve the problem of detecting randomly oriented objects, approaches based on the rotation of a rectangular bounding box or other polygonal bounding boxes are used. For example, the Rotation Region Proposal Network (RRPN) [12] obtains a region of interest based on the rotated anchor for feature detection. The Rotational Region CNN (R2CNN) [13] is based on the Fast R-CNN, using two types of pooling size with different width-to-height ratios. However, newly developed models using the approach of two-stage detectors based on traditional horizontal region detection do not produce results with the required speed and accuracy.

The CornerNet [14], CenterNet [15], and ExtremeNet methods have gained popularity, which select and group a set of certain key points of an object, such as corners, peaks, etc., to build a bounding box.

Single-stage detection methods (single-frame multi-box SSD detector [16], YOLO family of models, and RetinaNet [17]) are based on bounding box regression. YOLOv11 [18] is the most advanced model that supports all the previous ones, and is improved by a new backbone network, detection unit, and loss function [19].

Their advantage is the higher speed of object detection and recognition. The disadvantage of the considered approaches is that they do not take into account the cases of complex scenarios on satellite images when it is necessary to detect small, densely located, randomly rotated objects, the detection of which remains relevant under such conditions.

The YOLOv11 object detection system is a single-stage system, but its accuracy is higher than most two-stage detectors, and it is also fast. Therefore, in this paper, we use YOLOv11, on the basis of which we implement the detection of randomly rotated objects. The purpose of the article is the analysis of neural network models and their improvement as a tool for improving the accuracy of detecting and recognizing small, arbitrarily rotated objects on satellite images.

Theoretical background

The considered detectors for detecting objects in images use a rectangular bounding box parallel to the coordinate axes, which, when detecting randomly rotated objects with a large aspect ratio, increases the bounding box and, as a result, leads to overloading of the detector in the case of classification. In addition, it does not provide accurate information about the object's orientation and scale.

To implement the detection of objects with randomly orientation, each detector and dataset provides its own definition of the rotation angle. The DOTA dataset [20] stores the coordinates of the four corners of the object's bounding box. R2CNN [21] uses the coordinates of the first two clockwise corners of the four (𝑥 1 , 𝑦 1 ; 𝑥 2 , 𝑦 2 ) and the height of the rectangle to define the frame. A common method is five-parameter regression, which adds an angle 𝜃 parameter in addition to the basic parameters𝑥𝑦 and 𝑤ℎ, to represent the bounding box in any direction. As shown in figure 2) (left), this is an acute angle formed by the width (or height) of the bounding box and the axis 𝑥 in the range of 0 − 90 ∘ . Another method is that the angle formed between the longest side of the rectangle and the axis 𝑥 is between −90 ∘ and +90 ∘ , as shown in figure 2) (right). In the proposed model, the image label is pre-processed, in which the processing mainly concerns the label part, and the angle information is obtained from the spatial label 𝑥𝑦𝑤ℎ of the object. Data preprocessing allows us to obtain the object's orientation angle from −90 ∘ to +90 ∘ and the division into width 𝑤 and height ℎ.

The architecture of the YOLOv11 model, which is designed for improving small object detection and accuracy while maintaining real-time inference speed, is shown in figure 4. The network consists of the following parts: input, main, and prediction. Some modules are omitted in the figure and only the general structure is shown. The input part is used to extract features from the image, from which three feature maps are sequentially extracted, which pass through the main part, where a number of operations are performed on them, such as convolution (*), upsampling (↑) and combining ( ⨁︀ ). The convolution (*) consists of a 2D convolutional layer and a 2D batch normalization layer with SiLU activation function [22]. YOLOv11 uses C3K2 blocks to handle feature extraction at different processing stages. The C3K2 block optimizes information processing by dividing the feature map and applying a series of smaller kernel convolutions 3 × 3, which are faster and cheaper to compute compared to large kernel convolutions. It consists of convolution blocks at the beginning and end, followed by a series of convolution blocks with interval pooling that disregards residuals when negative, and ends with a pooling and simple convolution block.

A special feature of YOLOv11 is the use of fast spatial pyramid fusion (SPFF), which was developed to combine features from different regions of the image at different scales. To merge features, SPFF uses multiple maximal pooling operations (with different kernel sizes) to aggregate multi-scale contextual information. This improves the processing of fine-grained objects in images.

One of the significant innovations in YOLOv11 is the addition of the Cross Stage Partial with Spatial Attention (C2PSA) block. This block introduces attention mechanisms that improve the model's focus on important areas of the image, such as smaller or partially covered objects, by emphasizing spatial relevance in feature maps.

Prediction produces detection blocks for three different scales (low, medium, high) using the feature maps created by the previous processing steps. This approach ensures that small objects are detected in greater detail while larger objects are captured by higher-level features.

As a result of processing, the neural network produces three predictions for the scales 80 × 80, 40 × 40, 20 × 20. The format of the predicted object label for all scales is provided in the following format: 𝑐𝑙𝑠 -general label category and five parameters of the bounding box (𝑥, 𝑦 -coordinates of the lower left corner; 𝑤, ℎ -width and height; 𝜃 -angle of inclination to the 𝑥-axis).

In the proposed five-parameter model (𝑥, 𝑦, 𝑤, ℎ, 𝜃), regression is used to predict the rotation of the object bounding box, since weapons and military equipment samples on satellite images have a fixed aspect ratio, and the direction parallel to the longer side is defined as the direction of the object's movement. Therefore, to facilitate the regression task, the longer side is defined as 𝑤, and the shorter side is defined as ℎ. Thus, the direction parallel to is the direction of motion of the object. The angle between the longer side 𝑤 and the axis 𝑥 is the angle of rotation. Given that the required range of angles is [−90 ∘ , 90 ∘ ], the function 𝑎𝑟𝑐𝑠𝑖𝑛 is chosen to calculate the angle 𝜃. The rotation angle is calculated using the expression (figure 4):

𝜃 = arcsin[(𝑦 𝑥(𝑚𝑖𝑛) − 𝑦 𝑥(𝑚𝑎𝑥) )/𝑤],(1)

The two points on the longest side 𝑤, 𝑦 𝑥(𝑚𝑖𝑛) , describe the value of the point on the axis 𝑦 with the smaller value on the axis 𝑥, 𝑦 𝑥(𝑚𝑎𝑥) , describing the opposite.

To accurately determine the angles, it is also necessary to perform a conversion between the fiveparameter method and the four-point annotation 𝑥 1 𝑦 1 , 𝑥 2 𝑦 2 , 𝑥 3 𝑦 3 , 𝑥 4 𝑦 4 . The YOLOv11 model module, which performs data preprocessing, affine and color transformations of the image, receives four points of the object's corners as input. When recalculated, the final result of target detection is the coordinates of the four corner points with a rotating bounding box applied to the original image. An example of coordinate calculation for the coordinate 𝑥 𝑖 :

𝐹 𝑥 𝑖 = (−1) 𝐿(𝑂𝑥 𝑖 ,𝐶𝑥) 𝑤𝑐𝑜𝑠𝜃 − (−1) 𝐿(𝑂𝑦 𝑖 𝐶𝑦) ℎ𝑠𝑖𝑛𝜃 2 + 𝐶𝑥,(2)𝐿(𝑜𝑥, 𝐶𝑥) = {︂ 0 𝑂𝑥 > 𝐶𝑥 1 𝑂𝑥 < 𝐶𝑥 ,(3)

where 𝐹 𝑥 𝑖 is the final value of the point after the transformation; 𝐿(𝑂𝑥 𝑖 , 𝐶 𝑥 ) -is the relative value of the location of the initial corner point 𝑂𝑥 and the center point 𝐶𝑥.

The angle value is added to solve the problem of regressing the object's direction of rotation. For example, in a neural network, when processing an input image with 80 object detection and recognition categories and using the four-parameter method (𝑥, 𝑦, 𝑤, ℎ) to locate the target, the final output matrix is 𝐹 × 𝐹 × (80 + 4 + 1). 𝐹 provides the dimension of the feature map output by the last prediction layer, and is the probability that a certain pixel in the feature map is the center point of the object; the main part module is located between the two layers mentioned above and provides some modules such as FPN. Therefore, to use the five-parameter positioning method, an additional channel is added to the main part to predict the angle value (figure 4).

When using the five-parameter positioning method, the center point of the object in the classification and positioning prediction matrix in the original layer of the feature map is placed in a rectangular coordinate system. As a result, during the training of the neural network model, a significant distance between the training sample and the object prediction can lead to large values of the loss function, which will not contribute to the convergence of the neural network model.

Therefore, first, the cell of the coordinate grid where the label is placed is determined. Its upper left corner is the origin. Subsequently, the coordinates 𝑥𝑦 are calculated as the offset of 𝑡 The result given by the prediction part of the neural network model cannot be directly calculated for the loss function. To limit it within a given range, we use coordinate regression functions 𝑏 𝑥 , 𝑏 𝑦 , -[−0, 5; 0, 5]:

𝑏 𝑥 = 2 1 + 𝑒 −𝑡 − 0, 5 + 𝐶 𝑥 ,(4)

where 𝑏 𝑥 is the actual position of the center point of the predicted bounding box; 𝑡 𝑥 is the output value of the neural network model after calculation; 𝐶 𝑥 is the value of the grid origin; angle of inclination 𝑏 𝜃 -[−1, 5; 1, 5] -(calculated in radians):

𝑏 𝜃 = 3 1 + 𝑒 −𝜃 − 1, 5,(5)

The loss functions 𝐿 for training the neural network model for positioning and orienting the bounding box are:

𝐿 = 𝐿 𝑐𝑖𝑜𝑢 + 𝐿 𝑎𝑛𝑔𝑙𝑒 ,(6)

where the loss functions 𝐿 𝑐𝑖𝑜𝑢 are for calculating the size and location of the center, and 𝐿 𝑎𝑛𝑔𝑙𝑒 -for calculating the angle of rotation.

The function 𝐿 𝑐𝑖𝑜𝑢 [20] (figure 5) works with the width 𝑤, height ℎ, distances 𝑑 between the two center points of the bounding boxes and 𝑐 -between the outer corners of their union. In figure 5, the bounding box of the training sample ̃︀ 𝐵 is marked with a solid line, the predicted one 𝐵 with a dashed line, the intersection 𝐼( ̃︀ 𝐵, 𝐵) with a dashed line, and the union 𝑈 ( ̃︀ 𝐵, 𝐵) with a dotted line. The full loss function 𝐿 𝑐 𝑖𝑜𝑢 can be described as follows:

𝐿 𝑐 𝑖𝑜𝑢 = 1 − 𝐼𝑂𝑈 + 𝜌 2 ( ̃︀ 𝑏, 𝑏) 𝑐 2 + 𝛼𝜈,(7)

where 𝜌 is the Euclidean distance between the center points ̃︀ 𝑏, 𝑏 of the bounding boxes ̃︀ 𝐵 and 𝐵, 𝑐 is the minimum diagonal distance of their union, 𝛼 and 𝜈 is the penalty of the loss function for the distance between the center points and the aspect ratio of the bounding boxes.

The components of 𝐿 𝑐 𝑖𝑜𝑢 take into account the following: 𝐼𝑂𝑈 calculates the intersection area over the union of the training sample bounding box and the object prediction:

𝐼𝑂𝑈 = 𝐼( ̃︀ 𝐵, 𝐵) 𝑈 ( ̃︀ 𝐵, 𝐵) ,(8)

𝛼 takes into account the aspect ratio:

𝛼 = 𝜈 (1 − 𝐼𝑂𝑈 ) + 𝜈 ,(9)

𝜈 is used to measure the consistency of the aspect ratio:

𝜈 = 4 𝜋 2 (arctan ̃︀ 𝑤 ︀ ℎ − arctan 𝑤 ℎ ) 2 ,(10)

The rotation angle is calculated by the individual losses of 𝑆𝑚𝑜𝑜𝑡ℎ𝐿1:

𝐿 𝑆𝑚𝑜𝑜𝑡ℎ𝐿1 = ⎧ ⎨ ⎩ 0.5( ̃︀ 𝜃 − 𝜃) 2 ⃒ ⃒ ⃒ ̃︀ 𝜃 − 𝜃 ⃒ ⃒ ⃒ < 1 ⃒ ⃒ ⃒ ̃︀ 𝜃 − 𝜃 ⃒ ⃒ ⃒ − 0.5 ⃒ ⃒ ⃒ ̃︀ 𝜃 − 𝜃 ⃒ ⃒ ⃒ ≥ 1 ,(11)

where ̃︀ 𝜃 -rotation angle of the training sample ̃︀ 𝐵; 𝜃 -angle according to the forecast. Backpropagation will gradually reduce the training losses of the neural network model to achieve the expected object detection result.

Experimental results

To evaluate the results of the proposed rotating detector model, comparative experiments were conducted on the DOTA reference dataset. The DOTA images were collected from Google Earth, GF-2 and JL-1 satellite remote sensing data provided by the China Satellite Data Resource and Application Center, and aerial photographs from CycloMedia. DOTA consists of RGB and grayscale images. RGB images are taken from Google Earth and CycloMedia, and grayscale images are taken from the panchromatic range of GF-2 and JL-1 satellite images. All images are saved in png format. The dataset contains 11268 remote sensing images (whose sizes vary from 800 × 800 to 20000 × 20000 pixels) with 1793658 instances, which are divided into 18 categories. Dataset composition: 4622 images with 621973 instances are the training set; 593 images with 81048 instances are the validation set; 6053 images with 1090637 instances are the test set. Each instance is labeled as a rectangle with clockwise dots. 𝑥 1 𝑦 1 , 𝑥 2 𝑦 2 , 𝑥 3 𝑦 3 , 𝑥 4 𝑦 4 Half of the images in this set were used as a training set, one third as a test set, and one sixth as a validation set.

To evaluate the performance of the model, we used the mean accuracy metric (mAP), which calculates the average of the mAP scores for the variable IoU values. It allows penalizing a large number of bounding boxes with incorrect classifications to avoid over-specialization in a few classes at the expense of weak overfitting in others.

The model was trained for 120 epochs with a learning rate of 0.01 and a momentum of 0.937. To finalize the model, 3 TTAs were applied (minor image slicing with 650 × 650, 750 × 750, 850 × 850, and rotation (0 ∘ , 90 ∘ , 180 ∘ , 270 ∘ ). To take into account the location of the image in the image (to reduce the influence of objects with larger curved features at the edge of the image), we reduced the probability by a correction factor of 0.8.

As a result of tuning the developed model, along with increasing the image set and post-processing, the accuracy of mAP object detection and recognition was improved by 0.33%, which is 81.69 compared to YOLOv11-obb.