Agriculture faces major challenges in sustainably feeding a growing global population, making accurate crop yield estimation essential for informed decision-making by farmers. While traditional methods such as field surveys can be helpful, they are often limited by issues of accuracy, cost, and time eficiency.
Tomato (Solanum lycopersicum) is a crucial vegetable
crop globally, boasting 183 million tonnes in 2018 [
Several approaches have been investigated in the
literature to address the challenge of fruit weight estimation.
For instance, Yamamoto et al. [
Van Daalen et al. [
Similarly, Lee et al. [
In another study, Nyalala et al. [
Basak et al. [
ceur-ws.org late pixel numbers. Linear regression (LR) and non-linear SVR models were applied, resulting in training and testing accuracies of 96.3% and 89.6%, respectively.
This study focuses on applying recent advancements in computer vision, particularly object detection, and machine learning algorithms to estimate tomato weight from realworld images. The subsequent sections describe the equipment used, the structure and composition of the dataset, and the methodology employed to generate accurate quantitative measures such as projected surface area and total weight for detected fruits. Our findings demonstrate the efectiveness of this approach. Additionally, we discuss the challenges faced and propose recommendations for future research.
The data used in this study consists of tomato fruit images collected both online and in the field under real-world conditions. The dataset includes a total of 180 images obtained online and 100 images taken in the field, containing a total of 1143 tomato fruit instances. Table 1 illustrates the composition of our dataset.
Images captured in the field helped to collect additional information such as actual fruit area and actual fruit weight, which enriches the dataset by providing accurate and relevant measurements for tomato fruit weight estimation. Table 2 presents additional insights concerning field-captured images. Upon analysis of the table, the average fruit weight is 35.30 g , with a standard deviation of 14.56 g . The average true area is 2673.48 mm2, with a standard deviation of 873.68 mm2. Quartile values provide insights into the distribution of the data. Thus, 25% of the fruits have a weight of less than 25.21 g, 50% have a weight of less than 37.00 g , and 75% have a weight of less than 43.49 g. For the actual surface area, 25% of fruits have an area less than 2, 024.93 mm2, 50% have an area less than 2, 779.53 mm2, and 75% have an area less than 3, 219.12 mm2.
To estimate tomato fruit weights, we developed a four steps approach (see figure 1)
To train our segmentation model, we prepared a dataset of tomato images, labeled in the COCO format. The dataset consisted of 180 images containing 1043 instances of tomatoes, sourced from both the internet and field photography, and annotated using the ROboflow platform. We employed the Mask R-CNN instance segmentation model through the Detectron2 framework, selecting the mask_rcnn_R_50_FPN_3x configuration developed by Facebook AI Research. This model, pre-trained on the COCO dataset, combines the Mask R-CNN architecture with a ResNet-50 backbone and Feature Pyramid Network (FPN) for high-performance, multi-scale object detection. To evaluate the projected area of each tomato from images, a dataset was constructed, including individual images of tomatoes, their actual weight in grams, the total number of pixels in the image, the number of pixels corresponding tomato (obtained by semantic segmentation), and the total area of the image in square meters, obtained by camera calibration.
The estimation of the projected area took place in two steps: first, the segmentation mask allows us to calculate the area in pixels occupied by the tomato in the image. Then, a camera calibration converted this pixel area into an actual metric area, using a coin as a reference object. By photographing the tomatoes under the same conditions as the reference piece, the resulting conversion factor was used to convert the pixel area of each fruit into a measure of its actual projected area in metric units. This method uses a rule of three, where the actual surface area of the tomato ( ) is estimated based on the number of pixels corresponding to the tomato in the image ( ), using the conversion factor established during calibration: . = × (1)
With this method, we were able to estimate the real surface area of each tomato in physical space from segmentation in image space, thanks to precise calibration using a reference object.
To estimate the weight of the tomatoes based on their projected surface area, we tested several regression models, including Simple Linear Regression (SLR), Multiple Linear Regression (MLR), and Partial Least Squares Regression (PLSR). These models aimed to establish a mathematical relationship between the surface area (independent variable) and the weight (dependent variable) of the tomatoes. The performance of each model was evaluated on a validation set consisting of 20% of the total dataset, collected under real-world conditions. Standard metrics, such as Root Mean Square Error (RMSE) and the Coeficient of Determination ( 2), were employed to assess model accuracy. We also applied 10 -fold cross-validation to each model to reduce the likelihood of overfitting.
Figure 1 depicts the summary of the methodology adopted in this study.
to enhance performance.
The projected surface area of each fruit was derived from the segmented mask by calculating the pixel area, then converting it to real-world units using camera calibration information as defined in Equation 1. This method achieved a precision of approximately 95.
For tomato weight estimation, a subset of the dataset containing real-world images was used, which included precise data on both the actual weight of each tomato and their projected surface area. A mathematical relationship between the weight and projected area was established through the evaluation of several regression methods. The algorithms tested included Least Squares Regression (LSR), Multiple Linear Regression (MLR), and Support Vector Machines (SVM), and their performance was compared using cross-validation and Mean Square Error (MSE) as the evaluation metric.
Table 4 highlights the performance metrics of the tested models.
Among the evaluated models, Lasso Regression achieved the best performance, with a MAE of 5,776 and an MSE of 62.99.
The corresponding model equation is:
The study employed a multi-step methodology to estimate
tomato fruit weights from images. First, a Mask R-CNN
model, using the mask_rcnn_R_50_FPN_3x configuration,
was trained on a dataset of 180 images containing 1043
tomato instances. After detection and segmentation, the
projected surface area of each tomato was estimated
using a calibrated conversion from pixel area to metric units,
achieving approximately 95% accuracy. For weight
estimation, several regression models were evaluated on a subset
of real-world images with known weights and projected
areas. Among the regression models evaluated, the Lasso
Regression algorithm demonstrated superior performance
in estimating tomato weights. This model achieved a Mean
Absolute Error (MAE) of 5.776 grams and a Mean Squared
Error (MSE) of 62.99 grams2̂. Our model outperformed the
approach described by Lee et al. [
When applied to the test dataset, this model achieved a relative error of 7.09% in estimating the total weight of tomatoes. These results demonstrate the potential of this combined approach for automated tomato yield estimation, although the ideal conditions of the study (fully visible fruits) suggest that further research is needed to address real-world challenges such as occlusion.
While this study yielded promising results, it’s important to acknowledge its primary limitation: the experiments were conducted under idealized conditions that do not fully represent real-world agricultural environments. All tomatoes in the study were fully visible and unobstructed, which rarely occurs in actual fields where fruits are often partially hidden by leaves, branches, or other fruits. This idealization may lead to overly optimistic performance estimates.
To bridge this gap and enhance the model’s practical applicability, future research will focus on developing robust occlusion handling techniques, such as implementing advanced image processing algorithms for reconstructing partially obscured fruits or using ellipse fitting methods to estimate the full shape of partially visible tomatoes.
Additionally, creating more representative datasets that reflect the challenging conditions found in real agricultural settings, including various levels of occlusion and diverse growth stages, will be crucial. By addressing these limitations and training on more diverse and challenging datasets, future iterations of this system could significantly improve in accuracy and robustness, making it a more reliable tool for automated agricultural yield estimation in real-world scenarios.
This study successfully introduced an innovative approach for accurately assessing tomato crop yields through the use of advanced image processing, computer vision, and artificial intelligence techniques. The results align closely with the objectives of estimating both the quantity and total weight of fruits, highlighting the practical benefits of this methodology for farmers.
Looking ahead, future enhancements will focus on reifning the approach by integrating multispectral imaging to improve data acquisition. Additionally, algorithmic advancements, including image generation and ellipse fitting techniques, will be employed to tackle challenges related to occlusion. These developments will enhance the model’s scalability and robustness, facilitating large-scale deployment in real-world agricultural settings. The anticipated implementation of this approach in automated systems that utilize drones and ground-based robots presents exciting opportunities for digital agriculture, paving the way for precise, eficient, and automated yield estimation.