This study explores the application of deep learning architectures for image classification and segmentation in poultry farms with overlapping objects. Early detection of fallen birds is crucial for preventing disease outbreaks and maintaining animal welfare. We investigate the efficacy of various architectures, including U-Net, mU-Net, SegNet, and O-Net, for segmenting live and dead birds within poultry farm real time images. Our experiments, conducted on a dataset of 1805 images with varying lighting, distances, and object numbers, reveal that U-Net achieves the highest Dice coefficient (0.95128) for segmentation accuracy. We further demonstrate the potential of these models for classifying individual birds as alive or dead, with U-Net reaching a classification accuracy of 88.938%. The findings suggest that AI-powered image segmentation holds promise for enhancing poultry farm management by enabling early detection of deceased birds and fostering improved animal health and welfare.
Having sufficient production capacity to manufacture high-quality and safe products stands as a crucial element in ensuring the effective functioning of the poultry industry. Upholding the sector's efficiency entails not only maintaining favorable conditions for poultry farming that align with animal welfare standards but also overseeing the technological aspects of production and proactively addressing health concerns among poultry to prevent losses during the initial production stages. The concept of sustainable production has garnered significant attention of late, prompting analyses of the environmental repercussions of poultry meat production and the advancement of production techniques in accordance with the European Green Deal strategy and strategic directives from the Food and Agriculture Organization and the European Feed Manufacturers’ Federation. These directives prioritize efforts aimed at mitigating odor dispersal and greenhouse gas emissions, particularly targeting harmful gases like ammonia (NH3) and hydrogen sulfide (H2S), along with greenhouse gases such as CO2, CH4, and N2O. Furthermore, poultry farming contributes to environmental contamination through the emission of volatile organic compounds (VOCs), which constitute another category of pollutants. The organic compounds released during poultry meat production further compound environmental pollution by leaving residues of both macro and trace elements. The health of the flock can be indicated by the death rate of individuals and early detection of dead birds can prevent further spread of diseases. The advancements in the AI-driven technologies can help detect and remove fallen birds in the first moments after death.
The adoption and integration of AI technology within the agricultural domain are experiencing
rapid growth, driven by the escalating global population and the consequent rise in food demand.
Various factors including climate change, population growth, increased food consumption, and
employment challenges have fueled this trend [
Overlapping object segmentation is a crucial task for detecting anomalies in bird health in poultry farms. It helps to detect fallen birds in the early moments after their death in order to prevent further spread of disease to other hens. Several studies are performed trying to solve overlapping object problem, however, there is lack of study in the field of poultry farming. Different studies use deep learning, algorithm based or combined approaches.
The authors of [
Similarly, [12] authors uses CNN base approach for overlapping object segmentation. They proposed modified mask R-CNN structure [13] and tested it for an apple picking robots [12]. The proposed architecture uses ResNet [14] combined with DenseNet [15] instead of the typical mask RCNN backbone [12]. According to the authors this helps get better results while using less parameters [12]. This approach helped to reach 94.59% precision rate for overlapping fruits [12].
For chicken image segmentation [16] authors proposed a multi-scale attention based neural network (MSAnet), which reaches 94.6% accuracy, that is 2.6% better than U-Net using their collected dataset. MSAnet consist of four main parts: encoder-decoder, multi-scale module for image pyramid construction, double attention module and combined loss [16]. This model is trained using 330 annotated images, from which 60% - for training and others – testing [16].
Image segmentation stands as a crucial technique employed to distinguish and categorize individual objects within an image by allocating each pixel to a specific class. Initially, prevalent segmentation methods included thresholding, histogram-based clustering, and k-means clustering. However, over time, numerous advanced deep learning algorithms have emerged, significantly enhancing the efficacy of this process.
An exemplary illustration is the U-Net, initially crafted for medical image segmentation, marking one of the pioneering deep learning models tailored explicitly for segmentation endeavors [17, 18, 19]. Furthermore, the U-Net framework finds extensive utility across various iterations of Generative Adversarial Networks (GANs), such as the Pix2Pix generator [17, 18, 19]. The model's architecture is relatively uncomplicated, featuring an encoder tasked with downsampling and a decoder assigned with upsampling duties [17, 18, 19]. Moreover, the incorporation of skip connections serves to bolster the structural integrity of the model [17, 18, 19]. Similar to the original U-Net model, the modified U-Net framework (mU-Net) comprises two consecutive paths for encoding and decoding [20]. The encoder path, similar to the original U-Net, captures contextual features, while the symmetric and expanding decoder path facilitates segment localization [20]. However, mU-Net incorporates a Pix2Pix block [21]. Initially employed in generative adversarial networks for its ability to generate high-quality images across various image translation tasks, Pix2Pix has since become widely utilized as an upsampling block in a diverse array of applications [21]. In the final stage of the mU-Net model, a simple 2D convolution layer is utilized to map the extracted 64 features to the desired segmentation classes [20, 22, 23].
SegNet architecture is specifically engineered to excel in pixel-wise semantic segmentation tasks with a focus on efficiency [24, 25, 26]. It is primarily tailored for applications involving road scene understanding, where it is essential to model appearance (e.g., road, buildings), shape (e.g., cars, pedestrians), and grasp spatial relationships (context) among various classes like road and sidewalk [24, 25, 26]. In typical road scenes, the bulk of pixels pertain to major classes such as road and buildings, necessitating the generation of smooth segmentations [24, 25, 26]. The encoder network in SegNet closely resembles the convolutional layers found in VGG16 [24, 25, 26]. By eliminating the fully connected layers of VGG16, the SegNet encoder network becomes notably more compact and easier to train compared to many recent architectures [24, 25, 26]. A pivotal element of SegNet lies in its decoder network, comprising a hierarchy of decoders, each corresponding to an encoder [24, 25, 26]. These decoders utilize max-pooling indices received from their respective encoders to conduct non-linear upsampling of input feature maps, facilitating the segmentation process [24, 25, 26].
Drawing upon the strengths of Convolutional Neural Networks (CNNs) and transformers, the ONet architecture was proposed, aiming to integrate both architectures for the acquisition of global and local contextual features [27, 28, 29, 30]. In this architecture a CNN and a Swin Transformer as the encoder are merged, followed by routing them into separate decoders: one CNN-based and one Swin Transformer-based [27, 28, 29, 30]. Subsequently, the outcomes from both decoders are merged to yield the ultimate result [27, 28, 29, 30]. By leveraging the benefits of both CNNs and transformers, this network holds the potential to enhance the efficacy of medical image segmentation tasks [27, 28, 29, 30]. 3.2.
To evaluate the proposed method and compare different approaches following metrics were chosen: Accuracy (ACC) and The Dice coefficient (Dice).
Accuracy (1) is the measure of how well a model correctly predicts outcomes. It is often defined as the ratio of the number of correct predictions to the total number of predictions made. In image segmentation the accuracy is used pixelwise.
Number of correct predictions ACC = × 100 % ,
Number of total predictions
The Dice coefficient (2) bears a close resemblance to the Intersection over Union (IoU) metric. Nonetheless, it is computed by doubling the intersection of two sets and then dividing it by the sum of their sizes. The Dice coefficient spans from 0 to 1, with a score of 1 signifying a flawless overlap or segmentation alignment, whereas a score of 0 indicates no overlap whatsoever.
Intersection over Union (IoU) (3), a metric commonly used in machine learning for evaluating the performance of segmentation algorithms. It measures the overlap between two bounding boxes or segmentation masks by computing the ratio of the area of intersection to the area of the union of the two regions. Higher IoU values indicate better agreement between the predicted and ground truth regions.
Dice= 2 ×|A ∩ B| |A|+|B| , (1) (2) IoU = |A ∩ B| |A∪ B|
,
Mean Intersection over Union (mIoU) (4) is a commonly used metric in machine learning, particularly in tasks like semantic segmentation, where it's used to evaluate the performance of models that segment images into different classes or categories.To calculate mIoU, you first compute the Intersection over Union (IoU) for each class in your dataset. Then, you take the average of these IoU values across all classes to get the mean IoU.
The experiments were conducted using the collected images from two different poultry farms. The dataset consists of 361 images and was labeled using different color masks for alive and dead birds, the split between images containing dead birds and without them can be seen in the Table 1.
The dataset contains real images from poultry farm using different perspectives, distance and conditions. The images of the size 1920×1088 pixels. Each image contains from 2 to 120 different objects. The mask was created by manually annotating contours of objects. The overlapping objects can occur when hens are standing very close from each other, or one bird on the construct of the farm hiding another bird behind. For the images in the dataset five different augmentations were applied in order to expand it. Those augmentations consists of brightness, gamma and hue adjustments and flipping the image in horizontal and vertical directions.
To investigate segmentation methods, we implemented four algorithms: U-Net, O-Net, SegNet and mU-Net architectures. To find the best hyperparameters for each architecture, the experiments were conducted. In the Table 2 presented the Dice coefficient for each test. First of all, four optimizers where used, while Batch size was defaulted to 1. During this experiment, the best results for each algorithms except mU-Net got RMSprop optimizer, in the case of mU-Net – Adam. When the best optimizer for each algorithm was discovered, it is used for batch size experiments. During batch size experiments the best results were achieved using 8 images for a batch, except U-Net with 16 batch size. After those tests, each architecture was changed by switching activation function in the last layer. In this experiment, we employed softmax and sigmoid activation functions and compared the results with those obtained when no activation functions were applied in the final layer. The best results were achieved in each architecture using softmax activation function, except in case of SegNet were sigmoid reached marginally better results.
Segmentation accuracy and The Dice coefficient are presented below in Table 3. The best Dice coefficient was achieved with U-Net architecture (dice 0.95128); however, mU-Net has similar results (0.93637). Similar pattern can be seen by comparing MIoU values. U-Net reached best results (0.90675), while SegNet achieved worst results (0.66294). The segmentation task in our study is quite challenging under realistic conditions, although it is also relevant for poultry farms in other countries. But since there is no publicly available chicken dataset for the segmentation task, the dataset is constructed individually for each study, and the results are highly dependent on various conditions in poultry farm, technical details of the equipment (i.e. cameras), the AI model selected, etc. In one of the research in China, all the chicken images (white color hens) are captured from the top viewing by a monocular camera with 24 frames per second (fps) capturing images with resolution 790 × 930. They utilized two distinct datasets of cage-free chickens, one consisting of source images and the other of thermal images. In USA study to assess segmentation performance of segment anything model (SAM) on representative chicken segmentation tasks infrared thermal images have been employed. However, when flock density exceeds 9 birds/m2, SAM struggles to segment individual chickens due to their overlapping bodies. The comparison table (see Table3) of segmentation results with our selected models and dataset and state-of-the-art approaches is provided below.
In the image below (Figure 2) are presented the training loss and accuracy for each architecture. For the loss, the Binary cross entropy is used. From the graphs the trend emerges that shows that UNet is the best architecture in this field. It is worth to notice, that some interesting jumps occurred in the loss of SegNet training, however, they quickly stabilized only after one epoch.
The most difficult scenario to segmentate is when a hen is nearer to the camera while perching on the poultry farm structural elements obscuring other birds below it. The situation is visible in Figure 1. (b). Another difficulty is arising when chicken stands under feeding line, separating itself into two different objects in the camera view. The segmentation results of U-Net architecture are presented in Figure 3. From the figure below it is possible to identify, that the U-Net model reaches close results to the created mask. However, there are some differences, mainly that trained model can identify more individual chickens from image, than the label maker could, due to the mass amount of birds and slight changes in between birds.
Trained models used in 5.1. section differentiates between dead and alive birds by assigning different class label for each pixel. Classification accuracy is presented below in Table 4. The best accuracy was achieved with U-Net architecture (88.938%); however, mU-Net has similar results (87.394%). Similar pattern can be seen by comparing mIoU values. U-Net reached best results (0.86751), while SegNet achieved worst results (0.63467). However, the difference between mU-Net and U-net got even smaller while comparing mIoU results.
Figure 4 displays the classification results. The object of dead chicken is assigned correctly (red colour); however, some boarder pixels are confused and with low certainty assigned to a wrong class. Comparing the masks, there are some differences in the precision in which details age detected from the background. Trained U-Net model presents more rounded contours and loses some edge pixels. Due to this reason the accuracy drops.
This study explored the use of different deep learning architectures for image classification and segmentation tasks on poultry farms, focusing on the use of artificial intelligence to facilitate early detection of dead birds, thereby ensuring animal welfare standards and reducing the risk of disease transmission. The main findings revealed the effectiveness of deep learning, in particular the U-Net architecture, in accurately segmenting live and dead birds in farm images, achieving a Dice coefficient of 0.95128. Furthermore, the trained models successfully classified individual birds as alive or dead, with U-Net again leading the way with a classification accuracy of 88.938%. These results underline the transformative potential of AI-driven image segmentation in poultry farm management, enabling the rapid detection of dead birds and the timely initiation of interventions to control diseases and improve animal health and welfare. This study highlights the important role of AI contactless technologies in promoting sustainable and ethical poultry farming practices. However, it cloud be noted, that a large number of hens can create significant object overlap, making it challenging for the system to differentiate individual birds. This is especially true in low illumination and in the current camera placement. Lowering the cameras may seem to give a better view, but it creates another problem - feather accumulation. Chicken feathers, which are abundant, are likely to cover the lenses in a lower position and would require frequent cleaning and additional maintenance.
In future work we a planning to train the AI system to differentiate between healthy hens, hens exhibiting abnormal behavior, and dead hens. . Achieving this goal entails compiling a comprehensive dataset comprising labeled videos or images depicting hens in various states. Consequently, we have planned ongoing monitoring activities for a minimum of six months. Our strategy involves integrating algorithms designed to scrutinize hen behavior patterns. This process may encompass tracking movement patterns, recognizing postures, and observing interactions among hens to detect potential health concerns. Additionally, we intend to explore the feasibility of amalgamating visual data from cameras with data from environmental sensors. Monitoring parameters such as temperature, humidity, and ammonia levels could provide valuable insights into the well-being of the hens.
[11] Q. Ren, J. Pacheco, and J. de Brito, “Calibration of wall effects in mesostructure modelling of concrete using marker-controlled watershed segmentation,” Constr. Build. Mater., vol. 398, p. 132505, Sep. 2023, doi: 10.1016/j.conbuildmat.2023.132505. [12] W. Jia, Y. Tian, R. Luo, Z. Zhang, J. Lian, and Y. Zheng, “Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot,” Comput. Electron. Agric., vol. 172, p. 105380, May 2020, doi: 10.1016/j.compag.2020.105380. [13] X. Cao et al., “Application of generated mask method based on Mask R-CNN in classification and detection of melanoma,” Comput. Methods Programs Biomed., vol. 207, p. 106174, Aug. 2021, doi: 10.1016/j.cmpb.2021.106174. [14] D. Sarwinda, R. H. Paradisa, A. Bustamam, and P. Anggia, “Deep Learning in Image Classification using Residual Network (ResNet) Variants for Detection of Colorectal Cancer,” Procedia Comput. Sci., vol. 179, pp. 423–431, Jan. 2021, doi: 10.1016/j.procs.2021.01.025. [15] J. Hemalatha, S. A. Roseline, S. Geetha, S. Kadry, and R. Damaševičius, “An Efficient DenseNetBased Deep Learning Model for Malware Detection,” Entropy, vol. 23, no. 3, Art. no. 3, Mar. 2021, doi: 10.3390/e23030344. [16] “Chicken Image Segmentation via Multi-Scale Attention-Based Deep Convolutional Neural Network | IEEE Journals & Magazine | IEEE Xplore.” Accessed: Feb. 14, 2024. [Online].
Available: https://ieeexplore.ieee.org/abstract/document/9408579 [17] M. D. Alahmadi, “Boundary Aware U-Net for Medical Image Segmentation,” Arab. J. Sci. Eng., doi: 10.1007/s13369-022-07431-y. [18] S.-L. Chu, H. Yokota, K. Abe, D. Cho, and M.-D. Tsai, “U-net structures for segmentation of single mouse embryonic stem cells using three-dimensional confocal microscopy images,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2022, pp. 512–515. doi: 10.1109/EMBC48229.2022.9871127. [19] A. Gyorfi, L. Kovacs, and L. Szilagyi, “A two-stage U-net approach to brain tumor segmentation from multi-spectral MRI records,” ACTA Univ. SAPIENTIAE Inform., vol. 14, no. 2, pp. 223–247, Dec. 2022, doi: 10.2478/ausi-2022-0014. [20] M. Jahanbakht, W. Xiang, A. P. N. Waltham, and M. Rahimi Azghadi, “Distributed Deep Learning in the Cloud and Energy-efficient Real-time Image Processing at the Edge for Fish Segmentation in Underwater Videos,” IEEE Access, vol. PP, pp. 1–1, Jan. 2022, doi: 10.1109/ACCESS.2022.3202975. [21] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 5967–5976. doi: 10.1109/CVPR.2017.632. [22] V. Anand, S. Gupta, D. Koundal, S. R. Nayak, P. Barsocchi, and A. K. Bhoi, “Modified U-NET Architecture for Segmentation of Skin Lesion,” Sensors, vol. 22, no. 3, Art. no. 3, Jan. 2022, doi: 10.3390/s22030867. [23] M. Benazzouz, M. L. Benomar, and Y. Moualek, “Modified U-Net for cytological medical image segmentation,” Int. J. IMAGING Syst. Technol., vol. 32, no. 5, pp. 1761–1773, Sep. 2022, doi: 10.1002/ima.22732. [24] D. Gupta, “Image Segmentation Keras : Implementation of Segnet, FCN, UNet, PSPNet and other models in Keras.” arXiv, Jul. 24, 2023. doi: 10.48550/arXiv.2307.13215. [25] S. Kadry, D. Taniar, R. Damaševičius, V. Rajinikanth, and I. A. Lawal, “Extraction of Abnormal Skin Lesion from Dermoscopy Image using VGG-SegNet,” in 2021 Seventh International conference on Bio Signals, Images, and Instrumentation (ICBSII), Mar. 2021, pp. 1–5. doi: 10.1109/ICBSII51839.2021.9445180. [26] S. Almotairi, G. Kareem, M. Aouf, B. Almutairi, and M. A.-M. Salem, “Liver Tumor Segmentation in CT Scans Using Modified SegNet,”Sensors, vol. 20, no. 5, Art. no. 5, Jan. 2020, doi: 10.3390/s20051516. [27] K. Amara et al., “COVIR: A virtual rendering of a novel NN architecture O-Net for COVID-19 Ct-scan automatic lung lesions segmentation,” Comput. Graph., vol. 104, pp. 11–23, May 2022, doi: 10.1016/j.cag.2022.03.003. [28] T. Wang et al., “O-Net: A Novel Framework With Deep Fusion of CNN and Transformer for
Simultaneous Segmentation and Classification,”Front. Neurosci., vol. 16, 2022 [29] W. Kim, S. Jun, S. Kang, and C. Lee, “O-Net: Dangerous Goods Detection in Aviation Security Based on U-Net,” IEEE Access, vol. 8, pp. 206289–206302, 2020, doi: 10.1109/ACCESS.2020.3037719. [30] G. P. Luke, K. Hoffer-Hawlik, A. C. Van Namen, and R. Shang, “O-Net: A Convolutional Neural Network for Quantitative Photoacoustic Image Segmentation and Oximetry.” arXiv, Nov. 05, 2019. doi: 10.48550/arXiv.1911.01935. [31] X. Yang et al. SAM for Poultry Science. ArXiv, Computer Vision and Pattern Recognitionabs/2305.10254, 2023, pp. 1-10, https://doi.org/10.48550/arXiv.2305.10254