In recent years, the construction industry has witnessed a significant transformation, driven by technological advancements aimed at enhancing site efficiency and safety. The integration of artificial intelligence (AI) and machine learning (ML) technologies into construction operations has emerged as a pivotal strategy for addressing the perennial challenges of resource and equipment management. Among these technologies, the YOLOv5-based object detection model stands out for its potential to revolutionize the way construction sites operate, particularly in terms of equipment utilization, tool tracking, and vehicle recognition.
The global construction sector has long grappled with issues related to safety management, with accidents on construction sites posing serious risks to workers and project timelines. such as the detection of workers in complex terrains and the identification of safety hazards. This research exemplifies the versatility of YOLOv5-based models in adapting to diverse construction environments and safety requirements.
The deployment of YOLOv5-based object detection systems on construction sites facilitates a proactive approach to safety management and resource allocation. By enabling the real-time detection and classification of construction assets and potential hazards, these systems empower site managers to make informed decisions that enhance safety and operational efficiency. Moreover, the continuous improvement and customization of YOLOv5 models, as demonstrated by ongoing research, ensure their relevance and effectiveness in meeting the evolving needs of the construction industry.
The advent of YOLOv5-based object detection models has not only promised enhancements in construction site safety and efficiency but also opened new avenues for research and development within the construction industry. The ability of these models to accurately detect, classify, and track resources and equipment in real-time presents a significant leap forward in managing the dynamic and often hazardous environment of construction sites.
The application of YOLOv5 extends to the meticulous tracking of construction equipment and tools, a critical aspect for ensuring project timelines is met and reducing idle times. Yang et al. showcased the effectiveness of YOLOv5 in monitoring compliance with safety protocols, such as the wearing of helmets and masks, by construction workers [5]. Their work not only demonstrates the model's high accuracy and efficiency in real-world scenarios but also its potential to significantly reduce the risk of accidents and enhance overall site safety.
Moreover, the customization and improvement of YOLOv5 models to suit specific construction site conditions have been a focus of recent studies. Zeng et al. introduced an enhanced YOLOv3 model for equipment detection and localization, which, while predating YOLOv5, underscores the continuous evolution and refinement of YOLO architectures for construction site applications [6]. Their research highlights the importance of adapting object detection models to the unique challenges posed by construction sites, such as the detection of small or occluded objects and the need for real-time processing.
The integration of YOLOv5-based object detection into construction site management systems represents a significant step towards automating safety and resource management processes. By providing site managers with real-time data on equipment location, usage, and worker safety compliance, these systems enable more informed decision-making, ultimately leading to improved project efficiency and reduced costs. Furthermore, the ongoing development and customization of YOLOv5 models ensure that these systems remain adaptable to the ever-changing landscape of construction site management.
The precision and efficiency of YOLOv5 in object detection have significant implications for the management of construction resources. By automating the tracking of tools and equipment, YOLOv5 models minimize the likelihood of loss and misplacement, thereby ensuring that resources are optimally utilized and readily available when needed. This capability is crucial for maintaining project schedules and reducing downtime. For instance, the work of Wan et al. on utilizing YOLOv5 for object detection in high-resolution optical remote sensing images, though focused on a different application, underscores the model's robustness and adaptability in detecting objects across various scales and conditions [7]. Such attributes are invaluable in the complex and ever-changing environment of construction sites.
Moreover, the application of YOLOv5 extends to enhancing safety measures on construction sites. Through real-time monitoring and detection of safety gear compliance, such as helmets and vests, YOLOv5 models play a pivotal role in preventing accidents and ensuring the wellbeing of construction workers. The research by Zhou et al. on the detection of construction waste using an improved YOLOv5 model illustrates the model's versatility and high performance in identifying specific objects within cluttered scenes [2]. This capability is directly applicable to safety monitoring on construction sites, where the ability to accurately detect personal protective equipment (PPE) amidst the site's activity can significantly impact overall safety outcomes.
The ongoing development and customization of YOLOv5 models for construction site management underscore the potential for further innovations in this field. As researchers and practitioners continue to explore new applications and enhancements of YOLOv5 technology, the construction industry stands on the cusp of a new era of digital transformation. This transformation is characterized by increased automation, improved safety protocols, and enhanced resource management, all of which contribute to the overall efficiency and success of construction projects.
As the construction industry continues to evolve, the integration of cutting-edge technologies like YOLOv5-based object detection into resource and equipment management practices has become increasingly vital. This technology's capacity to enhance construction site efficiency and safety through advanced equipment utilization, precise tool tracking, and accurate vehicle recognition marks a significant leap forward in the sector's operational capabilities.
The adaptability and efficiency of YOLOv5 in various construction site scenarios have been demonstrated through numerous studies, each contributing to the model's ongoing refinement and application. For instance, the work by Peng et al. on CORY-Net for intelligent safety monitoring on power grid construction sites exemplifies the potential of YOLOv5-based models to enhance worker safety and operational oversight [4,8]. Similarly, the study by Yang et al. on the application of YOLOv5 for PPE compliance monitoring further underscores the model's utility in promoting construction site safety [5,9]. These studies, among others, provide a solid foundation for exploring new avenues for applying YOLOv5 in construction site management.
The purpose of this research can be summarized as follows: By addressing these objectives, this research aims to contribute to the body of knowledge on the application of advanced object detection technologies in construction site management. Through a detailed analysis of current practices and future potentials, we seek to illuminate the path toward a more efficient, safe, and technologically advanced construction industry.
The proposed study aims to enhance construction site efficiency and safety through the implementation of a YOLOv5-based object detection model. This section outlines the materials and methods used to develop, train, and deploy the model for resource and equipment management on construction sites.
Proposed Framework for Resource and Equipment Management System:
1. Data Collection.
1.1. Public Datasets. Initially, public datasets containing images of construction equipment, tools, and vehicles were utilized. These datasets offer a broad range of object types and scenarios, providing a solid foundation for the initial training of the YOLOv5 model. Public datasets (ACID [10], TTM [11]) are invaluable for introducing the model to a wide variety of objects and conditions it might encounter in realworld construction environments. 1.2.
Self-captured Images. To tailor the model more closely to the specific needs and conditions of construction sites, a significant portion of the dataset was composed of self-captured images and video footage. This involved on-site data collection at various construction projects, capturing images and videos of resources, equipment in different operational states (e.g., idle, in use), and under diverse environmental conditions. This step was critical for incorporating real-world variability into the dataset, ensuring the model's effectiveness across different construction sites and conditions. 2. Preprocessing.
Data Cleaning. The first step involved filtering out irrelevant images and correcting any errors within the dataset. This process ensured that only pertinent and accurate data were included, enhancing the quality of the training material.
Resize/Adjust Brightness and Contrast. To standardize the dataset, all images were resized to a uniform dimension suitable for the YOLOv5 model. Additionally, adjustments to brightness and contrast were made where necessary to simulate various lighting conditions, further improving the model's robustness and accuracy.
Image Labeling. Using annotation tools like YOLO Label, each image in the dataset was meticulously labeled to identify and classify different types of construction resources and equipment, along with their operational states. This step is crucial for supervised learning, as it provides the model with the necessary information to learn from the visual data.
Train the Object Detection Model (YOLOv5). The YOLOv5 model is configured and trained using the prepared dataset. The training process is optimized for accuracy in detecting various resources and equipment specific to construction sites.
Evaluate the Model. The trained model is evaluated on a separate set of images to assess its performance. Evaluation metrics include precision, recall, and mAP (mean Average Precision), providing insights into the model's effectiveness in real-world scenarios. 4. Integration and Deployment.
Object Detection Model Weights (YOLOv5). The trained model is deployed into the construction site management system, enabling real-time analysis and detection of resources and equipment. 4.2.
Input Source. The model utilizes input sources such as CCTV footage, static images, or live video feeds for continuous object detection and monitoring. 5. Real-time Detection and Management.
Detecting Resources and Equipment. The system identifies and classifies resources and equipment in real-time, distinguishing between different types (e.g., tools, machinery) and states (idle, in use), facilitating immediate action and decisionmaking.
Environmental Conditions. The model optionally integrates environmental condition detection to adjust resource management strategies in response to weather changes, enhancing operational adaptability. 6. Resource and Equipment Status Dashboard.
Visualization and Alerts. A dashboard presents detected resources and equipment, highlighting their status, location, and usage. Alerts are generated for underutilized resources or when equipment maintenance is due, ensuring optimal resource management. 6.2.
Decision Support. The system provides actionable insights for resource allocation, maintenance scheduling, and equipment usage optimization based on real-time data, supporting informed decision-making. 7. Feedback Loop for Continuous Improvement.
Model Retraining. New data and feedback are periodically collected to retrain the model, improving its accuracy and adapting to new types of resources or changes in the construction site environment.
System Updates. The management dashboard and decision support tools are updated based on insights gained from model performance and user feedback, ensuring the system's continuous improvement and relevance.
This comprehensive framework leverages YOLOv5 for object detection to manage resources and equipment on construction sites effectively and is represented as a model in Figure 1. By emphasizing the detection and classification of resources and integrating this information into actionable insights for site managers, the system ensures resources are used efficiently and effectively, enhancing overall site safety and operational efficiency.
Training an object detector is fundamentally a supervised learning problem that requires a wellcurated dataset to inform and refine the model's learning process. The dataset serves as the foundation upon which the object detection model, in this case, YOLOv5, is trained, validated, and tested. The construction of a comprehensive and representative dataset is crucial for the success of the model in accurately identifying and classifying various objects within construction sites [12]. The following outlines the meticulous process undertaken to build the dataset for this study.
The data collection process for enhancing construction site efficiency and safety through YOLOv5-based object detection focuses on gathering a diverse array of images representing various states of equipment utilization, tool and machinery tracking, and vehicle recognition. This comprehensive approach ensures the model is well-equipped to accurately identify and classify a wide range of objects under different conditions, crucial for real-world application on construction sites.
For the category of equipment utilization, images were collected to represent both idle and active states of essential construction machinery:
This category involved collecting images of handheld tools and machinery, differentiating between their usage states:
Hand Drill. 200 images were collected, distinguishing between drills in use and those stored. Power Saw. Around 170 images differentiating between power saws in operation and those turned off. Jackhammer. Secured 160 images identifying jackhammers, noting if they are being used on the site. Welding Machine. Collected 140 images of welding machines, with a focus on capturing them in active use for metal joining tasks.
For vehicle recognition, the dataset includes images representing both the operational and idle states of key construction vehicles:
Each image within the dataset was meticulously annotated to provide the YOLOv5 model with clear, diverse examples of each state or type of equipment, tool, and vehicle. This diversity is crucial for helping the model learn the nuances of each category, thereby improving its ability to accurately identify and classify objects in real-world construction site scenarios. The annotation process included labeling images from various angles, lighting conditions, and distances to build a robust and versatile dataset, ensuring the model's effectiveness across a wide range of construction environments.
Table 1 shows the number of cases across different classes. The fundamental idea is to analyze a sequence of images to identify whether an object, such as a concrete mixer, remains in the same state (indicating inactivity) or transitions between states (indicating activity). This determination is made by observing changes in the object's features or position across the image sequence.
Object Does Not Change Its State -Not Active. When a sequence of images is fed into a detection system where the object does not change its state, the object is classified as not active. For a concrete mixer, this would mean that across multiple frames, there is no visible change in its position, orientation, or any operational components (e.g., the mixing drum remains stationary). The lack of change suggests that the concrete mixer is idle. Detecting inactivity involves analyzing the object's features across the sequence and noting the absence of significant variation.
Object Changes Its State -Active. Conversely, if the object changes its state across the sequence of images, it is classified as active. For the concrete mixer example, this would be indicated by visible changes such as the rotation of the mixing drum, movement of the mixer from one location to another, or other signs of operation. Detecting activity involves identifying variations in the object's features, such as changes in texture (rotation patterns of the drum), position, or other operational indicators that signify the mixer is in use. An example of an active equipment recognition system is shown in Figure 2 This principle of object activity detection is not limited to concrete mixers but can be applied to a wide range of objects and scenarios where understanding the operational state is crucial. Implementing such a system requires careful consideration of the features to be extracted, the method for temporal analysis, and the criteria for classifying the state of the object.
Data cleaning is a critical step in preparing the dataset for training a YOLOv5-based object detection model, especially when the goal is to enhance construction site efficiency and safety. This process involves meticulously reviewing the dataset to remove any irrelevant, duplicate, or poor-quality images that could potentially hinder the model's learning and performance. The objective is to ensure that the dataset is as accurate and representative of real-world scenarios as possible [13].
The first step in the data cleaning process involved identifying and removing images that do not contribute to the model's learning objectives. For instance, images that do not clearly depict construction equipment, tools, or vehicles in the specified states (idle or active) were considered irrelevant. This step is crucial for maintaining the focus of the model on the target objects and scenarios relevant to construction site management.
Duplicate images can skew the model's learning process, leading to overfitting on specific examples. Therefore, the dataset was carefully scanned to identify and remove any duplicates. This ensures a diverse range of examples for each class, promoting a more generalized understanding and detection capability within the model.
Mislabelled images present a significant challenge in supervised learning models. Incorrect labels can confuse the model, leading to inaccuracies in object detection and classification. A thorough review of the dataset annotations was conducted to correct any mislabelled images, ensuring that each image accurately represents the intended class and state of the construction equipment, tools, or vehicles.
Quality control measures were implemented to remove images that are blurry, poorly lit, or obstructed, which could compromise the model's ability to learn effectively. Images were evaluated for clarity, lighting, and visibility of the target objects, with substandard images being removed from the dataset. This step is essential for ensuring that the model is trained on highquality images that accurately reflect the conditions under which it will operate on construction sites.
Upon completion of the data cleaning process, the dataset underwent a final review to confirm its readiness for model training. This involved a comprehensive assessment of the dataset's diversity, representativeness, and alignment with the study's objectives of improving construction site efficiency and safety through YOLOv5-based object detection.
The meticulous data cleaning process undertaken in this study ensures that the dataset is optimized for training a highly effective and accurate YOLOv5 model. By focusing on relevance, diversity, accuracy, and quality, the cleaned dataset lays a solid foundation for developing a robust object detection system capable of enhancing resource and equipment management on construction sites.
Data cleaning is Image preprocessing is a pivotal phase in preparing the dataset for the training of a YOLOv5-based object detection model, aimed at enhancing construction site efficiency and safety [14]. This stage involves several key processes designed to improve the quality of the images and their suitability for model training. The goal is to standardize the dataset, enhancing the model's ability to learn from the images and accurately detect and classify various objects under different conditions on construction sites.
To ensure consistency and optimize processing efficiency, all images in the dataset were resized to a uniform dimension recommended for YOLOv5 training. This standardization is crucial for maintaining computational efficiency and ensuring that the model receives input images of a consistent size, which is vital for the internal architecture of the CNN (Convolutional Neural Network) used in YOLOv5.
Given the variability of lighting conditions on construction sites, images in the dataset were adjusted for brightness and contrast to simulate a wide range of environmental conditions. This step is essential for training the model to perform reliably in different lighting scenarios, from bright sunlight to overcast or poorly lit conditions. By adjusting the brightness and contrast, the model is better equipped to recognize and classify objects regardless of the lighting environment.
Image normalization was applied to scale pixel values to a standard range, typically between 0 and 1. This process helps in reducing the variance among images and speeds up the convergence of the model during training. Normalization ensures that the model treats each image uniformly, improving the learning efficiency and stability of the YOLOv5 model.
To further enhance the robustness of the model, data augmentation techniques were employed. These included rotations, translations, flipping, and scaling of images. Data augmentation introduces variability into the training dataset, simulating a broader range of scenarios that the model might encounter in real-world applications. This approach helps in preventing overfitting and improves the model's generalization capabilities.
Considering the importance of color information in identifying and classifying construction equipment, tools, and vehicles, some images were converted into different color spaces (e.g., HSV or LAB) as part of the augmentation process. This conversion allows the model to learn from a wider variety of color distributions, enhancing its ability to detect objects across different environmental conditions and backgrounds [15].
After completing the preprocessing steps, the dataset was compiled into a format suitable for training the YOLOv5 model. This involved organizing the images and their corresponding annotations (labels) into training, validation, and test sets. The division of the dataset allows for comprehensive training and evaluation of the model's performance, ensuring its effectiveness in enhancing construction site efficiency and safety.
Through meticulous image preprocessing, the study ensures that the dataset is optimized for training the YOLOv5 model. By focusing on image quality, consistency, and variability, the preprocessing steps lay a solid foundation for developing an object detection system capable of accurately identifying and classifying objects in diverse conditions encountered on construction sites.
Image labeling is a critical step in the development of a YOLOv5-based object detection model for improving construction site efficiency and safety. This process involves annotating images with labels that accurately describe the objects present, their categories, and their states (e.g., idle or active). For this study, Label Studio, a versatile tool for annotating images for machine learning applications, was employed to facilitate the labeling process.
Label Studio was chosen for its user-friendly interface and flexibility in handling various types of annotations, including bounding boxes, which are essential for object detection tasks. Its compatibility with a wide range of data types and export formats makes it an ideal choice for projects requiring detailed and accurate annotations.
Based on the study's focus on construction site management, specific classes and states were defined for labeling: Equipment Utilization. Classes included bulldozers, concrete mixers, and generators, with states designated as idle or active. Tool and Machinery Tracking. Classes encompassed hand drills, power saws, jackhammers, and welding machines, with annotations indicating whether they were in use or stored. Vehicle Recognition. Classes covered cranes, dump trucks, excavators, and cement trucks, with states reflecting loading activities, digging, pouring concrete, or being idle.
To ensure consistency and accuracy in the labeling process, comprehensive annotation guidelines were developed. These guidelines provided detailed instructions on how to identify and label each class and state, including how to draw bounding boxes around objects and the level of detail required in annotations. The guidelines emphasized the importance of precision in bounding box placement to ensure the model learns the exact dimensions and features of each object.
A team of annotators was trained using the developed guidelines to ensure a uniform understanding of the labeling task. This training included practical exercises in Label Studio, focusing on accurately identifying objects, selecting the correct labels, and drawing bounding boxes. Regular review sessions were held to address any inconsistencies and refine the labeling process.
To maintain high-quality annotations, a two-step review process was implemented. Initially, each labeled image was reviewed by a senior annotator for accuracy and adherence to the guidelines. Following this, a random sample of the annotations was audited by the project lead to ensure overall quality and consistency across the dataset.
Upon completion of the labeling process, the annotated data were exported from Label Studio in a format compatible with YOLOv5 training requirements. This included the images and their corresponding labels (bounding box coordinates and class identifiers), organized in a manner that facilitates efficient model training and evaluation.
Through meticulous image labeling using Label Studio, this study established a comprehensive and accurately annotated dataset for training the YOLOv5 model. The detailed annotations provide the model with the necessary information to learn the characteristics of various construction site objects, enabling effective detection and classification crucial for enhancing site safety and resource management.
In our study, the comprehensive dataset was meticulously divided using a random selection process into three distinct subsets: 70% for training, 20% for validation, and 10% for testing. This division resulted in a training set comprising 1,897 images of construction equipment, tools, and vehicles in various operational states, including 1,610 images of active and idle machinery instances and 287 images highlighting tool and machinery tracking scenarios. The validation set included 542 images, with 460 images dedicated to equipment and vehicle recognition in different states and 82 images focusing on tool and machinery tracking. Lastly, the test set consisted of 271 images, with 230 images showcasing equipment and vehicles in diverse operational conditions and 41 images for the evaluation of tool and machinery tracking performance. This structured approach to dataset allocation ensures a balanced representation of all classes and states, facilitating a comprehensive assessment of the YOLOv5 model's capability to enhance construction site efficiency and safety through advanced object detection.
To rigorously test and evaluate the performance of the proposed YOLOv5-based object detection model for enhancing construction site efficiency and safety, imagery data collected from a local construction site using CCTV cameras were utilized. The evaluation process focused on measuring the accuracy and reliability of the model in detecting and classifying various construction-related objects, employing Intersection over Union (IoU) and a confusion matrix as the primary metrics.
IoU is a critical metric in object detection that quantifies the accuracy of the predicted bounding box against the ground truth (actual) bounding box. It is calculated as the area of overlap between the predicted and actual bounding boxes divided by the area of their union. The IoU value ranges from 0 to 1, where 0 indicates no overlap and 1 signifies perfect alignment between the predicted and actual bounding boxes. The equation for IoU is given by:
where area of overlap is the area where the predicted bounding box and the actual (ground truth) bounding box overlap; area of ∪¿ is the total area covered by both the predicted bounding box and the actual bounding box, minus the area of overlap. It represents the combined area of both boxes where either box has coverage.
The confusion matrix is a tool that helps visualize the performance of the object detection model. It categorizes the predictions into four types: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). From the confusion matrix, several performance metrics can be derived, including precision, recall, and mean average precision (mAP). Precision measures the model's accuracy in predicting positive observations and is defined as the ratio of TP to the sum of TP and FP. It indicates the reliability of the model's positive detections. The equation for Precision is given by:
where TP are the true positive predictions; FP are the false positive predictions.
Recall assesses the model's sensitivity or its ability to correctly identify all relevant instances. It is calculated as the ratio of TP to the sum of TP and FN. The equation for Recall is given by: Recall= TP TP+ FN ,
where FN are the false negative predictions.
Mean Average Precision (mAP) is used to evaluate the model's accuracy across all classes within the dataset. It is the mean of the average precision (AP) scores for each class, where AP is computed as the weighted sum of precisions at each threshold, with the increase in recall from the previous threshold used as the weight. The equation for mAP is given by:
where n is the total number of classes in the dataset; AP is calculated for each class and represents the precision at different recall levels. It takes into account the order of the predictions, rewarding models that return true positives earlier. The equation of AP is given by:
where k is the index used to sum over a sorted list of objects, thresholds, or intervals.
The proposed model was evaluated using the described metrics on the dataset split into training, validation, and test sets. The IoU threshold was set to 0.5, a common practice in object detection tasks, to determine whether a detection is considered a true positive. The precision, recall, and mAP values were calculated based on the outcomes of the confusion matrix, providing a comprehensive assessment of the model's performance in accurately detecting and classifying objects on construction sites.
This rigorous testing and evaluation process ensures that the YOLOv5-based model is not only accurate in identifying construction site objects but also reliable and effective in real-world scenarios, contributing significantly to the improvement of construction site safety and efficiency.
The model underwent training for 30 epochs on the dataset comprising construction equipment, tools, and vehicles, with a batch size set at 16. The training process was completed in approximately 23 minutes utilizing a Google Colab GPU. Figure 3 illustrates the model's performance across the training phase for the construction equipment and tools dataset, showcasing the metrics of precision, recall, and mAP at the 50 IoU threshold. The performance of YOLOv5 on the validation dataset, which included images of classes, is summarized in Table 2. The model achieved an overall precision of approximately 88%, a recall of 79%, and a mAP at the 50 IoU threshold of 85%.
Validation results on the different classes 4.
Thus, the implementation of the YOLOv5-based object detection model for enhancing construction site efficiency and safety has demonstrated significant potential in revolutionizing the management of resources and equipment. Through meticulous training, validation, and testing processes, the model has shown high accuracy in detecting and classifying various construction-related objects, including equipment in idle and active states, tools, and vehicles,
Class The model's training over 30 epochs, utilizing a dataset meticulously prepared with images of construction equipment, tools, and vehicles, resulted in a final precision of 0.852, a recall of 0.723, and a mAP_0.5 of 0.792. These metrics underscore the model's capability to accurately identify and classify objects, which is crucial for real-time monitoring and management applications. The high performance across different classes, particularly in vehicle recognition and equipment utilization, highlights the model's versatility and effectiveness in addressing the dynamic needs of construction site management.
The validation and testing phases further affirmed the model's reliability, with precision and recall rates consistently above 85% and 79%, respectively, across various object categories. This level of accuracy ensures that the model can serve as a dependable tool for construction site managers, enabling them to make informed decisions based on real-time data regarding the status and location of tools, machinery, and vehicles.
In conclusion, the YOLOv5-based object detection model represents a significant advancement in leveraging computer vision and deep learning technologies for construction site management. By providing a robust solution for real-time detection and classification of construction resources and equipment, the model paves the way for smarter, safer, and more efficient construction site operations. Future work will focus on further refining the model's accuracy, exploring its integration with other technological solutions, and expanding its application to a broader range of construction site scenarios, ultimately contributing to the ongoing digital transformation of the construction industry.