Reviewing relevant QA tools is essential for improving eficiency, collaboration, and software quality while reducing costs. It enables QA teams to meet the fast-changing demands of software development and deliver reliable software on time and within budget. Strategic tool selection, emphasizing standardization, integration, and alignment with business goals, streamlines processes and reduces complexity. Edge computing environments require a fundamentally diferent approach to testing compared to traditional cloud or on-premise applications. While traditional testing tools are valuable, edge computing requires specialized solutions that can handle distributed architectures and variable network conditions. This paper aims to analyze challenges and key approaches of Edge Computing systems' testing, evaluate the most suitable QA tools, organize them, and highlight their benefits to help QA teams identify tools that meet their requirements. Bibliometric analysis of researched field discovered that the topic of quality assurance of diferent types of edge computing systems is highly discussed and important nowadays. The main features of providing eficient quality assurance process for edge computing systems is to set up appropriate testing of their performance, reliability, security, integrity and usability. The authors have defined the criteria for selecting edge computing system testing tools, such as scalability, compatibility, integrity, cost savings, licensing, and ease of use. For each of them, several indicators are proposed that allow measuring the value of these indicators. The most relevant tools that match defined criteria are suggested.
Modern software applications are more complex than ever, involving multiple platforms, devices, APIs,
and integrations [
Edge computing can be defined as an emerging technology that uses cloud computing to leverage
edge data centers to process, store, and analyze data close to the source. Traditional cloud computing
architectures are not designed for latency-critical applications such as AI (artificial intelligence) and
IoT (Internet of things) because they rely on low data volumes generated by applications running near
highly-populated areas [
Performing functional and integration testing, along with scalability and robustness assessments on the codebases is generally limited to configurations involving a single or a small number of physical servers. In contrast, edge architectures necessitate rigorous evaluation of functionalities designed to address the challenges posed by geographically distributed infrastructures, particularly when architectural models exhibit fundamentally distinct configurations. To ensure stable and reliable results, it is advisable to adopt best practices from the scientific community.
Testing, should not only be rooted in precision engineering, but also involve creative problem-solving.
Low-level code testing, such as unit tests or API response validation, is relatively straightforward and
facilitates the creation of frameworks capable of executing automated test suites that meet criteria such
as repeatability, replicability, and reproducibility. However, testing integrated systems to replicate the
configurations and operational conditions of production environments presents considerably greater
complexity [
Reviewing and continuous monitoring of modern QA tools is crucial for eficiency, collaboration,
cost savings, and quality improvement. It helps QA teams adapt to the fast-paced, dynamic demands of
software development while maintaining high standards of software reliability and user satisfaction.
With proper systematization, teams are better equipped to deliver exceptional software on time and
within budget [
The purpose of this article is to analyze and estimate the most suitable QA tools for edge computing systems in 2024, review them and highlight their benefits to help QA teams to find tools that meet their requirements.
To explore the development of research in edge computing system’s quality assurance field and to define
the key concepts of the research by studying the keywords of related scientific papers and articles,
the bibliometric analyses was performed using data from two sources: Scopus and Web of Science
databases. As an example of bibliometric analysis workflow, the methodologies of Mintii and Semerikov
[
Scopus search query from January 6, 2025: TITLE-ABS-KEY ( software ) AND TITLE-ABS-KEY ( "Edge Computing" ) AND TITLE-ABS-KEY ( testing OR "quality assurance" ) AND ( tool* ) AND PUBYEAR > 2014 AND PUBYEAR < 2026 AND ( LIMIT-TO ( SUBJAREA , "COMP" ) OR LIMIT-TO ( SUBJAREA , "ENGI" ) OR LIMIT-TO ( SUBJAREA , "MATH" ) OR LIMIT-TO ( SUBJAREA , "DECI" ) OR LIMIT-TO ( SUBJAREA , "ECON" ) OR LIMIT-TO ( SUBJAREA , "MULT" ) ) AND ( LIMIT-TO ( DOCTYPE , "ar" ) OR LIMIT-TO ( DOCTYPE , "cp" ) ) AND ( LIMIT-TO ( LANGUAGE , "English" ) )
AK="Edge Computing" AND AK=(test* OR "quality assurance") AND AK=software
The Web of Science query line is much shorter than it is for Scopus, because the functionality of Web
of Science platform’s user interface is not automatically adding used checkbox filters to its searching
query. But all the searching conditions was the same for both sources. After uniting duplicate results,
the final documents list contained 70 files. VOSviewer [
Analyzing the researched field’s keywords grouped by VOSviewer and identifying associated thematic areas, we suggest to name formed clusters next way: • Cluster 1 (red) – Edge computing system types and their performance. • Cluster 2 (green) – Usage of machine learning in edge computing systems testing. • Cluster 3 (Blue) – Open source systems. • Cluster 4 (Yellow) – Network architecture.
To analyze generated map, we can use 4 main attributes: • weight<Links> (wl) – the number of links of an element with other elements • weight<Total link strength> (wtls) – general strength of links between target element and other elements • weight<Occurrences> (wo) – related to keywords it shows the number of their occurrences in analyzed documents • score<Avg. pub. year> (sapy) – average year of documents’ publication where target keyword is used.
Analysis data discovers that keywords Edge Computing and Software Testing from cluster 2 have the highest numbers of all attributes between all the other keywords (edge computing: wl – 20, wtls – 143, wo – 55; software testing: wl – 19, wtls – 70, wo – 27). It means that the topic of quality assurance of diferent types of edge computing systems is highly discussed and important. Moreover it keeps changing, developing and always requires new investigations due to average year of documents’ publication attribute values (edge computing: sapy – 2021,95; software testing: sapy – 2021.89).
The Internet of Things (IoT) is one of the largest and most influential network architectures. It
describes access and processing of the data given by devices with sensors via wired and wireless
networks. IoT requires scalable and flexible management of many interconnected devices and sensors
today [
Also due to Occurrences and Avg. Pub. Year attributes’ values we can note that the most relevant
and popular keywords of investigated field are cloud-computing systems ( wo – 10, sapy – 2023,43) and
general edge computing system’s performance testing (sapy – 2023,40). It shows that the discussion of
cloud computing and edge computing systems’ synergy is developing and becoming more attractive
for developers and scientists in many fields such as distributed computing [
The bibliometric analysis shows the rise of attention to such studies as the use of AI for deep learning
of edge devices [
Edge computing applications, such as autonomous vehicles and smart manufacturing systems, demand instantaneous data processing and response, leaving minimal tolerance for delay or error. Latency in data handling can lead to critical safety risks or significant operational ineficiencies. Data testing should ensure that systems meet the stringent time-sensitive requirements of edge environments by validating that data is processed and acted upon within milliseconds. For instance, in an autonomous vehicle, real-time testing verifies that the vehicle’s sensors and AI algorithms accurately detect obstacles and respond promptly, thereby mitigating the risk of accidents and ensuring safe operation.
System reliability is a critical consideration in edge computing, where edge devices function autonomously, often without direct dependence on centralized data centers. Testing should identify potential system failures before they escalate into critical issues. By continuously monitoring the performance of edge devices, testing must detect early indicators of hardware malfunctions, software anomalies, or network disruptions.
Security is a paramount concern in edge computing environments, where data is processed outside
the controlled and secure boundaries of traditional data centers. Data testing must enhance security
by continuous monitoring of data flows for vulnerabilities or breaches. For instance, real-time testing
can detect anomalous data patterns indicative of cyberattacks or unauthorized access to edge devices
[
In edge computing, the seamless coordination and performance of multiple devices operating across
a distributed network are essential for ensuring smooth and eficient operations. Data testing plays
a crucial role in verifying that each component within the edge network functions optimally. This
involves validating the eficiency of inter-device communication, evaluating the speed and accuracy
of data processing, and identifying potential bottlenecks that could impair overall performance. By
implementing continuous testing, organizations can enhance the performance of their edge systems,
achieving greater speed, reliability, and operational efectiveness [
Edge computing applications often operate in dynamic and challenging environments, such as ofshore oil rigs, remote industrial sites, or aerospace systems. These settings pose unique dificulties, including variable network connectivity, extreme environmental conditions, and restricted access to centralized resources. Testing must address these challenges by validating the adaptability and resilience of edge systems. We should ensure that edge devices maintain functionality and reliability under unstable network conditions or harsh environmental factors, thereby supporting continuous and dependable operations in critical applications.
Edge computing relies on uninterrupted data flow and processing to deliver the low-latency and highperformance advantages it promises. Continuous monitoring and verifying that data flows seamlessly from edge devices to local processing units is essential. This ongoing evaluation ensures that any interruptions, delays, or inconsistencies in data transmission are promptly detected and resolved, enabling edge systems to sustain a consistent and reliable flow of real-time data for optimal performance.
In edge computing, where data processing occurs closer to its source, real-time data testing plays a
pivotal role in ensuring systems operate with precision, eficiency, and dependability. Edge
environments frequently manage time-critical data, where delays, inaccuracies, or failures can lead to severe
consequences, particularly in domains such as autonomous vehicles, industrial IoT, and healthcare
monitoring. By continuously validating the integrity, performance, and functionality of edge devices
and systems under live conditions, real-time data testing mitigates these risks, enabling reliable and
responsive operations in dynamic and high-stakes scenarios [
The primary objective of real-time data testing is to guarantee the accuracy and reliability of data processed at the edge. In decentralized systems, edge devices—such as sensors, cameras, and smart gateways—generate and collect substantial volumes of data. Real-time testing ensures that this data is correctly formatted, transmitted, and synchronized across distributed networks. By detecting inconsistencies or corruptions in the data flow, real-time testing prevents potential inaccuracies that could otherwise result in erroneous decisions or malfunctioning operations in edge-based applications.
Burn-in testing is a rigorous process aimed at identifying early failures in components and minimizing
the likelihood of defects and malfunctions during field operation. This testing subjects computing
components to extreme operating conditions, such as high and low temperature extremes, intensive
usage cycles, and elevated voltages. The goal is to identify and eliminate defective components or those
with inherently short lifespans before their deployment in operational systems. The primary purpose of
burn-in testing is to ensure that components can endure severe environmental and operational stresses,
demonstrating their durability and dependability under diverse and challenging conditions. By doing
so, burn-in testing helps prevent costly or potentially catastrophic system failures, thereby enhancing
the reliability and safety of mission-critical applications [
Manual and automation testing leverages a wide array of tools designed to help QA engineers create,
organize, and track test cases, manage bugs, and ensure overall software quality. These tools can
focus on specific testing areas, such as functional, API, or web service testing, or be used for broader
approaches like exploratory testing. Comprehensive QA management systems support the entire
process, from planning and execution to analysis and reporting. They often integrate seamlessly with
collaboration platforms, making it easier to assign tasks and coordinate eforts within teams [
In general we can categorize testing tools by their usage in diferent test types (functional, performance, API, load, etc), by test management area (test case management, bug tracking, logging, monitoring, etc.) and by general testing purposes (test data generation, device simulation, screen recording). Suggested categorization is displayed on figure 3
Considering all the testing challenges and features of edge computing systems’ testing described at previous chapter, we can formulate list of criteria to provide appropriate QA tools selection (table 2).
Most of the above indicators can be assessed based on oficial documentation from software vendors or cloud service providers. However, such indicators as load handling, horizontal scaling, vertical scaling, stress testing, latency maintenance and tool customizability should be determined based on expert assessment with subsequent verification of the respondents’ degree of agreement. So, developing a methodology for integrated assessment according to the criteria and indicators we have defined using both approaches is advisable.
In general, QA team must aggregate full set of tools to be able to provide: • functional testing; • latency and performance testing; • security testing; • fault tolerance and resilience testing;
• monitoring and measurement.
To achieve a comprehensive coverage of special requirements to edge computing systems’ testing, there is a need for detailed selection of appropriate tools. As a result of investigation, tools indicated in table 3 can be suggested as the most suitable to perform needed workflows and assertions.
While traditional testing tools are valuable, edge computing requires specialized solutions that can handle distributed architectures and variable network conditions. Based on the papers analyzed above and our own experience, we ofer the set of essential tools for edge computing QA:
The Linux-based network emulation tool NetEm (Network Emulator) [
NetEm’s flexibility and granular control make it a valuable tool for testing the resilience of edge applications under challenging conditions. It helps developers assess system behavior during network disruptions, which is critical for applications requiring high reliability and low latency.
WANem (Wide Area Network Emulator) [
WANem provides a straightforward approach to emulating WAN scenarios, ensuring that edge systems can handle bandwidth constraints and varying latency. While it is efective for general testing, it may lack some advanced features required for highly specific edge environments.
Simple yet efective tool for injecting network faults into local network communications is Clumsy
[
Clumsy provides basic but efective network emulation, making it ideal for testing small edge deployments. Its limited feature set may not sufice for more complex distributed environments, but it’s a good starting point for basic network testing.
For performance testing Apache JMeter [
Indicators 1.1 Load Handling: Maximum number of concurrent users, devices, or data streams the tool can manage. 1.2 Horizontal Scaling: Support for distributed environments by adding more edge nodes or servers. 1.3 Vertical Scaling: Ability to optimize performance by increasing resource utilization (CPU, memory, storage) on existing nodes. 1.4 Stress Testing: Performance under high data loads, network trafic, or concurrent operations. 1.5 Latency Maintenance: Ability to maintain low latency as the system scales. 2.1 Multi-Platform Support: Compatibility with operating systems and hardware architectures 2.2 Protocol Support: Support for edge-specific protocols 2.3 Containerization: Support for container technologies (e.g., Docker, Kubernetes) often used in edge deployments. 2.4 Interoperability: Integration with diferent IoT platforms, cloud backends, and middleware. 2.5 Real-Time Processing: Support for real-time data processing and decision-making capabilities. 3.1 CI/CD Support: Compatibility with CI/CD tools like Jenkins, GitLab CI/CD, GitHub Actions, Azure DevOps, or CircleCI. 3.2 Automated Test Execution: Capability to trigger tests automatically during build, deployment, or code changes. 3.3 Version Control Integration: Compatibility with version control systems like Git for managing code and configurations. 4.1 Upfront Cost: Competitive price of purchasing the tool or system. 4.2 Subscription Model: Availability of flexible pricing models, such as pay-as-you-go or annual subscriptions. 4.3 Free Trial or Open Source: Availability of free versions or open-source alternatives. 5.1 Licensing Transparency: Support for commercial, academic, or non-profit licensing. 6.1 User Interface (UI): Presence of an intuitive and visually accessible graphical interface. 6.2 Documentation: Availability of comprehensive user guides, tutorials, and knowledge bases. 6.3 Support Channels: Availability of help via chat, forums, email, or phone support. 6.4 Customizability: Flexibility to tailor the tool’s settings, workflows, or reports to specific needs. valuable for testing APIs, microservices, and edge-based web applications, helping organizations ensure these components can handle real-world trafic loads and response times.
JMeter’s ability to simulate heavy user loads and test APIs ensures thorough performance validation in edge environments. While it excels in scalability testing, it may struggle to handle complex distributed setups without additional configuration.
As for the load testing Gatling[
Gatling’s intuitive interface and eficient resource usage make it a strong choice for edge performance testing. While it requires some initial learning, it provides comprehensive insights into system behavior under load.
Python-based load testing tool that enables developers to simulate user behavior across distributed
systems is Locust [
Type Name Network em- Linux NetEM ulators
Benefits for edge computing systems’ testing Simulates real-world network conditions like latency, packet loss, and jitter, helping to test edge systems’ performance under diverse and challenging network scenarios.
Provides an easy-to-use interface for emulating wide-area network conditions, enabling the testing of edge system performance under unreliable and constrained connectivity.
Allows network fault injection (latency, packet drops, etc.), ensuring robustness and reliability of edge systems operating in fluctuating network environments.
Simulates high user loads to evaluate edge systems’ performance, scalability, and ability to process simultaneous requests efectively.
Delivers detailed performance metrics for edge systems by testing data processing speeds, throughput, and response times under varying loads.
Facilitates distributed load testing, ideal for edge environments with numerous connected devices, to ensure the scalability of the overall system.
Scans edge devices for open ports, which is key strategy to find network vulnerabilities or system miss-configurations.
Search engine that can be used to scan IoT devices connected to the same network and find potential threats or vulnerabilities.
Framework for penetration testing to simulate attacks on edge devices and applications by supporting exploit matching, execution, and development.
Enables real-time monitoring of edge devices and applications, identifying performance bottlenecks and ensuring reliable data collection and processing.
Provides visual analytics and dashboards for edge system metrics, allowing quick identification of anomalies or performance trends.
Ofers comprehensive logging, monitoring, and data analysis for edge systems, ensuring eficient troubleshooting and root cause analysis in distributed environments.
Delivers full-stack monitoring for edge devices, providing actionable insights on system health, performance, and end-user experiences.
Monitors the health, uptime, and performance of edge devices and infrastructure, ensuring early detection of issues in critical environments.
Ex- Supports the end-to-end testing of AI/ML pipelines in edge environments, ensuring accurate model deployment, data integrity, and consistent predictions in real-time.
Provides automated AI/ML model testing and monitoring, ideal for validating the performance of intelligent systems deployed at the edge.
Simplifies functional, API, and UI testing for edge applications, ensuring robust performance across diverse device configurations. scripts, making it highly adaptable for edge applications that demand specific interaction patterns, such as smart manufacturing or autonomous vehicle networks.
Locust’s flexibility and scalability allow for detailed performance evaluations of edge systems. Its reliance on scripting requires programming knowledge but enables highly customizable testing scenarios.
Moving to network security testing tools, we can suggest Nmap [
Nmap’s scripting engine enables automated scans and custom security assessments, making it highly efective for distributed edge environments. Its command-line interface may be intimidating for beginners, but its versatility and accuracy in network reconnaissance make it a crucial tool for security professionals.
Another useful tool is Shodan [
Shodan’s ability to scan and categorize devices based on their fingerprints makes it invaluable for cybersecurity teams. Its premium features can be costly, but its powerful search capabilities make it an essential tool for identifying and securing publicly exposed edge infrastructure.
Penetration testing is a crucial stage of system’s security assurance flow. Metasploit [
Metasploit’s extensive exploit database and automation features make it a powerful tool for proactive security testing. Its complexity may require a learning curve for new users, but its ability to simulate advanced attack scenarios makes it indispensable for ethical hackers and security teams.
Really useful would be Prometheus [
Prometheus’s detailed metrics and real-time alerts make it indispensable for maintaining edge system reliability. Its complexity can pose a challenge for new users, but its integration with other tools like Grafana enhances its utility.
As an alternative tool for Prometheus, we can use Grafana [
Grafana’s customizable dashboards provide clear insights into edge system performance. It requires time to set up efectively, but its visualization capabilities make it a valuable monitoring tool.
Also, the powerful kit is ELK Stack [
It handles large volumes of log data eficiently, aiding in debugging and performance optimization. Its resource intensity requires robust infrastructure, but its analytical power is unmatched for log-based insights.
One more monitoring tool is New Relic [
New Relic simplifies observability with intuitive dashboards and detailed traces, ensuring edge systems operate smoothly. Its licensing costs and dependency on the internet may limit its applicability for some teams.
For infrastructure monitoring we would use Nagios [
Nagios’s customization options ensure efective monitoring of distributed edge systems. It’s dated interface may deter some users, but its reliability and plugin ecosystem are significant advantages.
When working with artificial intelligence and machine learning systems, it’s important to use specific tools, that are able to process models assessment and validation.
End-to-end platform for deploying and managing machine learning models in production is
TensorFlow Extended (TFX) [
TFX ensures robust model validation and deployment in edge AI setups. Its steep learning curve requires expertise but ofers unmatched reliability and performance for AI-driven edge applications.
Also, it’s worth mentioning DataRobot [
DataRobot accelerates AI development for edge applications, providing robust model validation. Its cost and limited customization may pose challenges for small teams, but its capabilities significantly enhance edge AI operations.
As an optimal test automation tool we would suggest Katalon Studio [
Katalon simplifies functional and API testing for edge applications. While limited for system-level testing, its ease of use and automation capabilities are ideal for validating workflows.
To perform real-time testing on edge computing systems like IoT sensors that require continuous
data streaming it is necessary to use systems that allow to create custom data pipelines and analyze
them. For that porpose we would ofer to use Apache products: Kafka [
Apache Kafka is a distributed messaging system designed for real-time data streaming. It is commonly used in edge computing to facilitate the movement of data between edge devices, processing nodes, and cloud systems. Kafka is particularly useful for building scalable and fault-tolerant edge architectures that handle high volumes of real-time data from IoT devices or edge sensors.
Kafka’s high throughput and scalability make it essential for edge data streaming. Its configuration complexity requires expertise, but its ability to handle real-time data flows ensures reliable edge operations.
Apache Flink is a distributed stream-processing framework optimized for real-time analytics. It supports low-latency processing of data streams, making it an excellent choice for edge applications like predictive maintenance, real-time monitoring, and analytics. Flink’s ability to process massive data streams in distributed environments ensures reliable edge data handling and analysis.
Flink’s low-latency processing capabilities make it a top choice for edge applications requiring real-time insights. Its complexity makes it suitable for advanced use cases and experienced teams.
Today, edge computing introduces challenges, including decentralized operations, time-sensitive data processing, and security vulnerabilities, all of which demand robust testing strategies.
Due to performed bibliometric analysis we can note that current state of edge computing system’s quality assurance field is mostly oriented on IoT, cloud-computing, embedded and open-source systems testing. But the problems of justification and definition criteria for selections edge computing systems’ testing tools are still releavant. At the same time, it is necessary to consider the specifics of modern peripheral devices.
Continuous validation of system reliability, adaptability to harsh environments, and seamless data lfow are essential to maintaining operational eficiency and safeguarding against failures or breaches. These testing approaches collectively ensure that edge systems meet stringent performance, safety, and security requirements, supporting their deployment in mission-critical and dynamic environments.
Burn-in testing and real-time data testing are fundamental methodologies in ensuring the reliability and performance of edge computing systems, particularly in critical applications. Burn-in testing identifies and eliminates defective components through rigorous stress testing, which enhances the durability and dependability of hardware in challenging operational environments. Real-time data testing, on the other hand, plays a pivotal role in validating the integrity, accuracy, and responsiveness of edge devices, ensuring smooth operations in dynamic and high-stakes scenarios.
To provide appropriate QA tools selection, next list of criteria is suggested: scalability, compatibility with edge architectures, integration with CI/CD pipelines, cost suitability, licensing and ease of use. Most of these criteria can be estimated by checking tool’s documentation or by their exploratory use. But the assessment of more complicated and data-driven indicators require expert analysis which can be discovered in future investigations.
So the main task for QA teams is to aggregate full set of tools to be able to provide functional latency, performance, security, fault tolerance, and resilience testing. Also, the tools for monitoring and measurement are required.
Author Contributions: Conceptualization, Viktor O. Maliarskyi and Vasyl P. Oleksiuk; writing – original draft, Viktor O. Maliarskyi; writing—review and editing, Vasyl P. Oleksiuk. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Data Availability Statement: No new data were created or analysed during this study. Data sharing is not applicable. Conflicts of Interest: The authors declare no conflict of interest.
Declaration on Generative AI: While preparing this work, the authors used ChatGPT and Grammarly to improve sentence structure and word choice.