Surekli entegrasyon modeli ile birlikte yaz l m testlerinin otomatize edilmesi konusunda onemli yol katedilmistir. Gunumuzde farkl urunler tek bir yaz l m test otomasyon cercevesi icerisinde test edilebilmektedir.Test otomasyon cercevesini benimsemis prosesler dag t m zaman ve kod kalitesi bak m ndan daha efektif olmaktad r. Ancak proje tak mlar guvenlik konusunun test sistemlerine entegre edilmesi konusunda zorluklar cekmektedir. Ayr ca, ayn test otomasyon cercevesini kullanan farkl proje tak mlar n n farkl test senaryolar na sahip olmas da endustriyel test proseslerinde bir sorun olarak gorulmektedir. Bu baglamda kompleks otomatik test ortamlar nda test senaryolar n n gruplan p modularize edilmesi konusunda bir cozum yolu bulunmal d r. Eger test senaryolar otomatik olarak kisilere ozel olarak kosturulabilirse farkl proje gruplar icin farkl test senaryolar kosturulabilecegi gibi ayn proje icerisinde is bolumu yap larak ayn ekipte cal san farkl kisiler taraf ndan farkl test senaryolar da kosturulabilecektir. Bu cal sma baglam nda onerilen kisi bazl test otomasyonu farkl endustriyel projeler icin Siemens ozelinde kullan lan bir endustriyel test otomasyon cercevesi icerisinde gerceklenip incelenmistir.Surekli entegrasyon dongusu biyometrik yuz tan ma ile tetiklenerek otomatik testlerin kisilere ozel olarak cal st r lmas hede enmistir. Tetikleyici olarak biyometrik sinyal kullan ld g icin test senaryolar farkl kisiler icin gruplan p modularize edilebilmekte ve sistemin genel guvenliginde de gelisme gorulmektedir.
In the innovation era, companies and individuals are always in competition
nowadays. To achieve success in this competitive world, software development teams
and projects need to be adapted to the rapid change. Requirements, needs and
demands are now changing more frequently. In order to serve those highly
alternating needs the agile manifesto [
In agile teams, the members might be distributed to the di erent locations. Teams work remotely and they need to work productively with high communication among themselves. The common practice is to use a remote platform to run codes and tests on, in order to avoid any locality problems. Any change in the code is committed to the repository, build and run. After a successful build, a set of tests starts to run. At this point the idea is to focus on how well those tests can be separated, divided into groups and run securely.
At the start of the project, the main focus was security. A test system needs to be secure, well protected and the rules need to be well de ned, clearly separated. Authentication systems usually work with username and password combinations which are known as credentials that the user knows. This is the most commonly used authentication method. However, this type of authentication starts to give its place to biometric authentication mechanisms, which authenticates users with their ngerprints, iris, face, voice or even with their behavior. Among these types of biometric authentication methods, the easiest and cheapest one to implement is face recognition in the industrial application eld. It does not require any complex hardware such as iris scanner and its data is easier to collect and train. In this project, authentication based on face recognition is chosen with the same reasons that collaborated. In industrial testing environment, face recognition seems to be easier to implement, and only needs basic camera hardware which most of the test computers have it already. After getting authenticated by the face recognition algorithm, a test user can view and run the test cases that he/she is authorized to. In other words, the face recognition is used as trigger mechanism in the solution presented here.
Continuous Integration (CI) is a common software development practice. It
motivates developers to commit their code early and often. The purpose of the
practice is to eliminate the need for unpredictable and long integration e ort
before the release time by integrating the software with a constant frequency
by di erent developers. CI needs higher degree of automation of code compiling
and testing activities since it goes through the same cycles many times during
a development of a product and these repetitive phases needs to get automated.
CI enables more frequent releases and this improves the companies' ability to
compete. Another bene t of CI is the early feedbacks which helps to ll the gaps
between what the customer values and what the developers o er. Early and
frequent feedbacks enable developers and testers to detect and x the defects
sooner [
In a CI process, developers' changes in the software trigger a set of automated
activities. These activities aim to prove that the changes successfully integrate
with the existing software. In this process, developers need to check in their
changes into a central control system where the changes are monitored by the
CI system. In the CI system there is a preset integrity criterion for the software
and each increment in the code is tested against it and presents feedback related
to the result. This feature enhances the quality of the product and noti es the
developers early with the feedback. Continuous delivery is the extended case of
CI where the software is deployed to production automatically after passing all
the tests [
A simple work ow can be as follows. After an increment in the code is checked in to the version control, the CI system monitors the changes which results in triggering an automated work ow called CI build. CI build rst compiles the source code. Then, CI system could run unit tests in order to verify the compiled code's quality. The result is reported by the CI system by e-mail to the developers who made changes in the increment. However, in practice more automated testing is required. To improve the quality in addition to static code analysis, automated integration and acceptance tests are performed by the CI system.
In order to achieve high test coverage in automated tests considered in the
CI system, a lot of work is required to construct stable automated tests. Having
unstable automated tests could lead to test results being ignored by developers.
And consequently they might miss the points where a x is required. Moreover,
having manual tests instead of these automated ones will take huge amount of
time and slow down the CI cycle [
This paper is organized as follows: Section 2 details the overall system overview. Section 3 explains deployed facial recognition algorithm for person identi cation where Section 4 presents test automation framework used for Siemens products without revealing the inner details. System integration is presented in Section 5, results for a sample project is given in Section 6. Section 7 is the conclusion part where we summarize our proposal. 2
Typical continuous integration cycle begins with the check in of the source code to be tested. The check in of the source code and hence start of automated test scripts is triggered by a trigger which can be either prede ned, or time scheduled or even can be done manually by a team member. In the case of manual trigger by a team member, a question arises, "Should any team member be able to execute all automated tests?". Traditionally, automated testing meant testing the end to end execution ow on the user interface level. However, agile test automation pyramid which is developed by Mike Cohn, suggests that test automation should be divided into three levels. Unit tests should form the base of this pyramid. The service level tests form the next layer and nally the user interface tests form the apex. In this concept, unit and service level tests form the major part of the test automation strategy. Hence, automated test scripts can be categorized under di erent test case suites. Furthermore, a test automation framework might be used for testing di erent products. Since every product has di erent functionalities, their test case suites can also be di erent. For example, one product might have cloud connectivity whereas another product has not. In this case, it doesn't seem plausible to run cloud test scripts on the product which has no cloud connectivity requirements. Considering a common test automation framework is used by every team, then giving di erent user rights to di erent team members seems logical. Here, di erent user rights would determine which test scripts are allowed to be executed.
There is also security aspect on the test automation. Currently, test computers in industrial test environments running automated test scripts are protected by simple means of user name and password authentication. However, an attacker may crack username and password in a likely scenario. Hence, forming an authentication scheme based on biometric features of de ned team members would undeniably increase overall security of the test environment. Such system is hard to break in because authentication is based on a biometric feature which is hard to copy. In this paper, we provide a bio inspired continuous integration trigger based on face recognition.
It is possible to nd di erent bio inspired identi cation methods in the
literature. Bebis et al. [
In our proposed system, multi task cascaded convolutional networks [
For the scope of the paper, it is necessary to implement a face recognition
algorithm with a strong real time performance in practice. The application must
try to nd human face in front of a camera than try to match it with the stored
images in the system using deep neural networks. Hence, face detection
algorithm should run smoothly before face recognition. In the literature, there are
various proposals for face detection and face alignment. Viola and Jones [
As mentioned in "System Overview" chapter, multi task cascaded deep
convolutional networks are utilized in the scope of the project for face detection and
face alignment without compromising performance. Yandong et al. [
Industrial communication is the backbone of modern automation solutions. The communication networks and products involved allow totally integrated communication between the widest possible variety of automation components and devices. And our test automation framework is the name of an entire family of communications networks and products from Siemens. The various networks meet the widest possible range of performance and application requirements in automation engineering.
Our framework provides solutions for individual customer requirements in industrial communication. The communication networks and products are tested by the aforementioned test framework. On this basis, branch speci c automation solutions can be implemented with comprehensive and highly integrated communication functions. The test framework simpli es the commissioning of automation systems regardless of the communication networks and products used.
In terms of their performance and range of functions, the communication networks and products of the framework can be represented in the form of an automation pyramid. The automation pyramid can be divided into three levels; eld, cell and management level. The eld level is where process or communication is handled. For this level, the framework o ers PROFIBUS DP and the AS-Interface. At the cell level, the acquired process data is distributed to the various automation systems or PCs for operator control and monitoring. In this level, the communication networks Industrial Ethernet and PROFIBUS are used by framework. Higher-level management functions are handled, process data is saved, processed further, or used for analysis by the management level. For such tasks, Industrial Ethernet is suitable as the communication network.
Furthermore, our test framework includes many other interfaces such as OPC - which is a standard interface to communicate between numerous data sources, including devices on a factory oor, laboratory equipment, test system xtures and databases. Although, our test automation framework covers variety of projects and therefore test case suites, the one of the most commonly tested functionality is OPC standards and speci cations. The OPC Foundation de ned a set of standard interfaces that allow any client to access any OPC-compatible device using a protocol now referred to as Classic OPC (OPC COM). This protocol utilizes the Microsoft based COM/DCOM technology to provide standard speci cations for data access (DA), historical data access (HDA) and alarms and events (AE). Although basing a protocol on this technology made sense in the 1990s, Classic OPC has several limitations because of this reliance on the Microsoft Windows platform, in the form of security issues and platform dependency.
OPC Uni ed Architecture (UA) is a new communication technology standard which was rst released by the OPC Foundation in 2006 as an improvement upon its predecessor, Classic OPC. OPC UA includes all of the functionality found in Classic OPC. This is done by bringing together the di erent speci cations of Classic OPC into a single entry point to a system o ering current data access, alarms and events, combined with the history of both. Furthermore, OPC UA is based on a cross-platform, business-optimized Service-Oriented Architecture (SOA), which expands on the security and functionality found in Classic OPC, instead of the Microsoft-based COM/DCOM technology. OPC UA supports two protocols: a binary protocol that employs minimal resources, allowing for easy enablement through a rewall; and a Web Service protocol (SOAP) which uses standard HTTP/HTTPS ports. Because of the bene ts of this new protocol, an increasing trend of industrial applications have adopted the UA protocol both in the traditional OPC-centric industrial automation space and emerging areas, such as energy.
Test automation triggering operation has been used for OPC interfaces. Along with other test cases, OPC part has been divided to three roles as admin, diagnose and standard for testing team to check COM, OPC UA, OPC DA, Alarms and Events (AE), Historical Data Access (HDA), Pro net, and Ethernet. In the conclusion section, exemplary results for a continuous integration cycle of an OPC project are presented. 5
Figure 1 depicts overall system schematic. Camera represents test PC's
camera and takes test user's facial properties as input and sends them to face
recognizer. Face recognizer algorithm is implemented with Python script
language. Python language is chosen as the development language because it is
very lightweight, easy to learn and has a good community support.
Furthermore, Python SDK has open source support for image processing, machine
intelligence, scienti c computation libraries such as OpenCV, Pillow,
TensorFlow, SciPy. Hence, face recognizer script has dependencies on aforementioned
libraries. The script runs on test PC and takes facial properties from camera
view via OpenCV library functions. Pillow and SciPy library functions such as
resizing, cropping, convolution, ltering are used for processing captured
images and implementing multi task cascaded convolutional network algorithm.
The core concept at the heart of face recognition algorithm is training Inception
Resnet model. Parkhi et al. [
After test user's successful identi cation, an SQL query is made to the database for user's rights and role. The database is SQL based and it holds user ID, name, role, rights elds in the corresponding table. In this concept, di erent user roles such as admin, diagnose, standard, remote access, OPC UA, DA, COM, alarms and events (AE), historical data access (HDA), Pro net, Ethernet, S7-communication can run subset of test case suites in our test automation framework. The aforementioned test automation framework is running on .NET framework and it enables automated testing of entire SIMATIC family of industrial communication networks and products from Siemens. Therefore, this common automated test framework consists of di erent test case suites such as OPC, S7 communication, Pro net tests. And with our proposed solution, different project team members can trigger automated tests in their continuous integration cycle speci c to them based on facial recognition. The proposed system also addresses security concerns in continuous development by suggesting a bio inspired continuous development trigger. 6
In this section, we run our proposed framework for an exemplary test scenarios. An OPC based product is chosen as example product under test, because of OPC' s wide popularity and trend towards it in the industrial automation community. We de ned 6 user roles namely "Remote Access", "View Controller", "Server Controller", "Alarm Controller", "Diagnostic", and "Admin".
All of the roles are derived from Admin role. Admin has all rights for running each test cases. Remote Access role is created for only running the remote test cases which are under Pro bus and Ethernet. View Controller role is created for View Controller people who are controlling OPC Uni ed Architecture (UA), Data Access (DA), and Alarm views in this system. And they will be responsible people for running view related test cases. Server Controller role is created for running server based test cases. These test cases are related with Classic OPC (OPC COM) and OPC Uni ed Architecture (OPC UA). Alarm Controller role is created for running Alarm test cases. Diagnostic role is created for only running the Diagnostic View test cases which are de ned under Data Access (DA) and Historical Data Access (HDA). Then, we assigned these user roles to di erent test users with user name, user ID and access rights. Hence, our local database and users table is designed according to our test users information. If the user is veri ed by facial recognition, then an SQL query is made to this database, and corresponding test user role is returned. According to returned user role, test user can only run subset of all test scripts if any user role is assigned to veri ed facial recognition. Figure 2 shows which test cases can be run by the de ned user roles. In this article, we proposed a person based continuous integration system which is triggered by bio inspired facial authentication for industrial test processes. Although, any real time person identi cation technique can be used, face recognition seems to be most suitable choice based on current industrial test environments. Siemens speci c test automation framework is combined with face recognition implementation to address automated test security concerns along with possibility to run person based test cases. We demonstrated exemplary automated test results in a single continuous integration cycle with di erent users for a chosen OPC based project.