In this paper a model for solving the problem of estimating the security level of mobile applications was proposed. The estimation is performed based on a fuzzy inference system of the Mamdani type. The input criteria were defined as the most important security threats by applying the Analytic hierarchy process method. The pairwise comparison matrix was constructed from mobile security research on OWASP Top 10 Mobile Risks. The proposed methodology can be applied for any kind of mobile applications available for modern platforms, except specific cases when security analyst does not have a sufficient amount of information about the chosen application for performing the security level testing. Mobile security analysts can easily make further decisions about comprehensive mobile application security based on the results obtained with the help of the introduced model.
The number of available applications for mobile platforms reached 2 million on
Apple AppStore and over 2.2 million available on Google Play Store [
Taking into account the mobile threats growth during the last few years and a visible lack of comprehensive testing techniques it is possible to state that mobile application security evaluation continues to be an urgent problem. Therefore, the goal of the given research is to analyze modern security methodologies and develop a new approach which would help security analysts to estimate the overall mobile application security level. The goal is to be reached in three major steps:
1) Firstly, defining most important vulnerabilities, which presence significantly
impacts the overall application security. By applying the Analytic Hierarchy Process
(AHP) to matrix constructed from expert analysis data published by OWASP [
2) Secondly, constructing a fuzzy logic model for estimation of the security level of mobile application.
3) Finally, creating software testing tool which implements aforementioned algorithm and analysis of the received results.
The rest of paper is organized as follows. Section “State of the Art” introduces the
background literature. It refers to the current scientific advances in mobile application
security and modern techniques and challenges concerned with assessment,
arrangement, and maintaining sufficient security of mobile applications, essential to avoid
major intruding attacks. Based on performed literature analysis it is possible to define
the main task of this work. Section “Methods” introduces main methods used for the
construction of the proposed model. The main instruments described here are the
Analytic Hierarchy Process, which uses the mobile security threats proposed by [
At the present, the well-known IT-companies such as Hewlett Packard, Gartner,
OWASP, McAfee and Pradeo publish numerous reports on mobile application
security. Experts working in this area focus their attention on the problem of various mobile
applications security issues. Hewlett Packard Enterprise report [11] places the
emphasis on the privacy issue. According to them, 97% of analyzed apps have access to at
least one private data source, and such massive data collection could potentially go
awry. The research team based their conclusions on a binary analysis of over 36,000
unique iOS and Android applications. McAfee’s Strategic Intelligence researchers
provided insights about modern mobile malware [
At the same time, business and academic community work together in order to
develop automatic tools for mobile apps security testing. The research by [
The team of researchers introduced a tool “Autoforge”, that can automatically send
requests from the client app to check whether the server side handles user data with
proper security measures [13]. It analyses server responses by imitating requests from
the client side. They performed an experiment on 76 most popular applications (more
than million installations) and found that 65% of apps back-end servers have login
credentials brute-force vulnerability. Another software tool (iMPAcT) performs
automatic testing based on analysis of UI Patterns, which is suitable for Android apps
with graphical user interface (GUI) [
The research conducted by [8] proposed a method to analyze the functionalities related to privacy of client’s information. The testing process is described using UI Automator Viewer, which is a part of Android SDK Tools. A comprehensive study by [9] demonstrated malware detection techniques based on sequence alignment algorithms. By applying proposed methods, it is possible to successfully identify and neutralize malware before it performs attacks. Authors of [15] recommend 40 possible ways to perform penetration testing; it describes software products for mobile security testing such as Burp Suite, Wireshark, MITM proxy and Androguard. The book focuses on iOS and Android platforms, but introduces basic testing methods for BlackBerry and Windows mobile platforms.
Another team of researchers announced a tool (K-Android), which is based on the K framework to test the effect of app-collision [10]. This condition can happen when several applications exchange data whereas they have an access to private information that could not be obtained per se. K-Android performs checking of APK-files, based on static security analysis methods. Netcraft’s research [16] on mobile security testing shows the required steps of comprehensive security threats analysis and the potential list of vulnerabilities. They focus attention on applications that use SSL encryption but have failed mechanisms to validate SSL-certificates, thus making them a potential target for the man-in-the-middle (MITM) attacks. In addition, they advertise The Mobile App Security Testing service performs security testing according to PCI DSS v2.0 requirement 11.3. The book [17] provides a list of various tools to perform penetration testing for Apple iOS developers. He describes iOS built-in security model, device Jailbreaking and TOP 10 Mobile security shortcomings identification and prevention methods. Most tests described in this book focus on static and dynamic analysis, penetration testing, and successful threat assessment.
OWASP Mobile Security Testing Guide proposed three major steps in security analysis, which are: Intelligence Gathering (security analyst gather as much information about application as possible to conduct further research), Threat Modeling (identifying threats by using published security reports and proposing countermeasures to prevent these threats) and Vulnerability Analysis (threats identification using test-cases) [18]. Vulnerability Analysis includes three broad classes of security testing methods: static methods (source code analysis), dynamic methods (network communication monitoring, analyzing running application) and forensic methods (app artifacts analysis, such as databases, log files, cookies, etc.).
The aforementioned tools have been developed based on the methods and
algorithms of various theoretical frameworks. The methods applied can be classified as
experience-based, simulation-based, and methods of mathematical modelling. Thus,
the experience-based approach implies the execution of multiple tests on the app
functioning and analysis of the obtained results. For instance, [
The simulation-based approach presupposes the intended injection of the malware messages/code/app into the mobile device. Then the estimation of the mobile app is performed with respect to its security under the created conditions. For example, the developers of the “Autoforge” software made it automatically generate a new input message with mutated fields satisfying the cryptographic constraints in a black-box manner [13]. The authors have analyzed 3 measures of message field differences (pattern matching, content matching, and the degree of difference – the Levenshtein distance calculated by the Wagner-Fischer algorithm). The difference occurs between a traced and forge messages when that last is simulated by the “Autoforge” in order to estimate the app security. Mathematical modelling methods provide the complex processing of security-related data collected during experiments with the targeted mobile app. Vidal [9] use the non-parametric Wilcoxon signed-rank test that operates on paired data vectors of the scores resulted from aligning the monitored sequences of system calls with the legitimate sample. This ranking method of Decision-making theory allows to detect the malware apps on mobile devices. The study performed by [19] suggests the probabilistic models to determine the set of relevant vulnerability detection rules and the geometric weighted average of frequency and impact scores of each threat in order to estimate their risk.
So, the topic of mobile apps security is broadly discussed by both industrial and scientific community. The given review of reports and research articles allows to state particular solutions of the mobile security problem have already been proposed. The considered works are mainly focused on separate threats detection and analysis. However, the authors do not pay much attention to the measurement of separate security threats identified in various models. Moreover, the problem of comprehensive assessment of the overall security based on the estimates of definite threats is not completely researched. In order to fill this gap, the given research considers the basic security detection principles and commonly known threats and proposes a new approach for estimation of mobile apps security level based on the fuzzy inference system (FIS) approach combined with the Analytical hierarchy process. 3
In order to construct the list of most valuable mobile security threats, the Analytic
hierarchy process (AHP) method was chosen. This method was developed by Saaty
[20]. AHP is the most widely used decision-making methodology that researchers use
in a huge number of fields [21]. Within this work, the AHP is applied to the matrix
which was constructed from expert analysis data presented by [
Insecure Authentication means that a mobile application contains security breaches in client-side or its server-side authentication mechanisms that can be employed by malicious users (attackers). Reverse Engineering means that the attacker gets the access to the app installed from the store and tries to investigate it using own software development tools. Insecure Data Storage occurs when mobile application stores critical or confidential customer information without encryption techniques (known as plain text); in this case, the third party could bypass system protections and obtain access to this private data. Code Tampering vulnerability may lead to serious sensitive data leakage, loss or system damage. Insufficient Cryptography could happen when developers use compromised or outdated encryption algorithms in their applications; an attacker could easily bypass them and the outcomes will be the same as storing sensitive data without any encryption. Improper Platform Usage leads to misuse of mobile operating system security controls. Insecure Communication contains a wide variety of potential threats. This condition could happen in case of improper configuration of encryption techniques (for example, SSL-certificates generation and validation), usage of outdated frameworks or libraries on a client and server-side, and enabling unnecessary access permissions, error logging or debug functions. By using this vulnerability, an attacker could obtain access to app databases and back-end servers. Client Code Quality exposes application to analysing, reverse engineering and source code modifying attacks and techniques. To avoid this, mobile applications must utilize solutions to detect source code modifications at runtime and prevent further access. The vulnerability could lead to data leakage as well as source code piracy. Extraneous Functionality assumes outside analysis of the mobile application and does not require involvement of end users. 1. Firstly, presenting the original problem in the form of a hierarchical descending structure. On the top of it, there is the overall goal of the problem, the next level – various criteria, then sub-criteria. The lowest level of the hierarchy contains all alternatives [22]. 2. Secondly, introducing experts opinions for paired comparisons at each hierarchical level. 3. Lastly, processing matrices of pairwise comparisons for finding local and global priorities [20].
To build comparisons matrix, a special scale of importance of weights of criteria/alternatives is used [20]. After the comparison of alternatives is completed, the weights of criteria importance should be calculated, similarly, by applying pairwise comparisons. The problem has N criteria and M alternatives. In the given case, it is necessary to construct N judgment matrices of M × M size and a single judgment matrix of N × N size. The final decision matrix of priorities AiAHP is determined by [23]:
AiAHP = ∑nj =1aij w j ,i = 1,..., M .
The problem of defining security level of mobile applications cannot be properly described using Boolean logic truth-values (which may only be “true” or “false”). It will not allow to make any meaningful conclusions about further necessary steps to improve the security of the selected mobile application. In our case, more accurate assessments could be made by presenting the results mapped on a spectrum derived after reasoning from inexact or partial knowledge (for example, a list of security vulnerabilities provided in the research). Hence, fuzzy logic can be applied for describing the aforementioned problem.
For the implementation of the proposed model for estimation of the security of mobile applications, a Mamdani-type fuzzy inference system (FIS) was used. Figure 1 features a set of steps mandatory in the Mamdani algorithm [24]. (1) (2) The problem should be presented as a set of rules:
F x= A ( AND / OR) y = B THEN Z , where Z is resulting condition, for example, Z = “unsecure”; A , B are fuzzy inputs and Z is a fuzzy output.
At the rule evaluation stage, to estimate the disjunction and conjunction of rule antecedents the following formulae should be applied: µA B = max[µA(x), µB(x)],
Among multiple defuzzification methods, in this work Centroid (centre of gravity COG) method was chosen [25]. This method finds the point where a vertical line would split the aggregate set into two equal ones. The following formula is used to calculate Centroid of the fuzzy set A on the interval [a, b]:
∫bµA(x) Xdx COG = a ∫abµA(x)dx (5)
The model requires a set of input variables, which are the corresponding numbers
of most important security threats found by a mobile security analyst. An analyst
should perform the manual security threats analysis of the source code and executable
file of a mobile application for the presence of potential security threats, which
description is presented in security reports such as Top 10 mobile risks [
According to the expert analysis data provided by OWASP [
In order to define the most valuable threats the AHP method was applied and the following hierarchy was developed (Fig. 2). In this hierarchy the target is to define the priority of security criteria (CRIT_PRIORITY); criteria - Exploit probability (EXPLOIT), Prevalence (PREVAIL), Detection probability (DETECT), and Technical impacts (IMPACT). Finally, the alternatives are the security risks defined the OWASP list (Table 1). In case of this hierarchy, they are Insecure and Authentication (AUTH), Reverse Engineering (REV_ENG), Insecure Data Storage (DATA_STOR), Code Tampering (CODE_TAMP), Insufficient Cryptography (CRYPTOGR), Improper Platform Usage (PLAT_USE), Insecure Communication (COMM), Client Code Quality (CODE_QUAL), and Extraneous Functionality (EXTR_FUNCT).
The next step is to construct matrices of pairwise comparisons for each security criterion. The intensity of importance was set according to the expert data (Table 3). The results of this step are presented in Tables 2-5.
The most important prior alternatives are Insecure Data Storage (DATA_STOR), Broken Cryptography (CRYPTOGR), Insecure Communication (COMM), and Improper Platform Usage (PLAT_USE).
Matrices of pairwise comparisons were constructed based on single expert opinions. In order to solve the assigned task, additional experts may be engaged, but there will be the problem of reconciliation of various experts’ results. Besides, specifics of the task do not necessarily require different expert opinions that is why matrices were constructed using expert opinions obtained from one expert. Based on the found security alternatives it is now possible to proceed to the model construction step. To develop mathematical models using fuzzy logic, Mamdani algorithm was selected.
The model was constructed using the following list of fuzzy variables: 4. Let’s consider the input fuzzy variables – DataStor, Comm, Cryptogr, and PlatUse – which define the number of security issues per the corresponding category (Insecure Data Storage, Insecure Communication, Insufficient Cryptography, and Improper Platform Usage). 5. Let’s consider the output fuzzy variable SecurityLevel. This variable is used to describe the overall mobile application security level. According to obtained value from this variable, a security analyst can make further decisions about the overall mobile application security and necessity of additional actions. 6) Let’s consider the fuzzy rules defined in Table 7.
The following scale of security level (Table 8) was introduced to interpret the output model results properly.
As a result, the model receives input variables (a number of security threats in the chosen mobile application) and after performing fuzzy logic calculations displays an output as an overall security level of the mobile app. Based on the model designed in the previous section, the real-time software has been built for defining security level of mobile applications. After that, a research on several open-source applications available on GitHub was conducted. The characteristics of the selected apps are presented in Table 9.
The analysis results (defined security level) were obtained using the software based on the developed model (Section 4). The number of security issues per each category and corresponding overall security level are presented in Table 10.
According to the provided results, it is possible to say that the mobile application “Gobandroid” is significantly risky and requires additional work to improve its security while the app “Shaarlier” could be safely used without additional security improvements.
App TVHeadEnd Xabber Classic Shaarlier TripleCamel AndroPTPB Quick Dice Roller Gobandroid MineSweeper 5 7 1 2 2 1 7 6 5 6 1 1 4 4 8 7 5 6 1 3 3 3 7 7
The review of sources on the topic of mobile apps security showed that the
problem of model design for the comprehensive assessment of the overall application
security is still relevant and requires a solution. In this research, a model for defining
overall application security level was proposed. This model was developed as a next
step to the OWASP [
The model is based on the third-party mobile security threats analysis of the
OWASP Foundation [
In this research, a model for estimating the security level of mobile applications has been proposed. A real-time software system has been developed to implement the suggested model. The experiments were conducted on a set of open-source mobile applications. This model also allows narrowing the range of potential vulnerabilities, which presence should be checked by the analyst in particular mobile app. This step allows reducing significantly the amount of work which mobile security specialist should perform to evaluate the level of mobile application security. It was achieved by defining which classes of security threats pose the most significant risks to the overall application’s security level.
The main contributions of the paper are: 1) An efficient method for estimating the security level of mobile application. 2) A set of most valuable mobile security threats. 3) The developed real-time software system which allows performing a security testing of mobile applications. In future, it is planned to make several improvements to this model and software product. The model has a great reusability potential – it is possible to modify easily the classes of security threats, according to threats list updates published by different software security companies. Additionally, it is possible to improve functionality by adding scanning methods, which could perform checks of threat presence in a source code of mobile application and automatically calculate inputs for model without the manual work of a security analyst. Moreover, the efficiency of the model can be advanced by redefining the list of security issue categories according to new researches which will be published in future. 7 8. Kaur S, Singh DK Mobile Application Testing based on Privacy information Encoding
Mechanism (2016) 9. Vidal J M, Monge MAS, Villalba LJG A novel pattern recognition system for detecting Android malware by analyzing suspicious boot sequences. Knowledge-Based Systems, 150, 198–217 (2018) https://doi.org/10.1016/j.knosys.2018.03.018 10. Asǎvoae IM, Nguyen HN, Roggenbach M, Shaikh SA Software model checking: A promising approach to verify mobile app security-A position paper. In Proceedings of the 19th Workshop on Formal Techniques for Java-Like Programs, FTfJP 2017 - Co-located with ECOOP 2017. Association for Computing Machinery, Inc. (2017) https://doi.org/10.1145/3103111.3104040. 11. Hewlett Packard Enterprise Mobile application security report (2016) http://h22168.www2.hpe.com/docs/capgemini/Mobile%20Report%20ver%2010.2.pdf. 12. Hewlett Packard Enterprise (2013) Mobile application security study. 2013 report.
http://files.asset.microfocus.com/4aa5-1057/en/4aa5-1057.pdf. 13. Zuo C, Wang W, Wang R, Lin Z Automatic Forgery of Cryptographically Consistent Messages to Identify Security Vulnerabilities in Mobile Services. Internet Society (2017) https://doi.org/10.14722/ndss.2016.23146. 14. Kim H, Yeo H, Hwang HJ, Ramos CH, Marreiros G. Effective Mobile Applications Testing Strategies (2016) 15. Meng H, Thing VLL, Cheng Y, Dai Z, Zhang L A survey of Android exploits in the wild. Computers and Security, 76, pp 71–91 (2018) https://doi.org/10.1016/j.cose.2018.02.019. 16. Netcraft Mobile app security testing (2018) https://www.netcraft.com/securitytesting/mobile-app-security-testing/. 17. Wilhelm T Professional Penetration Testing: Creating and Learning in a Hacking Lab:
Second Edition. Elsevier Inc. pp 1–445 (2013) https://doi.org/10.1016/C2012-0-00443-7 18. Stahl F, Stroher J Security testing guidelines for mobiles apps. (2013) https://www.owasp.org/images/0/04/Security_Testing_Guidelines_for_mobile_Apps__Florian_Stahl%2BJohannes_Stroeher.pdf. 19. Sadeghi A, Esfahani N, Malek S Mining mobile app markets for prioritization of security assessment effort. In WAMA 2017 Proceedings of the 2nd ACM SIGSOFT International Workshop on App Market Analytics, pp. 1–7. Association for Computing Machinery, Inc. (2017) https://doi.org/10.1145/3121264.3121265. 20. Saaty TL Decision making — the Analytic Hierarchy and Network Processes (AHP/ANP). Journal of Systems Science and Systems Engineering, 13(1), pp 1–35 (2004) https://doi.org/10.1007/s11518-006-0151-5. 21. Vaidya OS, Kumar S Analytic hierarchy process: An overview of applications. European Journal of Operational Research, 169(1), pp 1–29 (2006) https://doi.org/10.1016/j.ejor.2004.04.028. 22. Kousalya P, Mahender Reddy G, Supraja S, Shyam Prasad V Analytical Hierarchy Process approach – An application of engineering education. Mathematica Aeterna, 2(10), pp 861– 878 (2012) https://doi.org/10.1016/j.jvir.2016.06.036. 23. Triantaphyllou E, Mann SH Using the Analytic Hierarchy Process for Decision Making in Engineering Applications: Some Challenges. International Journal of Industrial Engineering: Theory, Applications and Practice, 2(No. 1), pp 35–44 (1995) 24. Fuzzy inference systems.
http://www.cs.princeton.edu/courses/archive/fall07/cos436/HIDDEN/Knapp/fuzzy004.htm 25. Mamdani fuzzy model (2015) http://researchhubs.com/post/engineering/fuzzysystem/mamdani-fuzzy-model.html.