The Internet of Things paradigm arises many issues in terms of privacy and security. Systems that are commonly con gured by personnel with limited experience manage incredible amount of personal data and have direct control over home systems (e.g. controlling home lights or home heating system). The purpose of our research is to de ne a methodology that automates as much as possible the penetration testing actions, in order to help a tester with limited security skills to nd possible attacks and demonstrate them clearly to the home user. The core idea is that we rely on an existing automated threat modeling technique in order to build up the possible attacks to the system under test. The threats are concrete and understandable even to a non-expert, like home users, and help them at identifying real risks and possible countermeasures. The paper will demonstrate the proposed approach over a very typical use case, a smart home controlled through the Alexa Voice Assistant, demonstrating how it is possible to nd a working attack on such a system, using very cheap dedicated hardware and with common tools.
Internet of Things (IoT) is the new paradigm that relies on the idea of widely distributed interconnected objects that constantly cooperate and automate many of the common actions of our life. Vocal assistant and their smart home applications are one of the most clear examples of the paradigms: home users are able to control lights and setting alarms using voice commands in natural language. However, such a paradigm has a side e ect in terms of privacy and security: all data are continuously shared through the local and public networks (e.g. vocal command sent to the voice service for processing, commands sent to devices and so on).
Home systems are con gured and installed by the nal customers (i.e. the home user), typically with limited computer science skills and without any competences in terms of computer security. However, even if a dedicated professional could be involved to con gure the house network, the e ort and cost needed to perform a valid security assurance procedure are too high to be considered in such a context.
One of the possible approach to address such type of issues is to nd (semi-) automated
techniques for penetration testing that helps at identifying the possible attacks and, consequently,
apply the needed countermeasures. Examples of such approaches can be found in [
The purpose of our research, which extends the results in [
This paper demonstrates the proposed approach over a very typical use case, a smart home controlled through the Amazon Echo Plus hub and the Alexa Voice Assistant, demonstrating how it is possible to nd a working attack on such a system, using very cheap dedicated hardware and with common tools. Next Section 2 summarizes the existing penetration testing techniques, outlining the steps involved and the needed competences, while section 3 describes the methodology we propose and its main features. Section 4 discusses our case study and demonstrates a successful attack on to an Alexa-based home system. Finally, section 5 summarizes the conclusions and proposes future works. 2
Penetration testing is mostly an expert-guided activity: despite the needs, as outlined in [
The NIST1 SP-800-115's special publication [
The non-pro t OWASP2 foundation, recently released the OWASP Testing Guide v4 [
The Penetration Testing Execution Standard (PTES) is an open document that has been
designed to provide a common language between the businesses world and security service
providers in order to perform penetration testing activities. It is divided into seven main generic
sections: Pre-engagement Interactions, Intelligence Gathering, Threat Modeling, Vulnerability
Analysis, Exploitation, Post Exploitation, Reporting. Since the standard does not provide any
technical guidelines to execute an actual penetration test, a technical guide have been created
to support the standard. [
The Open Source Security Testing Methodology Manual (OSSTMM) [
Type of tests and security properties to be tested are de ned during the induction phase. The interactions phase lets the penetration tester determine and select the target systems. During the inquest phase, the penetration tester retrieves as much information as possible about the targeted assets. Only in the last phase, security properties and measures are actually assessed. A reporting phase must follow after every test.
The Penetration Testing Framework (PTS)[
The Information Systems Security Assessment Framework (ISSAF) [
The rst phase sets the security objectives to assess and plans the security tests which must be conducted during the second phase. In turn, the second phase can be further divided into the following operational sub-phases: Information gathering, Network mapping, Vulnerability identi cation, Penetration, Gaining access and privilege escalation, Enumerating further, Compromise remote users/sites, Maintaining access, Covering tracks. The third phase covers the reporting process and removes any artefacts leftover from the actual penetration testing stage.
The Metasploit Framework (MSF) [
The above analysis outlines that all existing penetration testing methodologies rely on the security expert experience, o ering guidelines over an experience-based activity. Techniques like OSSTMM, which aims at supporting certi cation processes, introduce quantitative evaluation on the processes, while techniques like OWASP and PTES are more oriented to focus on identifying original attack processes. However, only PTES includes a threat modeling phase. The above-illustrated approaches focus on discovering technical vulnerabilities, instead of relating possible attacks to high-level threats understandable to the end user. The approach we propose, on the contrary, starts from the end-user perception of the risks, aiming at nding a possible attack whose success a ects the clear interests of the system customer. 3
It is provable that all the existing security testing methodologies and, accordingly, all the existing penetration testing techniques cannot grant full completeness and non-redundancy attributes against every possible security threats. Completeness and non-redundancy properties are two qualitative indicators which evaluate the goodness of a security testing methodology. A security testing methodology is said to be complete if it considers all the feasible security threats for a given SuA (System under Attack). It's non-redundant if it does not count not-applicable security threats.
Since none of the existing penetration testing methodologies is nor complete and non-redundant, in recent years considerable e orts have been made to try to obtain more reliable models and techniques.
The methodology proposed in this work evolves the automated penetration testing model
presented in [
This approach enables less-skilled penetration testers to perform a full threat-modelingdriven penetration testing security evaluation, guiding them to look for system vulnerabilities on a per threats-basis. Still, as will be clear further on this chapter, it is needed a suitable way to identify all the threats that are applicable to the SuT (System under Test). Our methodology relies in the following four phases, described in details in next sections : (i) System Modeling (semi-)formal description of the system under test, (ii) Threat Modeling identi cation of the threats, (iii) Planning planning the tests and possible attacks to perform and (iv) Penetration Testing: actual execution of the attacks. 3.1
The proposed methodology entirely relies on the correctness and the accuracy of the SuT model,
which must be built during phase 1 and will be used to drive the following activities. In a
whitebox penetration testing approach scenario, penetration testers have access to the whole system
description and information, whereas in a black-box approach he/she must retrieve those kind
of information by means of suitable scanning tools. In the intermediate grey-box scenario,
penetration testers have access only to a restricted set of information, which must be enriched
by the same information-gathering process as in the previous black-box approach.
Then, is necessary to put the collected information in a suitable representation form, so that
is possible to easily visualize the whole system architecture and the dependencies between
di erent components. According to our approach, the SuT model is described by the
Multicloud Application Composition Model (MACM) formalism, a graph-based system modeling
formalism initially introduced to describe architectural components and security properties of
cloud-based applications [
Based on the enriched system information available through the MACM representation,
penetration testers must identify the threats each component is potentially subjected to and build a
complete system Threat Model, used as a basis for actual penetration testing. However, threat
enumeration and identi cation is one of the most challenging tasks and may result very tough
or heavily time-consuming for the less experienced penetration tester. We use the same threat
catalogue-based approach introduced in [
Because the threat model built in this way could be very broad and not all the threats discovered could be real threats to the SuT, threat model entries could be further quantitatively classi ed through a risk assessment stage. This process aims to assign a risk value to each threat discovered, taking into account likelihood and impact of a successful exploitation. Thus, threats could be sorted by risk value and tests could be planned later on by threat severity. 3.3
In the planning phase, penetration testers select the right test planning schemes from a pre-build knowledge base. This knowledge base is continuously updated with exploitation techniques, described in terms of tools and actions to execute, associated with speci c threats. Each exploitation technique can be used to perform actual attacks. Based on the threat model obtained in the previous phases, the penetration testing planning step is substantially divided into two sub-phases: the selection of the threatened assets and the following threat testing actions planning.
It's worth noting that threats and assets test prioritization could be partially or fully driven by the risk value obtained in the risk assessment stage. In practice, we built up a planning knowledge-base in the form of a relational DB, that contains: asset- and threat- speci c attacks, described in terms of the tools, parameters and actions to be performed to execute the attack; a mapping among known vulnerabilities and known attacks to asset-speci c threats. It's worth noting that insertion of attack information into the DB is a manually-conducted task at the moment. In fact, we continuously enrich the knowledge-base by manually looking at multiple and often not normalized public sources. Though, we expect to automate this activity in the next future. The DB containing the knowledge base is still work in progress and will plan to release it publicly in future
Once selected the targeted component and the security objectives to be tested, penetration testers look for available vulnerabilities and, eventually, set up the appropriate tools or framework to exploit and put into e ect the chosen threat.
The output of this phase is the logical chaining of all the steps, the commands and the environment con gurations required to operate the planned attacks. Table 1 describes the eld attributes of the penetration testing planning knowledge base. Field
Asset Asset Type
Threat Description
Hardware Tools & Frameworks
Actions Associated commands
Notes Compromised asset In order to demonstrate the proposed methodology, we focused our attention on home assistant solutions. Among the various proposals on the market, we chose Echo Plus, the Amazon assistant based on Alexa, which make use of cloud-based voice services to perform actions like answering basic questions or controlling home automation devices. Echo Plus has a built-in hub that supports and controls ZigBee smart devices, such as light bulbs and door locks, which can be bound to the home assistant asking Alexa to "discover the devices". To apply our testing methodology we considered a test-bed composed of three additional devices beside the Echo Plus: a Philips Hue White and a Philips Hue Motion Sensor, respectively a smart light bulb and a smart motion sensor, and Osram Lightify, a smart light bulb equipped with the color change function. The testing environment also includes a penetration testing notebook located in the proximity of the home system network, con gured with the Kali Linux distribution and equipped with a USB interfacing dongle for ZigBee networks.
The communication view ([
Remind that the goal of the proposed methodology is to perform a penetration testing assessment by searching the SuT asset types from a threat catalogue knowledge base and obtain a list of the all associated threats together with the operations to be carried out to implement the actual attacks. 4.1
As already introduced in 3, we use the MACM representation to model the SuA
architecture. To correctly model our speci c testbed we made use of four types of nodes (IoTDevice,
IoTGateway, Network, Service) and two kinds of relation (use, connect ), all already de ned in
[
The goal of the Threat Modeling process is to list all the possible threats for the SuT and,
according to the technique described in [
Asset ZigBee Network ZigBee Network
A ZigBee network is subject to the same typical threats of generic networks, such as eavesdropping and message replay. In addition, a speci c threat of the ZigBee networks is shown in the last row of the table, the "Network Key Discovery" threat, which is the one we chose for the implementation phase. 4.3
Planning phase produces a detailed test plan to check the feasibility of each threat. The threat we decided to test into the planning phase is, as already aforementioned, the ZigBee related "Network Key Discovery" threat. Basically, the threat represents the intruder's capability to sni or inject packets into a ZigBee network by sni ng the private network key that must be shared during the join process and later used to secure the communication on the radio channel. Fig. 3 illustrates the implementation plan trying to exploit the vulnerability that would put into e ect this threat and that relies onto the automation pro le of the ZigBee protocol. The table follows the same structure introduced in 3 and clearly displays the steps and the preconditions needed to perform the actual attack against the ZigBee protocol, having as result the compromise of all the controlled smart devices. It's worth noticing that the attack described later on in the selected plan relies on a quite simple vulnerability. All the ZigBee Home Automation 1.2 devices, including Amazon Echo Plus, must implement a set of security attributes, namely Startup Attribute Sets (SAS). It turned out that Echo Plus utilizes the Default TC Link Key "ZigBeeAlliance09" to set up the same Network Key for every smart device, used in turn to encrypt the tra c owing among the ZigBee network. As consequence, it's su cient to intercept the initial handshake between the Echo Plus and one of the smart devices in order to get the Network Key and be able to read or inject commands to all other controlled devices. In the next subsection, we analyze more in detail the steps needed to the test and carry out the actual attack. 4.4
Looking at the prerequisite listed in Fig. 3, namely the Hardware and Tools & Frameworks columns, we used a CC2531 USB dongle to physically interface the penetration testing laptop to the ZigBee network, in addition to both KillerBee and Wireshark as software layer to capture and analyze ZigBee packets. The Associated commands column lists, per each tool, the commands that must be used.
Initially, it is necessary to insert the CC2531 in a USB port and check, through a terminal, the correct interface con guration with the zbid tool.
Successively, it is necessary to retrieve the correct channel on which the ZigBee network is currently operating, namely the Home Automation pro les preferred channels (one among 11, 14, 15, 19, 20, 24, 25). This can be achieved with zbwireshark, by testing each channel number and looking for tra c. In our testing environment, we found that the channel 25 was used by Echo Plus for setting up the ZigBee network.
To retrieve the Network Key it's necessary to con gure the Default TC Link Key into Wireshark settings. As previously outlined, this is the default key de ned into the ZigBee Home Automation 1.2 (HA1.2) standard.
Once set the Default TC Link Key, it's necessary to induce Echo Plus to start an association towards a smart device. We manually triggered Echo Plus through Alexa, asking it to search for the new devices and set the device into the association mode. The smart device joins the ZigBee network by following the procedure illustrated in Fig. 5. Fig. 4 shows the captured packets ow between Echo Plus and the Osram Lightify light bulb. The Amazon Echo Plus and the light bulb perform beacons exchange to recognize each other, as outlined by packet frames 46 to 50 in Fig. 4. At a certain point, the light bulb (with MAC address 7c:b0:3e:aa:00:ae:3d:20) sends to Echo Plus (with network address 0x0000) an association request. Fig. 6 shows the contents of the association frame.
There is no additional security measure enabled, except the Default Trust Center Key. The coordinator in response to an association request assigns a network address to the new associated device (short address). Lastly, the coordinator sends the Network Key to the new node as in frame 57, Transport Key. Fig. 7 shows the frame 57 contents and in particular the known Default TC Link key and the new Network Key. At this point, using the Network key and importing it into Wireshark is possible to decrypt all the packets subsequently exchanged between the nodes, e.g. intercepting light on/o command or when a di erent color is set. 5
We demonstrated that using very limited resources (a notebook and a USB dongle that cost less than 20 euros), it is possible to systematically test an home-based system, identifying and executing a successful attack against a common voice assistant-based system. Our analysis outlines the high level of risk of products in commerce, that need to address a trade-o among usability and security of the system: the device discovering systems adopted by the vocal assistant, that needs a minimal intervention from the customer, become a security hole that enables any attacker physical near to the house to acquire control over all the home devices, being able to read messages and eventually control the devices. The main consideration is that a securityby-default approach should be imposed to systems with such a large di usion. At the state of art, instead, the solutions o ered have a low level of security in standard con guration and countermeasures should be applied by the end user. Device discovery should rely on an explicit authentication of devices (that implies a manual intervention of users) and communication keys should not be shared among multiple devices. The penetration testing methodology we adopted supports IoT-based systems and enable personnel with limited computer security skills to identify and demonstrate working attacks. This paper is a little step in the direction of a technique that aims at automating the processes related to security assurance and penetration testing, which are, in our opinion, one of the key requirements for the new generation of computing paradigms, where everything is o ered as a service (cloud approach), self-managed (smart systems) and typically ubiquitous (IoT). In future works, we aim at improving our methodology in order to take into account the available knowledge base of attack techniques and tactics (e.g. ATT&CK) and attack patterns3.