The Internet of Things (IoT) has made huge progress from a generic vision to concrete realisations. IoT systems are becoming more and more ubiquitous and pervasive. Simultaneously, our daily lives depend even more on them, e.g., smart homes, smart cities as well as smart transportation, delivery, and manufacturing processes. By their nature, IoT systems can directly influence the real world and as a consequence their failures can cause harm and even fatalities. Thus, it is important that they are highly resilient and particularly, never enter undefined states.
While IoT systems are conceptually close to distributed applications that have been planned, implemented, and operated for decades, significant differences exist: (i) IoT systems span MeSS'21: International workshop on MDE for Smart IoT Systems, June 21-25, 2021, Bergen, Norway Envelope alexander.hess@uni-ulm.de (A. Heß); franz.hauck@uni-ulm.de (F. J. Hauck); david.moedinger@uni-ulm.de (D. Mödinger); jakob.pietron@uni-ulm.de (J. Pietron); matthias.tichy@uni-ulm.de (M. Tichy); joerg.domaschka@uni-ulm.de (J. Domaschka) Orcid 0000-0001-6837-2861 (A. Heß); 0000-0002-7480-9617 (F. J. Hauck); 0000-0002-5917-2419 (D. Mödinger); 0000-0001-8308-6636 (J. Pietron); 0000-0002-9067-3748 (M. Tichy); 0000-0002-5451-3480 (J. Domaschka) large geographical regions; (ii) they consist of a large number of heterogeneous, even batterypowered, devices with different capabilities, network bandwidths, and failure characteristics; (iii) depending on the region, replacing failed devices and patching broken software may be a matter of days and weeks instead of minutes and hours as with Web-like distributed systems.
Graceful degradation is a known approach to reduce a system's capabilities in a controlled manner [1]. Degradation thus can step in when classical approaches to fault tolerance are unavailable or can no longer mask failures. While classical approaches to resilience, e.g., replication or checkpointing, can be provided by libraries in an application-agnostic way, degradation has to be interlaced with application code. This makes the design and implementation of a degradable IoT system cumbersome, error prone, time intensive, and hard to test and maintain.
To overcome these limitations, we propose Morpheus, a degradation framework for IoT systems as extension of our previous work [2] within the SORRIR project-an internal DSL for IoT systems built in TypeScript, which allows the definition of components, their ports and connections, as well as the behaviour of components using state machines. The contribution of this paper is an extension that supports developers of IoT systems with the integration of graceful degradation into their systems. Morpheus ensures automatic degradation and upgrade of a component, as soon as developer-defined constraints are met. By design, our DSL ensures the principle of separation of concerns, by splitting the definition of a component's behaviour into different operation modes that can serve as degradation levels.
Section 2 discusses related work, while Section 3 introduces previous work and a running example used throughout the text. Section 4 introduces Morpheus' concepts and illustrates their use via the running example. Section 5 concludes and presents future work.
With graceful degradation, an IoT system is able to sustain some functionality when faults can no longer be tolerated by other resilience mechanisms [3]: the application copes with partial failures and adapts. Examples include to retain reduced functionality at the edge in case of loss of connectivity between cloud and edge. The degraded service could even abstain from executing certain tasks. Thus, the degradation can affect function, quality and performance.
Graceful degradation approaches are well known for design and operation of distributed embedded systems [4,5,1], but have recently also been introduced for IoT scenarios, e.g., for a video surveillance application [6], a smart-office case study [7], and a drone application [8]. Degradation often depends on defining a set of different service levels that determine the graduations of quality-of-service (QoS) a system can operate at. This results in an application specific mechanism, as the levels and their interpretation are specific to their application domain.
A degraded system may automatically switch back to the originally desired service level, or at least to a better level, if the cause of the degradation has disappeared or changed. This graceful upgrade is addressed in only few related works [9,4].
Degradation is supported by various modelling approaches such as the Architecture Analysis & Design Language (AADL) and its error annex [10,11]. Bozzano et al. extend AADL to enable the specification of multiple modes for components to define normal and degraded behaviour using event-data automata [12]. The approach supports formal analysis of such models by model checking as well as safety analyses like generation of fault trees. A similar approach has been presented in the area of self-adaptive systems where Borda and Koutavas [13] present a formal approach which supports the specification of adaptation transitions between automata. The approach supports the formal verification of safety requirements by a translation to CSP.
Morpheus builds upon our previous work-an internal Domain Specific Language [2] developed in TypeScript [14]-that covers the two relevant aspects: structure and behaviour. The structure of the IoT system is defined as an architecture covering the most relevant structural elements of an architectural description language based on [15]. Particularly, it provides atomic and hierarchical components, ports and connections. Components interact with one another by exchanging events over connections which connect two ports.
Each atomic component contains internal state consisting of a discrete finite state as in automata and local data structures (similar to the state in abstract state machines [16]) as well as a user-developed TypeScript-function which takes this internal state, the contents of the event queue, and updates state and queue. This basic definition of behaviour is complemented by a framework, which allows the specification of state machines using TypeScript data structures and TypeScript as action language and includes an interpreter for these state machines. Hierarchical components do not have their own behaviour, but aggregate the behaviour of their subcomponents to simplify the design of systems, particularly, the definition of degradation.
Our experience as presented in our previous paper [2] has been very positive. The usage of TypeScript as "meta-modelling" and action language streamline the development. Our result fits very nicely into the overall JavaScript/TypeScript ecosystem. For example, we can easily reuse libraries for communication via MQTT and HTTP as well as logging. Another advantage is our ability to use IDEs and all of their features, including debugging facilities.
In order to demonstrate the capabilities of Morpheus, we implemented a physical prototype1 of a Smart Entrance Barrier (SEB) that could be deployed in a parking garage. Customers of the garage reserve a parking spot through an online reservation system using credit card details and license plate number. The SEB automatically recognises customers on entrance and exit. Our SEB prototype is composed of multiple dependent sub-components: a barrier unit B U , a barrier sensor B S , a car sensor C S , a card reader C R , and a camera C A M . Further, it uses two external software components: a plate recognition service P R S and a parking management system P M S .
When a car approaches the S E B , the car sensor detects its presence and the camera scans the license plate. The image is then sent to the P R S , which extracts the license plate number in text format. This information is used to query the P M S , and if a valid reservation is found, the S E B will open. In this seamless authentication process with multiple interacting components, a faulty component shall not prevent a car from passing the barrier. The prototype is equipped with degradation functionality: Morpheus adjusts the S E B 's functionality based on the availability of internal and external components as detailed in Section 4.
In Morpheus, an operating mode defines a component's functionality. By default, each component implements the operating mode O f f , which specifies that the component is currently unable to provide any reasonable functionality. This operating mode is entered if essential sub-components are unreachable or have failed, or the component itself suffers from internal faults. It is required that each component is equipped with at least one additional non-trivial operating mode, which implements the component's intended feature set. Developers can now define further operating modes that provide a graduated subset of the component's capabilities.
In order to ensure that the specified functionality can actually be provided, it has to be defined which dependant sub-components have to be available. Our example uses the following operating modes for the S E B :
• O f f : The S E B does not provide any reasonable functionality.
The S E B is locked, no car can enter or leave, e.g., to lock the garage.
• O p e n : The S E B is open and allows entrance and exit of cars, e.g., for maintenance.
• M a n u a l : The S E B requires manual authentication with a credit card for every car.
• A u t o m a t i c : The S E B automatically recognises cars by their licence plates, authenticates the user, and authorises entrance.
Table 4.1 presents the pre-conditions for each non-trivial operating mode implemented by the S E B . We assume that all sub-components and external components only implement the two In case either the barrier unit BU or the barrier sensor BS suffer from a mechanical failure, none of the pre-conditions can be fulfilled which leads to the operating mode O f f . In this case only the physical state of the barrier is unknown and has to be inspected by a technician, but the application logic is still in a well-defined state.
Implementation -Framework: A component can be switched into one of the available operating modes either by human operators, software logic, or Morpheus. To realise these features, we first extend the basic type A b s t r a c t S t a t e [2] with an o p e r a t i n g M o d e value, as shown in Listing 1. Additionally, the D e g r a d a b l e S t a t e is equipped with a d e g r a d a t i o n H i s t o r y field that can be used to track the component's degradation behaviour. We further introduced a new type called D e p d e n c y F u n c t i o n that is used to define the preconditions for each operating mode. Essentially, this is a boolean function that takes the components current state and the sub-component's operating modes into account. These are called Shadow Modes since it can not be reliably determined wether an external component is unavailable due to a hardware failure or because of an interrupted network connection. We choose to realize this as a generic function, which evaluates declarative conditions like Table 4.1, while still providing great freedom, e.g., to perform arbitrary calculations.
At any given time, each component 𝐶 has a target operation mode. Under normal circumstances this is the operating mode that provides the full intended feature set (Automatic in our example). Yet, due to external dependencies to other components, 𝐶 may not be able to provide this ideal mode. In this case, Morpheus picks another operation mode for 𝐶 for which all dependencies are met. We refer to this operation mode as 𝐶's degradation level. For enabling Morpheus to select an appropriate degradation level Morpheus requires a developer to specify a directed acyclic graph (DAG) for each degradable component. The graph connects the component's implemented operating modes, such that each of them is directly or indirectly degradable to operating mode O f f . Morpheus selects the next degradation level along the graph that fulfils its preconditions, and executes adjacent transitions. Operating modes that are connected through a path that leads to the operating mode O f f should provide a graduated subset of the component's functionality. Whenever the operating mode of a subcomponent changes, Morpheus first checks if the current operating mode/degradation level can be kept and if not selects a new operation mode to degrade/upgrade to. Further, based on the DAG, it determines the path to reach that state and iteratively degrades/updates along the path from one operation mode to the next until the selected mode has been reached.
For each operation mode associated with its own state machine, a mechanism is needed to not only switch between modes, but also to transfer the state of one state-machine to the state of another. This functionality can be implemented using the T r a n s f e r F u n c t i o n whose signature is provided in Listing 4. While the main task of a transfer functions is to take one operating mode as an input and output another one, it can be used to perform additional tasks. First, it is possible to save a component's internal state on a stack, such that it can be restored if the operating mode is reached later again. Second, the component's degradation path can be recorded and analysed in order to make fine-grained adjustments to the component state. Listing 4: The T r a n s f e r F u n c t i o n is used for the state transfer between two operating modes.
Morpheus now provides the u p d a t e O p e r a t i n g M o d e function, which automatically selects the optimal operating mode for the component based on the defined pre-conditions, the degradation DAG and the availability of the dependent subcomponents. The implementation of this function is provided in Listing 5. Internally, it evaluates the implemented D e p e n d e n c y F u n c t i o n s in descending order. If a function returns true, it is checked whether the degradation level of the assigned operating mode is higher or lower than the degradation level of the current operating mode. If the degradation level is lower, the internal d e g r a d e function is executed, which performs a depth-first search on the degradation DAG, that produces an array of T r a n s f e r F u n c t i o n s . This array of T r a n s f e r F u n c t i o n s is then iteratively processed, and the function finally returns the degraded state. In case no path could be found the state is returned unaltered. The implemen-tation of the u p g r a d e function is quite similar. This function is executed if the degradation level of the new operating mode is higher than the degradation level of the current operating mode. However, it additionally guarantees that an upgrade path is only executed if it leads to the component's target operating mode. This property is especially important if only a partial upgrade can be performed, i.e. if the component can be upgraded, but the target operating mode can not be reached yet. 4.1. If the S E B 's current operating mode is A u t o m a t i c and the conditions are no longer true, e.g., the P R S switched to mode O f f , then the next possible mode is considered in the DAG. Concretely, operating mode M a n u a l is the first degradation level for target operating mode A u t o m a t i c . Further, the operating mode O p e n can only be reached through manual reconfiguration, if it is assumed that A u t o m a t i c is the standard target operating mode. Listing 6 provides a snippet that shows how the T r a n s f e r F u n c t i o n can be used to implement the transition from A u t o m a t i c to M a n u a l . Here, we utilise the d e g r a d a t i o n H i s t o r y to record the component's current operating mode and internal state.
In this paper we presented Morpheus, a framework that aims to assist developers in realising graceful degradation and automatic recovery. Morpheus provides lightweight integration into our previously presented internal DSL based on operating modes of interconnected components.
In our previous internal DSL [2] we provided integrated state-space exploration. Currently, this exploration is not realised for Morpheus, as the automatic recovery algorithm and transfer functions complicate the state-transition exploration. Otherwise, Morpheus provides a functional extension of our previous results. Morpheus allows developers to separate their concerns on the levels of functionality and interdependence of components, while providing clear guidance for realising resilience through graceful degradation and upgrade.
The authors acknowledge the financial support by the Federal Ministry of Education and Research of Germany (BMBF grant nr. 01IS18068, SORRIR). Further, this work was partially funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) -435878599 ; 453895475.