Safe Management of Software Con guration

Abstract

We tend to write software in a parameterized way with parameter values speci ed in con guration les. Such con gurability allows us to deploy software in a context not initially thought of, but it also has the downside that it introduces a new class of hard-to-track faults. This issue arises because the use of con gurations in programs is not integrated into con guration management needed by administrators. We propose a light-weight speci cation language to be used by both parties. From this speci cation we generate con guration access code that includes compile-time checks. Furthermore, we use the same speci cation to add run-time checks for safe con guration management. We expect that our approach averts many failures con gurable software faces today. Additionally, we think it improves the quality altogether, because the documentation resulting from the speci cation leads to a better understanding of the overall system.

-

Today many behavioral aspects of applications are not xed at compile-time, but are determined at run-time by examining con guration les, environment variables, and command-line arguments. Even for an average software system the con gurability is complicated due to a huge number of possibilities, constraints and dependencies.

As studies revealed [ 8 ], [ 13 ] much time and money is wasted because of conguration errors. Miscon gurations are one of today's major causes of system failures. Faulty con guration les sometimes trigger crashes and make services unavailable. These problems lead to downtimes, severe outages and a frustrating process of debugging con guration problems.

The state of the art in software con guration is to use schemata to describe the data in the key databases (they facilitate access to software con guration) and type systems to describe the corresponding variables in the programming languages. These two worlds are disconnected. We think that this gap causes most of these failures.

A recent paper supports our view and argues that users are not the ones to blame for miscon guration [ 12 ]. The authors found evidence that the dataow path from variable initialization to variable use contains many potential errors. We think that these errors are only the symptom of the state-of-the-art development. We are positive that consequent use of a software con guration speci cation mitigates these issues.

In the thesis discussed in this paper we design and study a novel speci cation language and its integration in a key database. The speci cation should include constraints for both the key database and the variables in order to achieve following bene ts and goals: { Provide safe use of variables within programming languages containing values of con guration les by exploiting compile-time checks using type systems. { Adding run-time checkers when compile-time checking is not possible, e.g.

for managing the key databases. { From the speci cation other artifacts can be derived, yielding improvement compared to state-of-the-art systems. E.g., in PostgreSQL1 5 artifacts needs to be maintained in the software engineering process.

The speci cation facilitates code generation in the programming languages used by the applications. When the application and the generated code is compiled, compile-time checks detect many problems at an early stage. The speci cation allows software architects, developers and end-users to have a better understanding of software con guration, e.g., it deals with documentation and improves traceability. So the speci cation can even lead to an entirely better software system.

For example, the OpenLDAP 2.4.39 daemon crashes when \listener-threads" is con gured to be larger than 15 [ 12 ]. The documentation for this con guration item does not even mention this limit nor that these values are internally changed to be a power of 2. In our approach we solve this issue by writing a speci cation: [/ openldap / listener - threads ] type = enum 1 2 4 8

We have the identi er /openldap/listener-threads and one property type. Because of this property, we know which values are permitted. When the user changes the value of \listener-threads" to 16, the key database tells him/her that 16 is not one of the allowed values 1, 2, 4 or 8. Using this speci cation OpenLDAP would not crash and the di cult process of debugging is avoided. For a developer the approach is intuitive: con guration items can be used like variables, e.g., the following C++ code prints the value of the con guration item: std :: cout << openldap . listener - threads << std :: endl ;

The library libelektra provides access to the key database. The code generator genelektra makes sure that con guration items used by the developer always match with the speci cations. We also generate documentation that includes the type information from the speci cation. Any other property in addition to type can be added, which means the speci cation is extensible. 1 Version 9.1.12, see http://doxygen.postgresql.org/guc_8c_source.html

We expect, given powerful properties in the speci cation, it is simple to write a speci cation that avoids crashes, because 90% of all options are covered with a dozen types [ 8 ]. Using this approach, mismatches are ruled out and faulty use of the variable is detected by the compiler. The substantial gain is that it enhances type safety without leaving the familiar programming environment. Other potential bene ts of our approach are improved software maintenance and evolution as well as reduced duplication of code.

The rest of the paper is structured as follows: Section 2 describes the details of our approach. In Section 3 we show how we plan to validate or falsify our research questions. In Section 4 we talk about expected results based on our current knowledge. Finally, in Section 5 we compare our approach to related work before drawing our conclusions in Section 6. 2

Elektra

Our approach, called Elektra, introduces a speci cation for con guration. Instead of many places containing constraints and types, Elektra de nes a clear way how to specify con guration. The key database libelektra enforces the constraints at run-time and a code generator genelektra ensures program code conforms to it.

The approach is still in its infancy and thus many vital questions are not yet answered. The aim of our thesis is to answer following question: What kind of in uence has the use of our con guration speci cation framework, i.e. Elektra, on software? The two subquestions, to solve or at least alleviate the problems stated in Section 1, are: 1. Which properties in the speci cation have the strongest in uence on avoiding software failures caused by invalid con guration les? 2. How does the speci cation interact during software engineering processes with software architectures, software evolution, and software quality?

Elektra's Architecture genelektra

generate is part of specification type safe access code program code uses access conf. data

access tooling access libelektra plugins load, store and check key database

In Fig. 1 we see how to apply our approach and which artifacts it consists of. We immediately spot the con guration data structure in the center. It contains all key/value pairs representing con guration items. As used in every con guration parser they form a generic, but unsafe, container. In our approach we pass it around between tools, plugins and the type safe access code to avoid tight coupling between these components. 2.1

Key Database The key database is responsible for retrieval and storage of con guration items. We inherit Elektra's key database and plugin system from our earlier work [ 5 ]. Con guration is retrieved and stored using di erent plugins. While some of the plugins are responsible for the obvious tasks, e.g. parsing con guration les, others take care of cross-cutting concerns or implement run-time checkers. Latter plugins use the speci cation as input and perform run-time checks and validate input before it is stored in the key database. 2.2

Speci cation As we see in Fig. 1 the speci cation is deployed as part of the key database. By including the speci cation in the key database, we build up an information system which supports administrators in the process of creating correct software con guration. Constraints, types and links support administrators in this process. The speci cation states how a valid con guration is structured and which values are permitted.

For our thesis we are particularly interested in speci cation validation. The speci cation validation needs to ful ll the following tasks: 1. Check if the speci cation is consistently typed and has no con icting constraints. 2. Compile a minimal list of plugins that can perform the run-time checks. 3. Check if the plugins will work together. For run-time checkers it is known that such a check is a non-trivial task. 4. Check if the speci cation has a safe upgrade path from its previous version. 5. Ensure that the particular con guration le (syntax and structure) works with this speci cation by checking if they have a common supertype. 2.3

Code Generation The code generator is used to generate all other con guration-related artifacts from the speci cation. We are especially interested in the generation of type safe access code. In earlier work, context awareness turned out to be useful to provide a type safe access code using C++ [ 6 ], [ 7 ]. The types used in the speci cation must be mapped to types or generated classes within programming languages for code generation.

The compile-time safety of the approach stems from the fact that no identi er string nor self written type conversions exist in the application's source code. Instead, the developer prefers to use generated variables. Without our approach, le names and other identi ers usually exist as strings in the code.

Validation &

Methods

The scienti c foundation and starting point of our research is in the area of modularity [ 5 ]. To validate our rst question, i.e. which properties have the strongest in uence, we rst must nd out which properties are good choices for our problem domain and need an implementation of them. The following run-time checkers are candidates as properties in the speci cation: { structure validation with CORBA data types (as shown in our thesis [ 5 ]), { more powerful data types, e.g. units of measurement, { novel ways to de ne subtyping, { types inference using uni cation, { global constraints, e.g. using Gecode, Coinor and Z3, { schemas, e.g. Relax NG Schema and XSD, { Data Format Description Languages, { con guration value deduction and { any combination of the approaches above.

With the described tooling and an implementation of run-time checkers the validation of the rst question in Section 2 is straight forward: 1. By analyzing real-world problems we nd out which kinds of typical and sophisticated con guration errors occur in practice, e.g.: (a) Typos (e.g. insertion, substitution, transposition), (b) Structural errors (e.g. missing sections, parameters in wrong sections), (c) Semantic errors (e.g. wrong version, documentation, confusing similar applications) and (d) Domain-speci c errors (e.g. no such resource) 2. We build a model [ 3 ] that allows us to construct such con guration errors. 3. We build a run-time checker that permits us to reject erroneous con guration based on a promising technique, i.e., one of those listed above. 4. We evaluate the run-time checker, e.g., by comparing the expressiveness and usability of the speci cation.

The question of the in uence during software development asked in Section 2 is much more challenging, because it involves user studies. Case studies of individual attempts can provide valuable insight. The following validation plan is even more precious: 1. We create an assignment (a list of requirements) that is speci cally designed to have a non-trivial, but not too complex con guration. To reduce the e ort for the participants, we implement most parts of the application, except of the con guration relevant parts. 2. We train all participants how to use our approach. The explanation includes how to write the speci cation. 3. We randomly choose two groups A and B out of the participants: (a) Group A solves the task by using a speci cation (with the best checkers from the previous validation step present).

(b) Group B solves the task without a speci cation. 4. During the development we make snapshots of the work. Each snapshot will be tested by injection of erroneous con guration and running unit tests. 5. Finally, the participant lls out a questionnaire to answer the usefulness of the speci cation and checkers on a Likert scale.

Using this method, we can answer the questions if there is a di erence between group A and B regarding: 1. The needed e orts. 2. Which applications are more safe. 3. If the participants think the speci cation was useful.

We identi ed following risks and threads to validity: 1. The selection of participants might be biased. 2. The participants may not have many years of experience and their learning curve might not be representative. 3. When we teach the speci cation we might give a group an unfair advantage. 4. The number of participants might be too small to give results beyond the group.

To mitigate these issue we add graduates and employees to our pool. Additionally, we will use case studies and benchmarks to show other properties. 4 4.1

Expected Results

Performance In previous work [ 7 ], we showed that the access of the variables representing the con guration values does not impact performance compared to the use of native variables. Because many applications use strings at run-time, we expect that applications will even bene t from our approach in respect of run-time. For initial startup we expect that only a reasonable overhead will be added. Some additional startup time compared to hard-coded solutions, however, is unavoidable because of the abstraction Elektra provides: no con guration le names are xed at compile-time and a generic container is used. 4.2

Speci cation We expect that the speci cation will present a powerful way to precisely de ne all in uencing parts of the software con guration. We also think that the quality of documentation will rise as a result of less duplication. The properties of the speci cation, that includes type information, will give valuable hints often not available in today's systems. More assumptions will be stated explicitly.

We expect the availability of the speci cation in the key database to play a crucial role for interoperability: It will allow us to facilitate validation on every access, even by applications not aware of a speci c speci cation.

Moreover the speci cation will allow us to add traceability links to architectural decisions [ 2 ]. As a result, we expect our approach to improve the traceability and decision making process.

Safety Type safety means that a system prevents certain kinds of errors. Because of the additional compile-time and run-time checks, we expect applications using our approach to be safer in respect to the problems mentioned in Section 1. We think that most problems can be solved by adding a minimal amount of properties in the speci cation. For some issues, more e ort will be required from the developers. 4.4

Less E ort We expect that a key database with integrated speci cation will make it easier for administrators to make the right decisions in shorter time. We also think that validation sometimes even will avoid the necessity of debugging con guration problems.

The integration with the key database will allow us to change many con guration items in a safe way across applications without manual intervention. We expect this property to have a similar e ect as has the use of DNS names instead of IP addresses. 5

Related Work

Currently, to the best of our knowledge, no other approach permits us to specify con guration independent of the used technology (e.g. XSD works with XML). Con guration parsers (e.g. Apache commons con guration) need the speci cation of con guration data additional to the speci cation of con guration variables. They do not detect mismatches between code accessing con guration and the schemata of the data. We conclude the use of these libraries leads to all issues described in Section 1. Moreover, they do not provide means to abstract over le location and syntax, but need this information hard-coded.

Pluggable types [ 4 ] tackle some issues mandatory type systems have and are still an active research topic. These type systems are both used for popular dynamic and static programming languages, but are currently not available for speci cation of software con guration systems.

ConfErr [ 3 ] is able to detect con guration errors by injecting wrong con gurations before starting the application. The main di erence to our approach is, that ConfErr does not use a speci cation. We cannot directly extend ConfErr for our benchmarks because it uses an internal representation which does not support all con guration standards Elektra supports.

Range Fixes [ 11 ] make use of constraints in order to support the administrator in the decision making process, but the authors did not tackle the problem of wrong use of con guration items in the code of applications.

AutoBash [ 9 ] and ConfAid [ 1 ] have similar goals as Elektra. In these approaches predicates, that test the application, must be available on the productive system. We think that testing should not happen on the productive system, but instead earlier in the software engineering process. In our approach, possible problems will be ruled out by the speci cation so that they cannot occur in the productive system.

Spex [ 12 ] can infer parts of the speci cation by analyzing the code. This approach is complementing our approach in the sense that it can be used for initial construction of the speci cation for legacy code. It is, however, not suitable for a software engineering process. Even though Spex is the best tool available at the moment, it can only detect less than 40% of bad reactions. Because in our approach constraints are explicitly de ned in the speci cation, the number is expected to be much higher, only limited by mistakes in the speci cation.

Software product lines often assume that di erent products have di erent deliveries. In our approach, the same binary can be used in di erent deployments. In approaches that delay variability up to the execution of the application [ 10 ] our work complements product lines by increasing safety on con guration changes. 6

Conclusion

In this paper we discussed further directions of a thesis with the objective to improve integration and safety of key databases. We propose a simple con guration speci cation language that is only data integrated in a key database. The speci cation provides support for administrators con guring the system. Additionally, the speci cation allows us to synthesize code in order to eliminate potential incorrect use of con guration items in the application.

So far, we have achieved: 1. A fully working key database [ 5 ] with several dozens of plugins to support many con guration le standards and to provide some run-time checkers. 2. A fully working code synthesis tool [ 7 ] with support for thread-local and global context awareness for embedded systems [ 6 ]. 3. No overhead when reading con guration items [ 7 ]. 4. An implementation of Elektra (see http://www.libelektra.org) is freely available and can be used to see current progress of our work. Elektra already includes all components as shown in Fig. 1.

These contributions are signi cant, because they lead to a speci cation language for code synthesis and run-time checkers that mitigate the issues as mentioned in Section 1. They are also practically relevant, because they provide stakeholders a good understanding of their system's con gurability and might even reduce crashes and downtime.

In the next steps we will: 1. further de ne a speci cation language and its properties, 2. implement tooling to verify speci cations and con gurations (run-time and compile-time checkers), and 3. conduct the implementation and study as outlined in Section 3.

References

1. Attariyan , M. , Flinn , J.: Automating con guration troubleshooting with dynamic information ow analysis . In: Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation . pp. 1 { 11 . OSDI'10, USENIX Association , Berkeley, CA, USA ( 2010 )

2. Harrison , N.B. , Avgeriou , P. , Zdun , U. : Using patterns to capture architectural decisions . Software, IEEE 24 ( 4 ), 38 { 45 ( 2007 )

3. Keller, L., Upadhyaya , P. , Candea , G.: Conferr: A tool for assessing resilience to human con guration errors . In: Dependable Systems and Networks With FTCS and DCC , 2008 . pp. 157 { 166 . IEEE ( 2008 )

4. Papi , M.M. , Ali , M. , Correa Jr , T.L. , Perkins , J.H. , Ernst , M.D.: Practical pluggable types for java . In: Proceedings of the 2008 international symposium on Software testing and analysis . pp. 201 { 212 . ACM ( 2008 )

5. Raab , M.: A modular approach to con guration storage . Master's thesis , Vienna University of Technology ( 2010 )

6. Raab , M. : Global and thread-local activation of contextual program execution environments . In: Proceedings of 11th International IEEE/IFIP Workshop on Software Technologies for Future Embedded and Ubiquitous Systems . pp. 1 { 8 . IEEE ( 2015 )

7. Raab , M. , Puntigam , F. : Program execution environments as contextual values . In: Proceedings of 6th International Workshop on Context-Oriented Programming . pp. 8 : 1 { 8 : 6 . ACM, NY, USA ( 2014 ), http://doi.acm. org/10 .1145/2637066.2637074

8. Rabkin , A. , Katz , R.: Static extraction of program con guration options . In: Software Engineering (ICSE) , 2011 33rd International Conference on. pp. 131 { 140 . IEEE ( 2011 )

9. Su , Y.Y. , Attariyan , M. , Flinn , J.: Autobash: improving con guration management with operating system causality analysis . ACM SIGOPS Operating Systems Review 41 ( 6 ), 237 { 250 ( 2007 )

10. Van Gurp , J. , Bosch , J. , Svahnberg , M. : On the notion of variability in software product lines . In: Software Architecture , 2001 . Proceedings. Working IEEE/IFIP Conference on. pp. 45 { 54 . IEEE ( 2001 )

11. Xiong , Y. , Hubaux , A. , She , S. , Czarnecki , K. : Generating range xes for software con guration . In: Proceedings of the 34th International Conference on Software Engineering . pp. 58 { 68 . ICSE '12, IEEE Press, Piscataway, NJ, USA ( 2012 ), http: //dl.acm.org/citation.cfm?id= 2337223 . 2337231

12. Xu , T. , Zhang , J., Huang , P. , Zheng , J. , Sheng , T. , Yuan , D. , Zhou , Y. , Pasupathy , S. : Do not blame users for miscon gurations . In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles . pp. 244 { 259 . ACM ( 2013 )

13. Yin , Z. , Ma , X. , Zheng , J. , Zhou , Y. , Bairavasundaram , L.N. , Pasupathy , S.: An empirical study on con guration errors in commercial and open source systems . In: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles . pp. 159 { 172 . SOSP '11, ACM , New York, NY, USA ( 2011 )