Mashup environments enable end users to directly engage in the design process of the information system. Traditional mashup technology offers users generic components and therefore targets technically skilled people. Recent research investigates domain-specific mashup platforms with the aim to be easier to understand and use. We present a proposal for research towards a generic framework that supports the design of domain-specific mashup environments through native analytics.
In this paper, we conceive of an information system as consisting of human beings and/or machines that are interconnected via communication relationships and that produce and/or use information [1]. Accordingly, we interpret an educational information system as a sub-part of a computer-based organizational information system where educational processes are at the center of attention. Core processes include learning, coaching, assessment and delivery of learning content [2]; supporting processes are e.g. authoring of learning material, development of learning applications or administration processes.
The traditional mindset considers the technological part of an information system as being designed by software developers and used by end-users. The concept of secondary design, however, interprets end-users as "designers in their own right", who are actively engaged in the design and modification of the information system within the context of use [3]. This is particularly desirable because organizations, and therefore their processes, are necessarily constantly evolving due to an ever-changing environment. This in turn demands for highly tailorable technology [4] that can be continuously and substantially adapted by its stakeholders, particularly by domain experts. In line with [5], we use the term Evolutionary Educational Information System (EEIS)1 to refer to this class of educational information systems.
In [5], we have tried to identify three highly relevant dimensions within the research field around evolutionary systems, thus providing the overall directions for further investigation -and the broader frame for the hereby proposed research: D1 -Systematic discovery of improvement potentials within the system. D2 -Incremental application of corresponding modifications into the system. D3 -Inclusion of stakeholders into the continuous design process of the system.
With respect to these objectives the emerging mashup paradigm [6] promises a possible solution space. In line with our notion of EEISs, a mashup environment comprises a mashup platform plus organizational structures and actors [7]. A mashup platform is a tailorable technology [4] that allows end users to create, use, modify and exchange mashups. Mashups are personalized, situational applications created by end users by dynamically combining web resources to address the current needs of a person or community [7]. An enterprise mashup stack comprises three central technological concepts, i.e. web-based resources, which are virtualized into widgets and finally combined on demand into mashups [7,8]. In general, mashup platforms contribute to the area of D2 by providing a means to adapt a system's behavior. In providing a means for end user programming [9], mashup environments tackle problem area D3. Furthermore, being frequently used for situational reporting and analytics, mashup platforms can generally contribute to research direction D1, thus enabling a "business intelligence for the masses" concept [7].
Recent efforts investigate Domain-specific Mashup Platforms, hence aim to apply the concept of domain-specific languages [10] to the mashup paradigm (see e.g. [11]). Here, the main goal remains to make the mashup experience as simple as possible for the end user. This is attempted by providing easy-to-understand domain-specific components instead of domain-agnostic generic components. For example, "ResEval Mash" [11] represents a domain-specific mashup platform for conducting research evaluation.
As we position this research within the field of Evolutionary Information Systems, we ultimately aim at enabling secondary design within an EEISs in a unified approach. We strive to integrate techniques from the broad fields of analytics [12], domain-specific languages and mashup technologies in a comprehensive framework. Within the mindset we have sketched so far, we formulate both a high-level research question and an incomplete set of derived sub-questions.
How should a framework be designed that supports domain experts in their secondary design of Domain-specific Mashup Environments through analytics within the context of an Evolutionary Educational Information System?
We are going to search for an answer to this question by investigating the related phenomena within the Technology Enhanced Learning (TEL) domain. Therefore, we consider relevant educational stakeholders (e.g. developers of learning resources, e-learning assistants, teachers) as the domain experts, and the systems that we have direct access to2 as the EEISs. Within an abductive reflection process, several more concrete subquestions arise. Note, however, that this tentative set is expected to be extended and amended in the course of our design-oriented research effort.
-What should be designed in the primary design process, and what should be intentionally left to the secondary design of the domain experts? -How can we enable secondary design of the Domain-specific Mashup Environments (DSMEs) and still preserve control of the system functions? -How should one account for and deal with "transitive secondary design", i.e. when the secondary design process of domain experts actually acts as primary design for the secondary design of other end-user groups? -What is a suitable "gentle slope" [13] deployment process for the incremental, evolutionary establishment of such mashup environments within an EEIS? -How can we incorporate analytics to facilitate the emergence and sustainable integration of a DSMEs within an EEIS?
We aim to tackle the identified problems by means of a design-oriented research approach [14]. In this section, we sketch a tentative solution suggestion. In a nutshell, the proposed solution shall be manifested in the form of a highly tailorable framework that empowers domain experts to design and maintain DSMEs based on analytics. Our domain-aware and design-oriented research approach implicitly requires us to construct and deploy working software within an EEIS, which implies technological and organizational opportunities and constraints. The actual implementation is particularly important, as its continuous introspection within the EEISs is essential for developing and evaluating it. The intention is to develop software within the technological framework that runs our real world learning platforms. In particular, this stack comprises the dotLRN learning management system that is based on the OpenACS community framework, which in turn relies on NaviServer, PostgreSQL and the Next Scripting Framework3 .
The overall goal of efforts in the field of EEISs is to make progress with respect to the three research directions (D1-D3). In addition to that, we present here a list of relevant high-level objectives of this concrete research effort. For brevity, typical requirements known from the field of software engineering (e.g. performance, scalability, usability, etc.) and from research projects (e.g. rigorous architectural design decisions, openness, etc.) are not mentioned explicitly.
High Tailorability for Enabling Secondary Design of DSMEs. Our main objective is to design, develop, and deploy a framework that allows diverse stakeholders within an EEIS to collaboratively construct mashup environments specifically tailored to their respective domains and needs. Thus, the design decisions made in the course of the construction of the framework shall account for the concept of secondary design. Firstly, this includes a generic infrastructure that allows technical experts to develop and integrate internal or external resources and make them available for the domain experts in the form of "virtualized components" (widgets) [7]. Secondly, it requires means for the domain experts to derive or introduce domain-specific components. Thirdly, end users within the organization should be enabled to efficiently find, evaluate, and use these tailored domain-specific components (widgets, mashups, and resources).
Native Analytics. The need for analytics is especially true for Evolutionary Information Systems (EISs), which strive to enable secondary design and therefore require means for investigating and interpreting the system's behavior in a quantifiable manner. Firstly, as a basis, data generated by (instantiations of) the framework shall follow clear semantics (e.g. via collaboratively defined ontologies) in order to facilitate data mining techniques [15]. Secondly, the framework shall facilitate domain-relevant analytics, e.g. via collaboratively-annotated situational analytics mashups. This is intended to support a continuous evaluation of the mashup environment and its underlying processes for all its stakeholders, ultimately contributing to D1.
Incremental Applicability. We state the applicability and deployability of the framework within an existing EEIS as an important objective (see D2), both from a technological and an organizational perspective. The latter demands a "gentle slope" system [13] that can be incrementally learned by the domain experts. Therefore, means for the inclusion or transformation of (legacy) components and artifacts from the existing system shall be considered.
The hereby proposed research effort is intended to contribute to the field of information systems research and to relevant reference disciplines and sub-areas such as TEL, domain-specific software engineering and learning analytics. The contributions, i.e. additions to the knowledge base, are going to be artifacts [14,16], and in particular open source software. The artifacts, which embody the new knowledge, are going to comprise (i) design principles for the construction of DSMEs within EEISs; (ii) concrete models describing the constructed environments and abstract models that generalize from these; (iii) experiences from and methods for constructing such environments.
In general, research is "an activity that contributes to the understanding of a phenomenon" [14,17]. In design-oriented research, these phenomena are partly created by the researcher [14]. The construction process is supposed to reveal design principles that can be applied to a class of similar systems [18]. Numerous process models can be found in the literature, from "micro-scale" cognitive models [19] over "project-scale" [20,21] to "macro-scale" aggregate models applicable to efforts of multiple research communities [14]. However, a common scheme among these process models remains. After some form of problem awareness or trigger, the researcher iteratively switches between constructive and evaluative activities. The construction allows for creativity, nevertheless design decisions must be grounded in the knowledge base and be made explicit [22].
A range of different qualitative and quantitative evaluation approaches [17] seems appropriate for the various parts of this research. Ultimately, the created artifacts are going to be evaluated against the defined goals and objectives. For example, while the architecture of the system will be evaluated by means of expert reviews, interviews or confirmatory focus groups [17] with domain experts will be considered for evaluating the appropriateness of the system for the business environment. As our intention is to natively incorporate analytics, we expect to be able to directly gain useful quantitative usage data.
We have presented our intention to succeed with research into DSMEs. We believe that incorporating analytics into these systems has the potential to make them even more useful and effective and can help to tackle some of the problems that arise within EEISs. We conclude this paper with a research agenda. M1 -Tentative Design. A crucial first step is to further elaborate the solution suggestion to result in a tentative design [14]. We expect this to include design principles, high level architectural models, and user interface mockups. M2 -Mashup Platform Prototype. As a basis for further developments, we plan to implement a prototype of a flexible mashup platform based on and integrated with our existing EEISs. M3 -DSMEs Prototype. This iteration of the prototype development shall actually enable domain experts to design DSMEs. M4 -Analytics-based DSMEs Prototype. In this phase we are going to enhance the process of designing DSMEs by using analytics. M5 -Evaluation. While various evaluation activities will have already happened at this stage, we will finally evaluate the prototype within its environment for its appropriateness to tackle the overall problem areas and defined objectives (see Sections 2 and 4). To this end, a framework for continuous evaluation is intended to be designed and included from the beginning.
The activities preceding each milestone are expected to take us about two months, and we plan at least one publication as a result.