Evidence-Based Approach for Information System Complexity Management

Abstract

Complexity is a relative term. It depends on the number and the nature of interactions among the variables involved. In this paper, the evidence-based management (EBM) is considered as an approach to cope with university information system (UIS) complexity. The EBM approach permits to identify the reasons for the system existence and it deals with the "why" question. In the paper, the enterprise architecture (EA) model of university will be presented and opportunities to collect evidence for complexity management will be revealed.

Keywords

• Evidence-based approach
• system complexity
• university information system
• enterprise architecture
• ArchiMate 3
• 0

Introduction

Information technology (IT) management in business organization is developed and strongly supported by the best practices and standards promoted by IT Governance Institute and Information System Audit and Control Association (ISACA) as very useful to manage complex issues of IT at big companies as well as at SMEs [ 1 ]. Auditors strongly support the evidence collecting for the further improvement of any organizations. EBM approach is a professional practice of methodical research, evaluation, and utilization of up-to-date systematic research findings to support decisions about practice. That approach is to emphasize the differences between professional practice grounded in science theories and the simplistic unproven ideas popularized by associations, training providers or company "gurus". The paper is to show that EBM provides the practitioners and professionals of information system (IS) management with effective and accurate knowledge and expertise to enable appropriate IS implementation, business - IT alignment, business organization performance quality and security assurance. Systematic collection of the evidence is to be the way to cope with complexity through successful controlling.

The paper consists of two parts. At first, system complexity and evidence-based approach are discussed. The next part covers a presentation of enterprise architecture (EA) description as fundamental for evidence-based IT management. The university formal education architecture model is described as a case study in the aspect of EBM for information systems management at university. The ArchiMate modelling language was used for the architecture model visualization. 2

Complexity Reduction Practices

According to North and Macal [ 2 ], complexity theory provides an analytical framework to integrate the complex social settings. The theory focuses in the interdependence and adaptation of systems and on the creation of certain order necessary for its further governance. Lemberger and Morel [ 3 ] add that mastering complexity implies clear understanding of the real nature of complexity within the business information systems as well as an identification of the primary causes, which contribute to uncontrolled growth. According to them, the system complexity is related to a quantity or value of the information contained in an object, a system or its description, relative to some previously set goals. Organizational complexity can be hidden in its simplicity. Organizational learning or just organizing are performed to cope with disorder, i.e., unstructured and unclear state. So, by putting structure into something, people start thinking about a complex system as being composed of a number of subsystems or parts. Modeling or organizing a system requires decreasing the number of possible configurations available to that system. Complexity management is to reduce the number of subparts of a system, which thus becomes more understandable to a human mind and more predictable. IS complexity can be expressed by information system performance, which is primarily about the response time of applications and about scalability when the number of repositories increases [ 3 ]. Since complex systems are ubiquitous, Haken tries to find unifying principles for dealing with such systems [ 4 ]. He proposes to implement an economy of data collecting and looking for complex system laws. He suggests to describe complex systems on microscopic, mesoscopic, and macroscopic levels and consider systems as self-organizations. The level of considerations has an impact on the specific information collected about the organization. Based on Ashby's theory, Beer formulates some strategies to deal with complexity, i.e., attenuation of the possible disturbances, amplification of the regulatory variety to cope with disturbances and recursion by viable system development [ 5 ]. 3

Evidence-Based Approach

Evidence-based approach can be considered as a strategic as well as an operational activity for the future decision making. In that approach, evidence has been systematically searched, critically appraised for its validity and usefulness, and rigorously analyzed according to explicit and transparent criteria. The approach encourages considering local system of values, to continuously verify the knowledge and constantly penetrate knowledge resources [ 6 ]. It entails striking a balance between arrogance (i.e., assuming you know more than you do) and uncertainty (i.e., believing that you know too little to act). In evidence-based approach, research-practitioners operate by sharing all of their technical skills with those being researched. Such research imposes neither hypotheses nor solutions, all findings are grounded in mutually agreed forms of practice [ 7 ].

Evidence is the object or substance of what is advanced to support a claim that something is true [ 8 ]. The meaning of evidence is recognized in the overall context in which the evidence is presented. The evidence ought to be relevant to an underlying concept of interest, verifiable, documentable, representative to an underlying situation, cumulative, and actionable [ 9 ]. Typical structure of a hierarchy of evidence includes randomized controlled trials, quasi-experimental studies, comparisons, cross-sectional random sample studies, process evaluation, formative studies, action research, qualitative case study, ethnographic research, descriptive guides, examples of good practice, professional and expert opinions, as well as customer opinions [ 7 ].

Supporters of evidence-based practice claim that approach results in the best practices and the best use of informational resources. The opponents have claimed that evidence-based practice is overly simplistic and constraints professional autonomy [ 10 ]. Evidence-based practice opponents argue that in qualitative research approach the generalization is difficult, therefore what is needed is a combination of evidence-based work and critical analysis based on a theory. In social sciences, for arguments’ verification, researchers ask for authorities who say that, mathematicians demonstrate and demand mathematical proofs, but IT professionals require an IT solution implementation. In management science, managers frequently base their business decision on benchmarks, hopes, fears, observations of what others are doing and what they have done in the past. To make decisions based on evidence, managers must get the evidence in the first place, so they have to learn how to do their own research. In other cases, they can consult existing evidence, evaluate and apply it according to sound standards. Either way, they constantly should confront facts with general opinions. Evidence-based policy has been defined as an approach supporting people making well informed decisions about policies, programs and projects by delivering the evidence from research. In contrast to that there is an opinion-based policy, which is rather based on selective use of evidence or on the views of individuals or groups [ 11 ]. 4

Evidence-Based Approach at University

Evidence-based management at university means making decisions about the management of university courses, learning outcomes, teaching staff efforts, and administrative staff and students' works through conscientious, explicit and judicious use of four sources of information: scientific evidence, organizational evidence, experiential evidence, learning outcomes evidence as well as organizational values and stakeholders' concerns. Taking into account that personal competences and learning outcomes are the basic categories in the university education, the computerized learning management system should include the modules concerning students', teachers' and courses registration [ 12 ]. Each course should ensure the achievement of the pre-specified competences. Courses are grouped into programs and plans of studies. Programs explain what courses are offered to students and by which teachers. Plans present when the courses will be provided. Evaluations of student works during their studies are also included in the computerized system. The presented in Figure 1 enterprise architecture (EA) covers formal education university information system, including applications useful for educational process evidence gathering. The collected evidence could be used by learning controlling system, which is to control if the courses were provided by teachers, cancelled or postponed. The proposed EA model was develop with the ArchiMate tool and language, which are open, independent, free, and for general modeling. The primary focus of ArchiMate language is to support stakeholders on how to address concerns regarding their business and the supporting IT systems. ArchiMate language conforms the ISO/IEC 42010 standard [ 13 ]. The ArchiMate metamodel consists of three layers; the Business layer, the Application layer and the Technology layer. In the metamodel, the technology supports the applications, which in turn support the business. In this paper, the proposed, formal education architecture model in ArchiMate 3.0 is organized into the following layers (Figure 1): • • • •

BUSINESS containing the following elements: actor (i.e., Student, Teacher), role (i.e., System Developer, Patron), process (i.e., General University Education Process consisting of eight sub-processes), service (i.e., Learning Object Specification, Program and Course Description Browsing, Courses' Collecting, SLOs Specification, Student Enrolment Controlling, Learning Outcomes Controlling). In the paper, teach course is assumed to consist of some components i.e., Learning Objects, which are developed by teachers and re-used.

APPLICATION covering elements such as University Politics, Students Enrolment System, Students' Evaluation System, Learning Controlling System for the control of the course realization by teachers, Students' Portfolios' Registration System, IT Support.

TECHNOLOGY including elements such as Data Server, Application Server. MOTIVATION containing the following elements: drivers (i.e., Course Participation, Learning Management and Knowledge Dissemination), principles (i.e., Guides for Plans and Programs of Studies), assessment (i.e., Accreditation Commission Assessment), goals (i.e., Graduate Satisfaction, Appropriate Competences), requirements (i.e., Programs', Plans' and Courses' Proposals), stakeholders (i.e., Student, Teacher, Employer), constraints covering Course Registration Availability.

According to Midgley, system complexity can be evaluated by the quantity of relationships between its parts [ 14 ]. The specification of inter-layer and cross-layer relationships and their measurement as an evidence would be possible looking at the model in Figure 1. However, the university architecture modeling allows for the visualization of objects, deliverables, requirements, principles, and assessments useful for controlling the university organization and which are evidence of university operations. In this way, complexity of university system is phrased and controlled by the EA elements in Figure 1: • • • • • • • • • 5

Programs, Plans & Courses Proposals as system requirement.

Guides for Plans & Programs of Studies as system principles.

University Accreditation Commission Assessment.

Student Enrollment Report as deliverable.

Course Description Card as business object, containing SLOs specifications. Student Evaluations Protocol as object, Student Portfolios as deliverables. Teaching Process Evaluations as business object.

Library Report as business object Learning Guidelines & Course Materials as system deliverables.

Conclusion

The EA modeling is a way to order the collecting of evidence on organization information systems assets and to visualize their complexity. The EA model is fundamental for evidence monitoring, collecting, measurement and evaluation for the constant organization improvement, not only at the stage of its creation, but in its whole life. Implemented applications support evidence registration and systematic controlling of assets, activities and information in the organization. Taking into account the paper case study, the monitoring of selected indicators permits to improve organization as well as to deal with the organizational complexity.

References

1. Moeller , R.A. : Executive's Guide to IT Governance Improving Systems Processes with Service Management, COBIT and ITIL . Wiley & Sons, Hoboken ( 2013 )

2. North, M.J. , Macal , Ch.M.: Managing Business Complexity, Discovering Strategic Solutions with Agent-Based Modeling and Simulation . Oxford University Press, Oxford ( 2007 )

3. Lemberger , P. , Morel M. : Managing Complexity of Information Systems, The value of simplicity . Wiley, London ( 2012 )

4. Haken , H.: Information and Self-Organization, a macroscopic approach to complex systems . Springer, Heidelberg ( 2006 )

5. Beer , S. : Diagnosing the system for organizations . Wiley, Chichester ( 1996 )

6. Scott , S.V. , Venters , W.: The Practice of e-Science and e-Social Science, Method, Theory and Matter . In: K. Crowston, S. Sieber , and E. Wynn (eds.) Virtuality and Virtualization, pp. 267 - 280 . Springer, New York ( 2007 )

7. Trinder , L. , Reynolds , S. : Evidence-based practice: a critical appraisal . J.Wiley & Sons, Chichester ( 2000 ).

8. Evidence Guide , A Guide to using evidence in the accreditation process: a resource to support institutions and evaluation teams. Accrediting Commission for Senior Colleges and Universities, Western Association of Schools and Colleges ( 2002 ) http://www.wascsenior.org/search/site/Evidence%20Guide% 20Jan % 2002 % 20 . Accessed January 2010 .

9. Rousseau , D.M. , Manning , J. , Denyer , D. : Evidence in Management and Organizational Science: Assembling the Field's Full weight of Scientific Knowledge Through Syntheses ( 2008 ) http://www.evidence-basedmanagement.com/ research_practice/articles/ evidence _draft _ annals_of_aom_jan08.pdf. Accessed January 2010 .

10. Pankowska , M. : Evidence-Based Practice for Information Technology Project Management . In: H. Sroka , S. Stanek (eds.), Creativity Support Systems , pp. 105 - 112 . University of Economics, Katowice ( 2011 )

11. Davies , P. : Is Evidence-Based Government Possible? 4th Annual Campbell Collaboration Colloquium , Washington D.C. ( 2004 ) http://www. nationalschool.gov.uk/ policyhub/ downloads/ JerryLeeLecture1202041.pdf, Accessed January 2010 .

12. Pankowska , M: University 2. 0 Architecture for Informal Learning . International Journal for Latest Research in Science and Technology . vol. 5 , Issue 1 , 134 - 141 ( 2016 )

13. Systems and software engineering - Architecture description . International Standard. ISO/IEC 42010. First edition 2011 - 12 - 01 . ISO, Geneva ( 2011 )

14. Yolles , M. : Organizations as Complex Systems. An Introduction to Knowledge Cybernetics . IAP, Greenwich ( 2006 )