Learning Tools in Higher Education: Products, Characteristics, Procurement

Abstract

Technology-enhanced learning has gained momentum in European Higher Education, especially in recent years. In what way this movement has influenced organisations, their individual members, processes, and accordingly information system support, however, still remains an open question. Within the European IST projects ICAMP and PROLEARN, 100 European Universities in 27 countries have been interviewed by means of questionnaires in order to investigate which tools are offered to facilitate learning in these institutions, how intensively they are used, and what procurement strategies are applied.

Introduction

Already a few years ago, the European Commission assessed most European Higher Education Institutions to have invested in some sort of learning management system [ 5 ]. Furthermore, a study on the European supplier market revealed that more than 250 commercial software providers and more than 40 open-source products exist that offer a large and heterogeneous learning tool product variety [ 6 ]. For tools portfolios, i.e. the set of tools deployed in universities to facilitate learning, this offers a great variety in choice. This choice is not without consequences, as Pituch and Lee [ 1 ] found evidence that system functionality and interactivity directly affect the usage of a system for distance education and for supplementation of classroom activity.

Within this paper, the authors investigate which tools are in use today, how intensely they are used, and what procurement strategies are being followed in creating a university tools portfolio. Where possible, data will be contrasted against the findings of the CISAER and the Web-edu projects [ 4,2 ] which also investigated some of the indicators on learning tool offerings back in 1998/99 and 2001/02.

Among the 100 contributors to this survey, 66 are universities and 21 stated to be applied universities; most of them (78) are public organisations, a minor share non-for-profit private institutions (15), private for profit (2), or publicprivate partnerships (2).

Products in the Market

All together 182 distinct tools were reported to be offered which occured 290 times within the institutions in the three categories learning management, content management, and collaboration tools. Among these were 71 different learning (content) management tools (LCMS) which appeared 146 times in 94 institutions. In average an institution runs 1.6 LCMS.

The most widely offered system among the contributing institutions is Moodle with 44 instances, however, only in 15 cases the only system not running in parallel with an open-source (14x) or commercial product (15x). WebCT has a similar distribution with 14 installations. Other commercial and open-source products follow with significantly less instances.

Among the rest of the non-LCSM tools, we find 15 pure content management systems in 20 installations, 18 pure administrative information systems or course management systems (19x), 22 different authoring tools (26x), and 14 pure learning object repositories (18x). Furthermore, 10 different assessment tools in 10 places can be found. Considering pure collaboration tools, we could identify 32 different tools with 51 installations.

In a study conducted within the Web-edu project interviewing 113 European experts in 17 countries, 52 different commercial (with 134 instances) and 35 self-developed LCMS (with 35 instances) could be revealed [ 2 ]. Considering the slightly bigger sample size in the Web-edu study, todays tools landscape seems to be quite similar. However, the amount of self-developed systems (44 systems now) has increased compared to the situation several years ago (see also Section 4).

Among the most heavily used systems (selected by the highest active number of users), the following LCMS can be found: WebCT (2x), learn@WU/.LRN (1x), CampusNet (self-developed, 1x), Blackboard (1x), and eLSe (self-developed, 1x).

In the seven years since the first Paulsen report [ 4 ], the landscape of tools has significantly changed with some brands disappearing and new brands entering the market. From the tools listed in the report – i.e. FirstClass, Lotus Learning Space, Next Generation Learning, TopClass, Virtual U, WebCT, LUVIT, Web course in a box, DisCo - in our study five already seem to have vanished. Only eight of the brand names of commercial systems mentioned in the second Paulsen report [ 2 ] could be found again within the sample of this study. 3

Portfolio Characteristics

Looking more closely at the functionalities offered by and used within the systems listed in the survey responses, the contributors reported their tools portfolio to support the following activities: Almost every contributor indicated the institutional tools portfolio to support text-based communication (87 out of 100) and assessments (81 out of 100). Facilities for quality assurance and evaluation (53/100), and collaborative publishing (52/100) are still widely spread.

In less than half of the cases, individual publishing tools (44/100), tools for social networking (34/100), and tools for authoring learning designs (31/100) or tools for audio- and video-conferencing (31/100) are offered. Less frequent, tools for audio-/video-broadcasting (25/100), support for user portfolio management (23/100), and tools like simulations and online labs (21/100) are provided. All in all, ‘classical’, text-oriented activity types are predominant, multimedia-oriented activity types are lacking support.

For 1998/1999, Paulsen [ 4 ] reports in a study among 130 institutions that that only few institutions have offer a high number of web-based courses. In his findings, 68% of the institutions offer up to 15 courses, whereas 25% offered more than 15 (ten chose not to answer). At the time of being, Paulsen judges activities to be more experimental than pivotal for the institutions. These institutions were primarily based in northern Europe, however, 51 of the 130 respondents also came from different continents. Thus, results are not directly comparable but can indicate a trend at that time. The second Paulsen study [ 2 ] reports 38% of the institutions to have up to 15 online courses and 50% offer more than 15 (11 opted for no answer).

In our findings, 22% of the institutions offer up to 15 courses and 56% offer more than 15 (22 choosing not to answer), see Figure 1. Compared to the situation in 1998/99, universities offer significantly more online courses today. Already 36% of the institutions offer more than 100 courses, 5% even have more than 1000. Compared to the two Paulsen reports, the trend still seems to point towards more online courses. At the same time, however, the increase seems to have slowed down.

0 0 ittrruobno 0100008 reC 00 p 0 rsseou 600 fC 40 roe 00 ubm 20 N

Courses

Courses

Al Courses Courses in the System Al Courses Courses in the System 0 0 ittrrbnouo 8000010 reC 00 p 0 rssoue 006 fC 40 reo 00 ubm 20 N 0 0

Contributor (sorted by number of courses in the system)

Contributor (sorted by number of all courses)

Regarding the usage frequency of the different tool categories – discerned according to key functionalities –, in almost half of the cases, delivery (49/100) and course management (54/100) are frequently used by students and staff of the respondants’ institutions. This differs from collaboration and authoring tools that exhibit the whole range of usage intensities among the contributors, each with slightly less for experimental use. Collaboration and authoring obviously lack tool support within the institutional environments.

Active & Registered Users, Students, and People (absolute values) Registered and Active Users vs. People (in % of people) 0 0 0 0 6 sa 0 d frhoe 0400 e b m u n 0 0 0 0 2 people students registeredusers activeusers 0 registeredusers 03 apcetoivpeleusers sda 200 e freoh 150 b m nu 010 0 5 2 0 5 0

0 0 20

40 60 Contributors sorted by people 80 100 0

20 40 60 80 Contributors sorted by share of registered users 100

Looking more closely at the normalised figures as shown in the right diagram of Figure 2, almost two third of the institutions can be considered to have a high saturation rate with more than 10% of the people registered in the tools. Almost 20% of the institutions have more registered users than they have students. This is probably due to late deactivation of accounts of former students who graduated and left the university, external readers who do not lecture in the current term (but again in the next), test accounts, and the like. In case of the value at the very beginning, the number of active users was erroneous.

Compared to the situation Paulsen [ 4 ] reports in 1998/99, the number of online students per institution today has risen: at that time, there were up to 100 for 29.3% of the institutions, another 20% with up to 1000, and 13.1% with more than 1000 online students (four of the latter even with more than 5000 online students). Today, 19% have up to 100, another 25% with up to 1000, and 40% with more than 4000 (20 with more than 5000), see left diagram in Figure 2.

All academics use technology-enhanced learning in three cases, many in 42, some in 29, and few in 21 cases.

The size of teaching staff applying technology-enhanced learning varies greatly, with even 19 institutions having more than 100 people teaching technologically enhanced, 38 institutions with between 11 and 100 people, and 48 institutions with fewer persons.

Procurement Strategies

Procurement decisions are taken based upon three different types of organisational requirements. Speculative requirements are unique to the organisation or involve uncertainty when specified. Standard requirements are common to organisations of a particular domain. Routine requirements are considered invariant across domain boundaries.

In [ 8 ] and [ 9 ], the authors claim that – when considered optimal – organisational choices are driven by these requirements, a principle which they refer to as the procurement principle. Each choice represents a configuration of software type and procurement strategy. Software types are custom developed, packaged, and off-the-shelf. Procurement strategies are in-house development (internal procurement), contracting, and acquisition (both external procurement).

The authors of [ 8 ] suggest the following alignments between requirement types and organisational choices: for predominantly speculative requirements, internal development of custom software is recommended. Standard requirements are best met by customised, packaged software with customisation being contracted to an external partner. Routine requirements, finally, are optimally satisified by acquiring off-the-shelf software.

In our survey, only a minor share of the institutions (14%) performs activities in the area of software design and development, thereby expressing routine or cross-domain requirements. Furthermore, two similar-sized groups (each more than 55%) follow standard or speculative requirements. Most institutions require software to be adapted to an organisational context (reflecting standard requirements) or develop custom software.

Regarding the procurement strategies, we find that the major share (75%) organises procurement internally, while only the remaining quarter (27%) reverts to delegating to private contractors. Public contractors do not play a significant role.

Nearly three quarters of all institutions opt for internal development, regardless of their driving requirements type. While internal development, in general, dominates, no predominant configuration of requirement types and procurement types can be found in our study sample: Institutions that reported adopting an internal development strategy are equally distributed across the requirements types (with 21% speculative, 19% mixed, and 18% being standard).

In general, external procurement doesn’t seem to be important. External procurement from private or public contractors is adopted only by a minority of 5%. Furthermore, external procurement co-occurs predominantly with speculative requirements.

Furthermore, we find that both mixed requirements and procurement strategies seem to be highly relevant to higher education. Mixed configurations are documented by 44% in the entire study.

To conclude, only a minor share of the institutions investigated (40%) follows procurement configurations considered optimal as proposed by [ 8 ] and [ 9 ]. Synchronous, multimedial communication and collaboration tools currently lack support. Moreover, emerging social software tools e.g. for supporting social networking activities demand deliberate strengthening.

Traditional technology-enhanced learning functionalities such as delivery and course management are in the focus of learning technology use within the institutions, whereas authoring and collaboration tools need targeted support in the future.

Regarding the tools portfolios among the institutions investigated, it seems to be very important to have an institutional platform run by the institutions themselves, however, with strong connections to the open-source world. Paulsen‘s [ 4 ] exploratory statement, that many of the educational institutions have developed in-house solutions, could be further refined. This finding stresses the important relationship between open source and higher education: open source, in this perspective, is a research artefact from academic practice (see also [ 11,10,12 ]).

References

1. Pituch , K. , and Lee , Y. : The influence of system characteristics on e-learning use . Computers & Education . 47 ( 2 ) ( 2006 ) 222 - 244

2. Paulsen , M. : Experiences with Learning Management Systems in 113 European Institutions . Educational Technology & Society. 6 ( 4 ) ( 2003 ) 134 - 148

3. Paulsen , M. F. , Keegan , D. , Dias , A. , Dias , P. , Pimenta , P. , Fritsch , H. , Fo¨llmer, H., Micincova , M. , Olsen , G.: Web-Education Systems in Europe . Zentrales Institut fu¨r Fernstudienforschung, Fernuniversita¨t Hagen . ( 2002 )

4. Paulsen , M. F.: Online Education . An International Analysis of Web-based Education and Strategic Recommendations for Decision Makers . NKI Forlaget, Bekkestua, Norway ( 2000 )

5. Massy , J.: The eLearning Industry and Market in Europe. DG Education and Culture , European Commission. ( 2004 )

6. Massy , J.: Study of the e-learning suppliers' 'market' in Europe. DG Education and Culture , European Commission. ( 2005 )

7. Va¨ljataga, T., Fiedler , S. , Kikkas , K. , Laanpere , M. , Sobernig , S. , Wild , F. : D2 . 1 iCamp Space Specification . iCamp Consortium . ( 2006 )

8. Saarinen , T. , Vepsalainen , A. : Procurement Strategies for Information Systems . Journal of Management Information Systems . 11 ( 2 ) ( 1994 ) 187 - 208

9. Heiskanen , A. , Newman , M. , Simila ¨ , J.: The Social Dynamics of Software Development. Accounting Management And Information Technologies . 10 ( 2000 ) 1 - 32

10. Ambati , V. , Kishore , S.P. : How can academic software research and open source software development help each other? Proceedings of the 4th Workshop on Open Source Softwaren Engineering - 26th International Conference on Software Engineering (ICSE 2004 ). ( 2004 ) 5 - 8

11. Bezroukov , N.: Open Source Software Development as a Special Type of Academic Research . First Monday 4 ( 10 ). ( 1999 )

12. Ghosh , R. , Glott , R. , Krieger , B. , Robles , G.: Free/Libre and Open Source Software: Survey of Developers . Technical Report . International Institute of Infonomics, Unversity of Maastricht. ( 2002 )