E-learning is the network-enabled transfer of skills and knowledge [23]. There are many Course Management Systems (CMS) which support e-learning [3,10,12,21].

One group of CMS uses a standardised approach to development and usage. A second uses artificial intelligence methods to support the individualised learning process. Their architecture does not satisfy the needs of learning and administration at the same time [2].

The standardised approach is typified by WebCT [22], one of the most popular elearning environments in the world. The number of parties potentially concerned with the described problems and solutions can, therefore, be very large.

The introduction of an e-learning environment such as WebCT at a university influences a variety of processes-management processes such as registration of students for courses and the workload of instructors, as well as learning processes, e.g., how e-learning is conducted, whether it replaces the teaching of some course at all or is combined with traditional learning methods, how knowledge acquired during such a course is assessed, etc. It is usually difficult to select suitable indices and methods for the assessment of e-learning and its influence. Quantitative indices or qualitative aspects can be analysed [1,18]. The attitude of students toward their experience with courses is the most important factor when it comes to the quality of courses [4,14].

In the traditional learning process, students are evaluated through formal evaluations-tests, assignments, exams, and by informal evaluations based on teacher's observations of participation, interest, body language. In distance learning, instead of observations, e-learning systems allow to evaluate student's interaction with the e-learning environment. For instance, the interest and participation can be expressed by the number of times the student accesses the course and the number of messages he sends [16]. E-learning environments allow logging, that is, gathering information and analyzing user's actions [13].

Course management environments usually have a built-in student tracking tool that enable the instructor to view statistical data such as a student's first and last login, the number of accesses, etc. [17].

The assessment opportunities supported by these tools are often inadequate, because the visualisation of data is insufficient or absent. These data are usually oriented toward the instructor's view, because statistics are represented for one particular course. There is no management view of the overall use of CMS.

In this paper, we offer a solution in which WebCT log files are combined with the university management information system and the WebCT internal data.

In Section 2, we present related work. In Section 3, we introduce our own approach and characterise the research environment. Section 4 discusses a data warehouse model which supports the data analysis objectives described in Section 3. WebCT as a data warehouse data source of multiple formats, including Web log files and XML files, is described in Section 5. In Section 6, we define the objectives of data analysis, identify business processes influenced by the introduction of WebCT at a university, and define four views with analysed indices. We conclude with directions for future work in Section 7.

Various studies have focused on the evaluation of e-learning. In [11], two objectives were discussed for e-learning assessment-the recognition of problems among students in e-learning (e.g., they are not reading the materials, they are spending too much time in discussions), as well as the evaluation of e-learning to improve the quality of courses (recognising those course materials that are not being used). WebCT was used in this research as the CMS. To facilitate the analysis of results, a new, extra tool was proposed, which enables the visualisation of analysis results.

Another study [13] performed evaluation of e-learning to assess and control the study process. Log file analysis (time and number of accesses) was used. Log files used in the study were not Web server log files, although the authors spoke to the possibility of using Web server log files in research about student activity.

Web log analysis was used in [15] with the purpose of assessing the effectiveness of course usage, defined as course usage intensiveness, manner and usefulness. The author of the study asserts that three sources can be combined to obtain an overall insight into a students' online learning process-Web logs, student demographics, and survey results. Statistical methods were used for the data analysis.

Many researchers are devoted to the idea of using data mining for the evaluation of e-learning. Log files are also used as data sources in [9,19].

A data warehouse is usually used for management information analysis of various kinds [5,6,8], as well as for Web data analysis [7].

There is only one study of e-learning which proposes the use of data warehouse facilities. The developed e-learning evaluation model described in [16,17], among others, supports three levels of evaluation -user communication tracking, evaluation of foreseen activities, and application of data mining. The data warehouse model is proposed for storage of data, which satisfies the needs of the aforementioned threelevel evaluation. It has not, however, been specified which CMS are compatible with this model.

The primary goal in this research has been to present an environment which supports analysis of the influence of e-learning on various university processes. Four factors characterise the proposed method: 1. The objectives of the analysis:

• Estimating the usage of the system to determine whether further course development is necessary and to recognise the problems which users of the course experience; • Estimating the activities of instructors to know how their workload increases;

• Estimating the usage of tools-which resources are used more, whether CMSspecific resources or just 'content delivery' features are used; • Usage of courses by students-how this influences grades. Students can be divided into groups in accordance with their grades. Analysis can be conducted to determine which group's students are using the courses and tools to a greater extent. 2. An orientation toward a broadly used and standard CMS such as WebCT. 3. A data warehouse as the environment-the choice was based on the need to integrate many WebCT data sources with MIS data to achieve the goals of the analysis. There is also the need to analyse data at the scale of the entire university. 4. The approach to data analysis-the definition of views with indices that have different granularity of dimension hierarchies, which illustrate the different needs for information in accordance with the business functions of the data users.

The data for the research were collected with reference to all course instructors and students for fall of 2004 (22 weeks). During this time, 274 of the 402 courses developed in the WebCT environment were taught in various subject areas. The courses were developed and taught by 213 course designers. A total of 4,171 students took the courses. There are 29,090 students in all at the university.

Data for the data warehouse were obtained from the two source systems-WebCT and the university MIS, from which data about courses, persons, study programmes and student grades were extracted. WebCT as a data source is discussed in detail in Section 5.

The data warehouse was implemented in the Oracle RDBMS. Oracle Discoverer is used for data access. Data are loaded into the data warehouse on a weekly basis.

The data warehouse star schemas represent information about the structure and usage of the courses. These consist of three fact tables: Structure Fact, Usage Fact, Activity Fact, which contain measurements, and of dimensions, composed of descriptive data.

This description will make reference to the concept of tools, which are separate WebCT resources, e.g. mail, discussions, etc. A static tool is produced once and rarely or never updated; a dynamic tool is modified or developed over the teaching period [15]. Fig. 1 The hierarchies were created in several dimensions to make it possible to analyse data about the course structure and usage at different levels. The Tool hierarchy of the Tool dimension, for instance, consists of three levels-the type of tool (static/dynamic), the category, and the sub-category. Tools were classified by category and sub-category in accordance with the WebCT resource classification [22].

The data warehouse usage star schema is shown in Fig. 2

The activity star schema (Fig. 3) includes information about student activities in WebCT during course acquisition. Access place is identified by an IP address. The following classifications were used in the Length category: Short (0-1 min.), Average (1-10 min.), Long (10-60 min.), Very long (more than 60 minutes).

The data sources of the data warehouse implemented include the university management information system (MIS), the WebCT Web server log files, and the WebCT's internal database. MIS is a relational database, and data are extracted by methods well known in data warehousing [8]. This process, therefore, is not analysed in detail. The structure and usage of other data sources are considered to a further extent.

WebCT log files are consistent with the Common Log Format (CLF) [20]. Data from a log file are first partially processed and loaded into a database table. An excerpt from the table is shown in Table 1. After partial processing, URI does not contain parameters, because these are unnecessary in identifying course tools. Records corresponding to WebCT standard icons and pictures are eliminated. This is possible because they include no usernames. The table with the processed data from the whole analysed period contains a total of 5,107,197 records.

Tool Identification. The WebCT log file URI consists of a course code, activity or accessed file name. All potential WebCT activities with courses (scripts) which can appear in a log file were analysed, and each activity was matched with a record from the Tool dimension. The activity-to-tool matching example for the log file records from Table 1 is shown in Fig. 4. Session Identification. Session is defined as a sequence of user activities from login to logout or moving to other Web site and not returning [15]. The maximum time during which a user remains logged in without performing any activity with WebCT is set at 60 minutes. If the period of time between a user's consecutive records in a log file is less than 60 minutes, all such records are considered to be a single session. The following information is obtained about sessions:

• User-because WebCT authorisation is required for all users, and all usernames are stored in the university MIS, it is possible to extract user information by integrating data from these sources. • Session length in seconds from the first activity to the last.

• The IP address which allows for identification of the place where the system was accessed (faculty, computer lab, etc.). • The session category, determined by session length. Activity Identification. Activity here is defined as a single hit by a user on a WebCT tool. Many log file records can occur for a single hit. A hit on an html file with multiple images, for instance, generates a log file record containing an accessed file name and additional records for each image. Counting log file records, therefore, does not result in the true number of activities. Due to this problem, all records with the same session, tool and time are considered to be a single activity. The number of activities is stored in the fact Hit number.

The transferred data amount and time, defined in our case as the difference between the current and the next record, are calculated for each activity. When there are several log file records for an activity, their data amount is summarised.

To enable separate analysis of the activities of students and instructors, user roles are extracted from the WebCT internal database, which is not a relational database. WebCT API is used for this purpose. It transfers all course users, their roles, and other data into an XML file.

Fig. 5 illustrates the structure of the XML file created in this way. Ovals denote elements, rectangles represent attributes. Elements and attributes used for the data warehouse are outlined with thick lines. The file contains a tree with a root group for each course. That is where course information is stored. It includes a unique course code (the element id), a WebCT category (the element orgunit) which identifies the faculty, etc. There is also a tree with a root membership for each course. It stores data about registered course users. The id element represents the course code. A sub-tree with the root member is created for each course user. It includes a username (the element userid), a role (the attribute roletype), etc.

Data about the course structure (the number and file size of tools) and the number of registered and active students are extracted from the WebCT internal database. The data are summarised and loaded into a table via the specially developed script. The resulting table contains the following columns: timestamp, course code, indices. Indices related to each course are merged into data strings composed of records separated by a semicolon. Each record contains a value and a special index code assigned to every tool, and to the number of registered and active students. Each record in the table is processed, and a set of fact records is obtained. It is loaded into the data warehouse tables Structure Fact and Usage Fact. The total number of records in these fact tables which refer to data about courses taught during the analysed term was 110,624 and 4,774 respectively.

Fig. 6 summarises the whole data extraction, transformation and loading process from the WebCT sources described in Section 5.1-5.3. The full data warehouse loading process also uses data from MIS.

implies that only those records with the corresponding attribute value (in this case-'student') are selected. The notation Course(course=Value) means that the index is calculated separately for each possible value of the attribute.

University management are interested in an assessment of e-learning from the viewpoint of course usage. Indices shown in Table 3 characterise course usage for the management view.

These indices can be compared to the financing for WebCT purchase and maintenance and the amount of money invested in course development. The analysis covers the entire university; the granularity is until the faculty level; the time dimension covers the entire reporting period or offers monthly data. For language comprehensibility, samples of analysis results are represented as tables. Two original screenshots of reports are included in this paper as examples of the department management and instructor view. Data analysis results from the management view indices are shown in Table 4. The definitions of registered and active students are given in Section 4.2.

The goal of the faculty dean data analysis view is to evaluate course usage indices within the framework of the faculty: 'How much' students are taught. Also of importance is the assessment of those indices which describe the learning process-'How are students taught', for instance, which course tools (static or dynamic) are used. It is also interesting to see which instructors are most active in terms of usage time and number of accesses, the aim there being to know how the WebCT usage affects the workload of instructors. The faculty dean view is defined by the indices shown in Table 5. When top-management view indices are used, the analysed dimension hierarchy level is changed, e.g., index MV2 from the management view (number of sessions) uses the following level of detail of the hierarchy: Time(month), Course(course), Session(category). Besides, the data are analysed within a particular faculty (Course(faculty=value)).

The analysis in Table 6 contains data from index FV3, for the period from September 1, 2004, until January 30, 2005.

This view refers to department managers who organise the learning process and ensure its' quality. Here, again, the activities of instructors are of interest, except that this time, as opposed to the faculty view, the entire university is analysed. Department management are also interested in the way in which WebCT is used-the tools used more often, the final grades students achieve through the use of dynamic and/or static tools, etc. The department management view is defined by new indices shown in Table 7. The results of the department management view index DV3 are shown in Figure 8, which is the original screenshot from the report. The report demonstrates the average time, in minutes, during which a student was working with different course tools, broken down on the basis of student grade satisfaction (i.e., students with satisfactory and unsatisfactory final grades).

The instructor view related to course assessment is associated with analysis of the usage of taught courses, i.e., the activity of particular students (hits, time) with tools, the activity among all students with course tools (time, hits), the number of sessions in one month, as well as analysis of session length. The instructor view is defined by the indices shown in Table 8. This paper discusses the use of the data warehouse in analysing the WebCT usage at a university. The potential data warehouse data sources and methods usable for the extraction and integration of interesting data were identified.

Four views were defined to identify the indices necessary to evaluate course usage at different management levels, as well as at the level of course instructors. The results were demonstrated through one index from each view. All index results were obtained, but these were not displayed in the paper, because analysis and interpretation of results have not yet been finished. The demonstrated results were not commented upon, although they were included to illustrate certain views.

Integrated WebCT usage analysis at the university level would not be possible without the data warehouse. At this stage, not all data stored in the data warehouse and interesting for the evaluation of e-learning are actually used for the discussed definition of views-e.g., the analysis of quantitative indices by IP address, etc. A survey on e-learning quality assessment is also necessary. The survey results can be analysed together with the indices which characterise course usage. These topics should be the subject of further research.