Digital educational documents are growing in size and variety, and scientists are facing di culties to nd their way through them. One of the initiatives that have emerged to solve this problem involves the use of automatic classi cation algorithms. However, it is di cult to analyze implicit relationships among topics of materials. This paper presents CIMAL, a framework for enabling exible access to material stored in arbitrary repositories. CIMAL combines semantic classi cation, taxonomies and graphs to elicit relationships among topics of educational documents. We validated our work using materials from Coursera (courses o ered by Johns Hopkins University and University of Michigan) and a Higher Education Institute, from Brazil.
Usually, lecturers use educational material repositories to publish, store and share materials with their peers in academia and students. The access to those documents is usually open. Given such availability, how to nd and choose the material(s) more suitable to study a given topic?
Sites such as the International Bank of Educational Objects, the ACM Learning Center and the ACM Techpack, the Coursera platform, MERLOT and SlideShare show that the access to collections of educational materials in di erent formats and the analysis of their contents are still done in a restricted way. Even simple queries through the interfaces of these repositories can result in a large number of items, making it di cult to understand them and select the relevant ones. Furthermore, none of these repositories o ers means to analyze relationships among the stored objects, which would help select material.
This paper presents the design and implementation of
CIMAL (Courseware Integration under Multiple relations
to Assist Learning), abstractly presented in [
According to Romero [
Typically, research towards helping users to select educational material can be roughly classi ed as (i) development of tools to analyze, access or store materials in repositories, (ii) mechanisms to integrate heterogeneous materials via user monitoring, and (iii) use of learning objects to encapsulate and standardize contents. 2.2
The strategy we adopted to extract and represent topics of educational material is inspired by a concept that we name components of educational material. Components are positional structures that highlight information of a given material in order to facilitate its understanding. Header, body, footer and numbering of slides are examples of components of slides; titles, subtitles and the progress bar are examples of components of videos. This information also can be used for analysis; in our work, we use these characteristics in classi cation, indexing, comparison and retrieval tasks.
Unlike other approaches in the literature that use the entire text of a document equally, we also extract information of components from di erent types of material to guide classi cation tasks. Our work presents a novel strategy for documents analysis, which considers the components present in the documents to facilitate the identi cation of topics in the documents. 2.3
To classify educational materials, we use a technique called
Explicit Semantic Analysis. In natural language processing
and information retrieval, According to Egozi et al. [
According to Jiang et al. [
Mining of metadata (e.g., number of accesses to data or
identi cation of entities in the documentation of objects) is
often used to derive relationships among data, such as the
work of Pereira[
Another approach to recognize relationships is to use
external taxonomies ([
We can characterize a graph database through its data
model that di erentiates it from traditional relational databases
[
Each graph database management system has its own specialized graph query language, and there are many graph models. For example, many graph databases based on Resource Description Framework (RDF) use SPARQL (SPARQL Protocol and RDF Query Language), but Neo4J, a graph database widely used in research, uses the Cypher language. Finally, integrity rules in a graph database are based on its graph constraints.
Several researchers have adopted graph representations
and graph database systems as a computational means to
deal with situations where relationships are rst-class
citizens (e.g. [
CIMAL's architecture is a novel design to support the analysis of relationships among educational material based on their implicit topics. This architecture combines multiple algorithms for content extraction and classi cation of topics given a suite of educational material repositories.
Figure 1 presents an overview of our architecture, which comprises three layers. The Persistence Layer is composed by six repositories: Local Courseware, Components and Contents, Representations, Enriched Taxonomy, Classi cation and Relations. The Preprocessing Layer prepares data from educational material for subsequent search. The latter provides all the services needed to look for materials using graph algorithms. These services can be accessed through the User Interface by lecturers and students.
The rst step is to set up the repositories (actions represented by arrows with letters 'a' and 'b') before users can perform a search (arrows with letter 'c') . Preprocessing starts when the Courseware Crawler imports such materials from external resources (1a) and stores them in a Local Courseware Repository (2a). Next, the Components and Contents Collector extracts texts and the position of these texts from the materials in the Local Courseware Repository (3a). Extracted data are stored in the Components and Contents Repository (4a). Next, the Intermediate Graph Representation Builder creates a graph representation for each material from the repositories via the components and contents stored by the previous step (5a). These representations are stored in the Representations Repository (6a).
In parallel, the Combiner, also proposed in our research, imports an external taxonomy from a Taxonomy Repository, and a set of external expert texts from Domain textual documents Repository (1a). These data are uni ed in an Enhanced Taxonomy, in which each concept of the taxonomy has a reference to a text by experts, and stored in the Enriched Taxonomy Repository (1b).
Once representations and enriched taxonomy repositories are created, the Classi er is ready to de ne the topics covered in each of the materials (2b,3b,7a). This information is then stored in the Classi cation Repository (8a).
Lastly, the Relationships Analyzer looks for prespeci ed relationships among the items and their topics in the Classi cation Repository (9a), creating the Relations Repository (10a).
All preprocessing steps must be performed every time we add educational material, taxonomy or texts from a domain textual base.
After such preprocessing, lecturers and students can run queries through the Interface Layer (1c). It redirects the query to the Graph Engine and the Search Engine (2c). The latter accesses the Relations Repository (3c) to nd relevant educational materials that are related to the user query.
The CIMAL software is the rst implementation of the architecture described in Section 3. We have developed the components of Interface and Preprocessing Layer using JAVA code, our texts come from Wikipedia, the taxonomy from ACM Computing Classi cation System, and methods of Apache Lucene, a high-performance full-featured text search engine library.
Since CIMAL uses graphs to perform relationships analysis, the Persistence Layer stores all data in a database with native support for graphs (Neo4j). With this approach, we are able to use already established technologies and solutions for processing graphs. We chose the Neo4j database system because it is the most popular graph database in big companies (e.g. eBay and Wallmart) and in research, according to the Db-Engines site, an initiative to collect and present information on 341 database management systems.
Our main implementation is divided in four steps: (Step A) Extraction of elements of interest; (Step B) Intermediate Representation Instantiation { based on the schema de ned
At Step A, the Components and Contents Collector extracts components from material based on a Java Framework called DDEx and several APIs for document handling. It scans educational material based on a set of positional rules de ned by users and identi es the desired components. Each identied component is encapsulated in a standard representation and forwarded to Step B.
The texts from header and body, and number of slides were extracted automatically using DDEX as components of each slide. In addition, the texts present on the body of slides were also extracted. Through the subtitle le, available for each of the videos, the texts and the time stamps of each of the lecturers' statements were extracted. 4.2
Step B creates the Intermediate Graph Representation and stores this representation in a repository. The use of this representation enables the manipulation of parts of educational material without interfering with the material themselves.
The components and contents of a material are transformed into a graph where the nodes represent the elements of interest that are used in our work. These elements di er according to the kind of material, for example in a video we would like to extract the subtitles and in a slide we extract sections. 4.3
Step C has three software modules we implemented: The rst module ("Combiner" tool) is concerned with creation and storage of an enriched taxonomy. The second (Classi er tool) recognizes the topics of each Intermediate Representation according to the taxonomy and creates a document about the "Classi cation of Representations". In our studies, we de ned that the words present in the components of the slides or that are among the ve most repeated in videos subtitles should be 3 times more important in the classi cation than the words in the rest of the documents. The third module (Relationship Analyzer tool) concerns the production of information about relations, based on the "Classi cation of Representations".
The Combiner tool adds one page of Wikipedia to each node of the Taxonomy, thus producing an Enriched Taxonomy. Next, the Classi er tool calculates the similarity of each text of Intermediate Graph Representation (related a each educational material) for each pages of the Enriched Taxonomy. 4.4
At last, in Step D users can perform queries to nd relevant content. Here we implemented in Java programs and 2graph the Interface layer tools. 2graph is a java-based API to perform Extract, Transform and Load (ETL) resources to graph structures/databases, to handle the information produced by CIMAL and interact with users. 5.
To achieve the objective of this research the following obstacles have been faced:
1) Although widespread, the idea of sharing teaching materials still faces resistance from lecturers. In order to perform classi cation tests and also to verify relationships between the topics, it is necessary to nd di erent materials but with similar approaches to explain topics. The solution found was to use materials from the same repository (Coursera) and from the Computing area, in which the idea of electronic sharing is more popular.
2) Most of the lesson videos are produced for a speci c audience. Consequently, many lectures only explain concepts in a speci c language, and do not produce subtitles for other audiences. Automatic transcription of captions is still a research problem. Therefore, we have selected only videos that had their subtitle produced manually, which drastically reduced the amount of educational videos available in educational repositories that could be used. Thus, we used videos from the Coursera platform, which follow a standard of subtitle production, thereby making the analysis of video content more adequate.
3) The use of graphs for analysis of relationships is very common in many research domains, but this practice is not yet widespread in the educational eld. In our work we only use volunteers with knowledge in graphs to analyze the contributions of this research.
We collected 97 sets of slides and 97 videos from the
Specialization course in Data Science, o ered by Johns Hopkins
University, to be used as a case study. Using our system,
we are able to discover the topics covered throughout the
specialization course without requiring annotations or other
extra tasks for teachers. We point out that CIMAL can
thus also be used by lecturers to annotate and classify their
materials. More details on this case study can be found at
[
A second case study was conducted at an educational institution in the state of S~ao Paulo, Brazil. We show how we nd similarities among di erent courses, thereby highlighting possible intersections, thus revealing potential multicourse activities.
We were able to extract the contents and topics covered in each of the documents that regulated the courses of this institution and relate each of their contents through graphs. Documents with many relations revealed possible interactions between their respective courses. 6.3
To nalize our study, we designed a questionnaire to evaluate the classi cation of topics extracted from 6 materials (randomly chosen for the questionnaire does not get too long) from the "Python for Everybody Specialization", provided by University of Michigan. Thirty volunteers of di erent levels of education and specialties in sub-areas of Computer Science gave opinions for each of ve topics extracted using the CIMAL implementation. After this activity, we can see that CIMAL classi es the materials using pertinent topics, since 64% of the topics indicated by the framework were evaluated "Some related (16,5%)", "Related (15%)" or "Closely related (32,5%)" by the volunteers.
This paper presented the design and implementation of CIMAL, which allows searching content from educational material, and eliciting relationships among topics. This framework contributes to helping lecturers and students navigate through collections of materials. Our implementation is validated on slides and videos from case studies and showed that the components on slides and videos can be used to classify text and relate topic of these materials.
One particular question is of interest to us: "Can the history of courses taken by students in uence the topics that the students are looking for in educational material repositories?"
To answer this question, it is necessary to collect data of user accesses to these materials. For example, data on the last courses that a student held in Coursera could be used to construct a personalized study guide on subjects that would be interesting for this student; the recommendation system could also recommend more Coursera courses.