Internet is the broadest means of communication that has existed and is a highly effective means for the dissemination of information that allows access to millions of pages of textual and multimedia content, this leads to an information overload and a problem called infoxication, and Researchers and / or teachers are not the exception when searching for information on educational trends in research. For this reason, we propose a model to search for educational trends using Data Mining techniques, which will allow us to capture, analyze, disseminate and exploit the main topics that are currently being developed on educational trends.
At present, we live in an era where information is easily accessible and due to the
large amount of information, and that this information that exists on the web, is
increasing, according to an IDC report (International Data Corporation), that
only 33% of the information is valuable, if it is analyzed, and that by 2020 this
information will increase about 5GB [
Slamet in 2018 in his research “Web Scraping and Na¨ıve Bayes Classification for
Job Search Engine” proposes that many organizations use websites to share
information about new hires for workers and that this information is overflowed in
thousands of sites with different attributes and criteria. However, this availability
of information is very complex in the selection process and leads to inefficient
execution time, which is why it proposes a simple method to simplify the job search
through a construction and development of web techniques scraping and sorting
using Naive Bayes in the search engine. In 2016, Meschenmoser in his research
“Scraping Scientific Web Repositories: Challenges and Solutions for Automated
Content Extraction” proposes strategies to programmatically access data in
scientific web repositories. We demonstrate the strategies as part of an open source
tool (MIT license) that allows comparisons of research performance based on
Google Scholar data, emphasizing that the Scraping included in the tool should
only be used if the operator of a repository gives its consent [
3.1
Web Scraping is the practice of collecting data through any means other than a
person, which is usually a program that interacts with an API. This is generally
achieved by writing an automated program that consults a web server, requests
data and analyzes that data to extract the necessary information.[
Data mining is a discipline that has emerged at the confluence of several other
disciplines, driven primarily by the growth of large databases. The basic
motivating stimulus behind data mining is that one looks for surprising, novel,
unexpected or valuable information, and the goal is to extract this information.
This means that the subject is closely related to the exploratory data
analysis. However, problems arising from the size of databases, as well as ideas and
tools imported from other areas, mean that data mining is more than just an
exploratory data analysis. [
Data filtering From the set of data collected and already defined the objectives
that we want to achieve, we proceed to choose available data to carry out the
study and integrate them into one that can favor reaching the objectives of
the analysis. Many times this information can be found in the same source
(centralized) or can be distributed [
Knowledge Extraction Algorithms Knowledge extraction in databases (KDD)
is ”the non-trivial process of identifying valid, novel, potentially useful and,
ultimately, understandable patterns from the data” Data mining only constitutes A
stage of this process whose objective is to obtain patterns and models by
applying statistical methods and machine learning techniques. Finally, the process of
knowledge extraction also involves the evaluation and interpretation of the
patterns or models obtained in the data mining stage [
if the results are consistent. Once the learning model is obtained, it must be
validated, checking that the conclusions it produces are valid and sufficiently
satisfactory. If several models are obtained by using different techniques, the
models should be compared in search of the one that best fits the problem [
Dimensionality reduction refers to the process of mapping an n-dimensional
point, in a lower k-dimensional space. This operation reduces the size to
represent and store an object or a set of data in general [
Ant Colony Optimization Algorithm The reduction is very important since
having a large amount of data. In 1992, Marco Dorigo, in his PhD thesis proposes
an algorithm based on the behavior of ants, in search of food, being its first
application in the problem of the traveler [
An SVM (Super Vector machine) is a discriminative classifier formally defined
by a separation plane. In other words, given the training data labeled (supervised
learning), the algorithm generates an optimal hyperplane that categorizes new
examples. In two dimensional spaces, this hyper-plane is a line that divides a
plane into two parts where each class is on each side [
Data visualization is the presentation of data in illustrated or graphic formats.
Allowing people to see the analytics presented visually, so that they can capture
complicated concepts or identify new patterns. With interactive visualization,
you can take the concept one step further using technology to deepen diagrams
and graphs to observe more detail, interactively changing what data you see and
how it is processed [
This section describes the proposal to search for educational trends in research based on Data mining techniques, below in Fig.1, The whole procedure is shown. 4.1
For data collection, the ERIC database has been selected, which provides us
with scientific articles related to the area of Education, this first stage is divided
into three parts:
– Web Crawling : The website of the following website is inspected [
After performing these steps, we will proceed to apply the algorithm of Term Frequency - Inverse document Frequency (TF-IDF), with which we will obtain how relevant each word is in the document, where “t” is the term, “d” the document and “D” is the set of documents. Applying the multiplication of these two values will give us a score, the higher the score is then the more relevant is that word in the document.
tf ∗ idf (t, d, D) = log(1 + f req(t, d)) ∗ log(
N count(d ∈ D : t ∈ d) ) (1) – Term Frequency : the frequency of a term is denoted tf(t, d), it is how frequent a term “t” is in the document “d”. – Inverse document Frequency : Indicates how common a word is in a whole set of documents. It is calculated by taking the total number of documents (“N”) and divided by the number of documents that contain a word.
The pre-processing output will be the ant colony algorithm input, this entry is of the following form:
[(’computer’, 0.0651), (’learn’, 0.1789), (’web’, 0.0601)] Selection of Unsupervised Features For the selection of characteristics, the bioinspired ant colony algorithm is applied, for this, before carrying out the characteristic selection process, we will create an unguided graph, denoted by G = (F, E), where F are the characteristics and E are the edges, to find the value of the edges the similarity of the cosine between characteristics is used (2). Pp
i=1(aibi) SA,B = | (qPip=p ai2)(qPip=p bi2) | (2) SA,B (3)
Where A and B are two characteristics of dimensionality “p”, according to the equation the value of similarity ranges between 0 and 1 if the characteristics are similar, 1 is obtained, otherwise 0. After having the graph, the Ant Colony Optimization algorithm is applied, this algorithm has two important characteristics, the first is its “Heuristic Information” and the second is its “desirability”. The Heuristic Information is defined as the inverse of the similarity between characteristics, that is (3) and the desirability is the amount of pheromones, this desirability is denoted as τ . 1 4.3
Once the dimensionality of the feature vector has been reduced, it serves as input to our algorithm of Clustering, which in our case we are using K-Means, this in order to find common characteristics, and that can be grouped, to be able to visualize and interpret the results of the algorithm. And as part of the verification of the results obtained, we have applied the SVM supervised learning algorithm, with them we verify that the labels generated for each document have coherence and their corresponding classification. 5
This section describes the experiments performed applying the clustering algorithm K-Means, to see the grouping of scientific articles related to educational trends, and the application of the algorithm of SVM, to validate the learning model.
Database The database used in this work is a compendium of ERIC - Education
[
Parameter Settings:The following proposed parameters have a maximum number of cycles numCi = 10, the amount of ants will be equal to the number of threads numHor = numHeb, the initial amount of pheromone for each characteristic is τi = 0.2, in the same way the evaporation coefficient will be δ = 0.2, the parameter qini it will be equal to 0.7 with which the exploration and exploitation value will be controlled, the value of β Indicates the importance of pheromone. According to the database collected, a maximum number of 50 features will be available. Results: The results obtained by applying the parallelization of the PUFSACO algorithm (Parallelization unsupervised future selection based on Ant Colony Optimization). The tests were performed on an HP computer, Intel Core i7. The methods were written in c ++ which runs on Ubuntu 18.04.2 LTS. In the experiments, the ROC curve is used to measure performance. In this work, one third of the database was used for the test stage (41,160 data). WEKA software is used to classify the text, for which the SVM algorithm is applied (Support Vector Machine), the kernel used is polykernel, this algorithm is only used as validation.
The ROC curve shows the balance between sensitivity (or True positives) and specificity (1 - False positives). The classifiers that give curves closer to the upper left corner indicate a better performance, in the tests carried out there is a ROC of 0.9. As part of the results is that we use the visualization of information in order to graph in a faster and more concise way the quantity of labels that were collected referring to a single topic, the visualization technique is known as word cloud or cloud of labels, you can enter for review in the following link http://tendenciaseducativas.rf.gd/. Below is the graphic representation of the educational taxonomy and the tags and their categories, in tagging clouds or tag cloud, in the same measure the labels, related to education, are displayed. [a]
[b] and finally the visualization of the information referring to the years that were published and their quantity, and linked to the category they belong to. The results of this investigation that in its first stage of Data Collection, Web Crawling was used to inspect websites and Web Scraping to extract the information, can be visualized in a graphic repository of educational trends of type word cloud or tags, which They allow us to better understand how they were grouped by categories and what relationship they have in the number of searches with the search year, you can enter for review in the following link http://tendenciaseducativas.rf.gd/. An important part of the research is also aimed at reducing dimensionality, using bioinspired algorithms, helping to reduce the large amount of information, leaving us with relevant information.The unsupervised learning application shows us that it can help us discover information that does not stand out with the naked eye, but when processed by this type of algorithm, it allows us to notice more relevant information, and that it can be applied to the taking of decisions. 7
The present research work was carried out within the framework of the research project IBA-0029-2016 “ Technological Surveillance Services for research centers and Technological Innovation Classroom, Oriented to the Development of R + D + I Projects in ICTs and Education ” , we express our deepest gratitude to the Universidad Nacional de San Agust´ın, for making this study possible.