The current human knowledge is written. Documenting is the most used manner to preserve memories and to store fantastic stories. Thus, to distinguish the reality from fiction, the scientific writing cites previous works moreover than become form experimental setups. Books and scientific papers are only a small part of the existent literature but are considered more thrust as information sources. It is useful to find more relations and to know where to focus the lookup of a topic using the information about the authors and the keywords on the titles and abstracts. This is possible using relational databases or knowledge graphs, a semantic approach, but with the tensor memory hypothesis, that adds a temporal dimension, is possible to process the information with an episodic memory approach. If well, knowledge graphs are of extended use on question answering and chatbots, they need a previous relational schema generated automatically or by-hand and stored in an easy-to-query file format. I use JATS, a standard format that allows integrating scientific papers in semantic searches but is not spread on all scientific publishers, to extract the markup tags from PDF files, current year journal articles of one particular topic, and then construct the tensors memory with their references to extract relations and predictions with statistical relational learning techniques.
Memory is defined as the ability to record information and after recall it. Writing is a human
invention that facilitates this capacity in particular for declarative memories that are facts or events
that can be expressed with language and it could be of two types: semantic or episodic
The memories and knowledge of humanity are stored on written documents, getting more reliability if they include references to previous works from others authors. Scientific articles are the model of well-structured presentation and storage of information, each one of them with an own title, explicit authorship, and references to information related to other documents or within the same document. But, what almost always is relevant for the consideration of reading them, the retrieval action, is their publishing year. Thus, their ordered structure makes possible to use them as a representation of global human episodic knowledge and memories. Also, scientific publication as a human activity could be modeled as a social network. From this kind of networks the expression “trending topic” emerged to call the more frequent term or word used in a specific temporal window and it is understood as the principal theme or main subject that is related to the information described in a piece of content.
In a mathematical and computational framework, semantic memories could be represented
as knowledge graphs, where the entities are nodes and the links are relations between them.
A relation between entities is then possible to define as a triple .s; p; o/ or as a simple sentence
subject-predicate-objective. An episodic memory adds a time marker, thus a temporal
prepositional phrase is added to the simple sentence: subject-predicate-objective-temporal_preposition
or a quad .s; p; o; t/. This approach is widely used on semantic web technologies under the Linked
Data methodology
Thus, it is plausible to use complex networks analysis tools to search for the most relevant
relations between authors, paper titles or keywords. The scientific publication databases can easily
contain millions of authors, papers and their respective citations. A reduced number of relevant
documents is expected from a specific topic query, and not thousands of results that search engines
like Google Scholar or publisher’s own engines could generate for a given chain of words. The field
of science of science studies these relations and the former works were realized using knowledge
graphs, that are expressed as adjacency matrices. If the temporal dimension and various types
of relationships are considered, then its possible to form tensors of fourth order. A matrix X
of the network could be bipartite (X Ë Rnm) if there are two types of nodes (authors-articles,
authors-words, articles-words) or monopartite (X Ë Rnn); unweighted ( xij Ë ^0; 1`) or weighted
(xij Ë R), directed or undirected (XT = X)
The quantity of latent components is not associated with a specific statistical measure of data. However, to have an approach, table 1 presents the correspondent percentage of variance if the same number of PCA components were employed. stimulation, sleep sleep, memory sleep sleep sleep, stimulation sleep, memory brain, consolidation sleep, memory oscillations, sleep sleep, memory activity, memory sleep, memory oscillations, humans sleep, memory reactivation, slow-wave sleep, memory sleep, brain sleep, memory neuromodulation neuromodulation stimulus, presented stimulus, presented presented presented sleep, memory sleep stimulus, memory stimulus, cued memory, sws memory, spatial, sws sleep, stimulus sleep, stimulus assr, memory assr, memory wireless, monitoring sleep, slow
Words neuromodulation stimulus, technique presented sleep stimulus, cued memory, sws sleep, stimulus assr, memory sleep, slow
The words with more relations in the complete tensor, before decomposition, are sleep, memory, stimulation, slow, brain, consolidation, auditory, spindles, reactivation, and activity. Table 2 is populated using a selection strategy of most frequently word from queries of the type
wordi = argmaxo^P .s; o; t/`; (1) where s is each author, paper title or word in the database, o a word, t a year and, i is the index of a entity .
The most probable words, from the same queries, using more latent components are more than using a few latent variables. For example, there are 21 different words from query results using 200 latent components. In the other hand for few latent components, the results of queries are only the words shown in table 2.
Table 3 is populated using the of NMF decomposition in the collapsed on time matrix, adding the weights of each year. The more frequently words are selected from which are maximum for each topic or k-row in the matrix H of the decompositions. The same processing using nsNMF decomposition results with the words sleep and memory as the most probable for all the cases.
The analysis of relationships between entities needs a metric of distance. Each entity is represented by latent vectors, then one metric selection could be the Euclidean distance but given this particular type of data, content from documents, the usual metric employed is the cosine similarity. However, the use of distances on the original data space demand high computational costs, the use of a reduced space alleviates the computational cost of calculating distances but requires a previous high cost of space transformation. Figure 1 is an example of the Euclidean distance and cosine similarity that was extracted from the R tensor of the RESCAL factorization. The difference between the years of sources and the years of only cited papers is most evident with less latent components. Moreover, the similarity is greater, then lesser Euclidean distance, between the entities of the previous years.
There are scientific papers meta-data databases or it is possible to extract article’s meta-data from a specific journal or publisher. But in practice, it is usual to have few references from a previous search and they are from different journals or publishers, then to extract the meta-data I used the JATS format 1, a semantic web standard format for scientific papers popularized by National Center for Biotechnology Information (NCBI). A most popular format is the Resource Description Framework (RDF) and various scientific publishers are adopting this one.
The analysis of the statistical features of the tensor without any other process could give information of the most related entities, as the most cited author, most cited article or most used word in each slice of time. However, employing a tensor decomposition technique allows the use of a latent components space, where more information could be extracted given that the relationships are expressed in fewer variables, thus, clustering some properties of data. This work is an example of how from a small sample of documents with a known relationship between them, the topic was already known, some words that are not the most frequent could be extracted and provide a new perspective of the topics covered on the documents. The figure 1 is an example of extracted information that is not easy to visualize in the original space of the data. BArysearysea11500050 00 yyee3a3arrss 1100 0000100000000.............687901100010042486002 11500050 00 yye2e2aa55rrss 1100 00001000000000......00........02460876243501..42 11500050 00 yy22ee00aa0r0rss 1100 0000010000000.............0246805624310 wtsssa2oycied0fetpimi1velboaeTa7nnathtn)teoensietofinfcatacfcthidgcnocndecadmaooootonamnlctpfoh.dusrapgemieAredioleilspaseesnseotoinresiaiino,tnsnorsnttcgodnhsaahfioencltidtshfddmopireaffwirertteoeoasktimrhpst,rnerkteoaoionhcsvtwrotatoaaietpteliledebldstaodnieuombafgsenldlo(eeulTlotorymorfgwnoedtrsersosenaepvctphumfoaooehmarmarttvsimnoabipanuktaeaoliegssc.rl-, Figure 1. Distance metrics on the latent space. A. a framework that links computational memory Cosine similarity between years with 3, 25, and 200 with the biological one. Its capacities and defects latent components. B. Euclidean distance between need to be explored. Curiously, the etymology of years with 3, 25, and 200 latent components. “topic” comes from the Greek topos or place, that as memory is other of the known hippocampus cognitive functions.
Finally, from the results obtained is evident that sleep and memory are the most relevant words of the selected papers, these words and slow are the few words that are the result from queries too with RESCAL decomposition. The nsNMF decomposition gives for any number of components the same words, then it is more robust to the change in the number of components.
The meta-data from 11 articles from different publishers (Table 4) related to “Stimulation during
NREM sleep” in PDF files was obtained using the software CERMINE
For each year, a zeros square matrix Xk Ë R.na+nt+nw/.na+nt+nw/ was populated with weighted and directed values using the next rules only in the k-year correspondent to relations: sig.x/ = 1 +1e*x (3) (4) s;p;o;t = f e.aes ; aep ; aeo ; aet /; (6) r r r r f e.aes ; aep ; aeo ; aet / = É É É É aes;r1 aep;r2 aeo;r3 aet;r4 ge.r1; r2; r3; r4/: (7) r1=1 r2=1 r3=1 r4=1
The analysis of tensors, as for matrices, is possible to perform using a reduced form obtained by factorization. One popular factorization method of tensors is the Tucker representation, however, there are other matrices and tensor decomposition algorithms. Here, I used RESCAL and the construction of the tensor with weighted values allow to omit the predicate dimension, then the characteristic function becomes
r r r f e.aes ; aeo ; aet / = É É É aes;r1 aeo;r2 aet;r3 ge.r1; r2; r3/: r1=1 r2=1 r3=1 (8) (9) (10) (11) (12) (13) (14) (15) (16) Where Rk is a slice of the tensor R and for optimization a singular value decomposition of matrix A is employed. P is the matrix such that diag.vec.P // = S, which can be constructed by rearranging the diagonal entries of S via the inverse vectorization operator vecr*1. / Then, for regularization, the Kronecker product of the diagonal matrix is employed.
A = U V T ;
S = ä Sii =
Sii
Si2i + R X ù W H;
XHT W } W W HHT + ;
H } H W TWWTHX+ :
This matrix factorization method finds two matrices W Ë Rnr and H Ë Rrm which multiplication minimizes the Froebenius norm with the original matrix X Ë Rnm.
The updates using the algorithm proposed by
X ù W SH; S = .1 * /I + 11T ;
k D =
H m É j=1
I
Hi;j I;
W = WhD*1S*1; X ù W D*1S*1SDH: (18) (19) (20) (21) (22)
Finally, the matrix decomposition could be expressed as
This work was supported by Beca Doctorado Nacional Conicyt, Folio No 21180640. doi: