Recommendations for network studies in learning analytics emphasize that network construction requires careful definitions of nodes, relationships between them, and network boundaries. Thus far, researchers in learning analytics have discussed how to operationalize interpersonal networks in learning settings. Analytical choices used in constructing networks of text have not been examined as much. By reviewing examples of text network analysis in learning analytics we demonstrate that convenience-based decisions for network construction are common, particularly when the ties in the text networks are defined as the co-occurrences of words or ideas. We argue that such an approach is limited in its potential to contribute to theory or generalize across studies. This submission presents an alternative approach to network representations of the text in learning settings, using the concept of Forma Mentis Networks. As reported in previous studies, Forma Mentis Networks are network representations either (1) elicited from individuals through free association tasks that capture valence or (2) constructed by analysts creating shared mental maps derived from text. Forma Mentis Networks is a theory-based and scalable approach complementary to the existing set of tools available for the analysis of teaching and learning.
Quantitative text analysis describes a family of the methodologies commonly used in learning
analytics, including but not limited to content analysis [
Automated content analysis is well suited for a quick high-level summary of the frequent concepts
across large quantity of texts. However, its utility for nuanced research insights has been challenged
in seminal work [
Carley, as map analysis, draw on the scholarship in text analysis and network science. Therefore, analyzing text as a network is an inter-methodological endeavor: representing text as a network requires theoretical justifications grounded in text analysis, whereas analyzing such a network requires the knowledge of graph theory.
This short review argues that currently analyzing text networks in learning analytics research can
benefit from further theoretical and analytical rigor. This argument is not new. Recommendations for
network studies put forward by the participants of NetSciLA21 workshop similarly emphasized that
when researchers construct networks in learning settings, they need to carefully define nodes and
relationships for the network representation. How to operationalize networks of people who interact
in learning settings had been discussed elsewhere [
Learning analytics research commonly uses networks to represent and analyze texts created by
learners. Examples of student-produced text analyzed through networks include student personal
reflection essays [
The use of co-occurrence of words in a sentence is a common approach to defining network edges in
text networks. This section provides a few examples outlining this approach. The basic steps of such
an approach can be found in Wise & Cui [
More nuanced automated approaches to text processing that precede the construction of
cooccurrence networks are also available. For instance, Joksimovic and colleagues [
A different way of defining nodes and edges in co-occurrence networks has been suggested by
van Labeke et al. [
A variation of the analysis of co-occurrence networks is offered by Yun and Park [21]. The researchers used the transcripts of science teachers’ classroom talk to construct the networks of words co-located in the same sentence. The words prominent in such networks were compared to the words frequently appearing in scientific corpus related to the same subject area. Peculiar to this study is that the researchers used external discourse to evaluate and understand how teachers were explaining content in the science classroom.
As shown through the above examples, analytical decisions around co-occurrence networks vary. The questions remain as to whether some of the automated approaches offer insight into theoretical perspectives, as well as whether the more theory-grounded analytical steps that require human interpretations, can be successfully replicated. 2.2.
Epistemic network analysis (ENA), an approach that has recently gained prominence for the analysis of texts generated in the classroom, essentially is a co-occurrence network too. In ENA, the networks reflect ‘the structure of connections among coded elements by quantifying the co-occurrences of codes within a defined segment of data, or stanza’ [22]. That implies that in many cases, ENA requires researcher-imposed segmentation of the text (i.e., decision around the unit and level of analysis), as well as qualitative interpretation of text (i.e., content analysis or thematic analysis). Consequently, ENA inherits methodological assumptions required for content analysis that has developed knowledge base and systematic protocols for qualitative decisions around the unit and level of analysis, as well as the assignment of themes. ENA leverages these decisions by applying cooccurrence to link qualitatively derived codes within a network. In addition to these components ENA utilizes innovative technique to create a visual projection of this network, as raw counts of code cooccurrence within the stanzas are transformed. Vectors of code co-occurrence are processed using single vector decomposition and normalized to control for the more ‘crowded’ stanzas. These transformations allow to reduce the noisy co-occurrence networks to highlight more prominent relations and to visualize the co-occurrences in a more replicable and comparable manner. The ENAspecific transformation enables comparisons between these processed representations at the levels of individuals or groups.
Replication and scalability of deriving codes (network nodes) within the stanzas (units of analysis) remain in tension with theoretical grounding. For instance, a theory can guide which codes are selected and how, but replication of such analysis requires detailed report of the content analysis approach in the study, whereas automating it requires further human annotation and training of supervised machine learning models. Researchers have been working to create automated approaches to deriving key codes [23]. These techniques (described below) offer a significant advancement of the method, and yet, they are less theoretically grounded than human interpretation of the texts. For instance, Fereira and colleagues applied natural language processing, Latent Dirichlet Allocation (LDA) in particular, to identify topics within posts made by students. By way of background, LDA estimates the probability of a topic to be associated with a particular unit of analysis, here a post in a discussion forum. Fereira and colleagues substituted the matrix of binary co-occurrences between codes in stanzas, which is commonly used as input for ENA, with a different matrix, capturing the relationship between the posts and LDA-derived topics, with edge weights representing probability of topic to appear in that post. The authors explain that the ENA pipeline processes this matrix with probability weights, similarly to the co-occurrence matrix, using single value decomposition with the possibility to transform the weights via direct product, square root, or natural log methods.
Computational innovations help ENA evolve to eventually overcome the tensions between theoretical insights, replicability, and scalability it offers. However, so far, the ENA pipeline is methodologically eclectic. Its input can be both derived using ‘interpretivist’ data collection requiring content analysis and thematic analysis, as well as using data that is not interpreted but created by machine learning approaches. In either case, the relationships between the codes, regardless of when derived from human interpretation or via NLP, are operationalized through co-occurrence, suggesting that ENA does share certain methodological and theoretical assumptions with the methods described in the previous section. 2.3.
Learning sciences offer theoretically grounded frameworks to the analysis of learning, where
relational thinking (about students and text they produced) is inherent to the theory explaining the
learning processes. A prominent example is the work by Oshima and colleagues around modelling
knowledge building processes [24]. Oshima and colleagues [
A dynamic extraction of keywords from text prior to network construction is offered through the Network-Text Analysis (NTA) approach by Taskin, Hecking, and Hoppe [25]. The approach presents yet another graph-based application for filtering the noisy co-occurrences, which requires decisions around window size and thresholds that can be challenged around its theoretical considerations. They proposed a technique for extracting networks of concepts appearing in texts as linked by certain measures of proximity. The authors emphasize that the reduction of the number of relationships between words/keywords to exclude those that are not meaningful are among key challenges for cooccurrence networks. To this end, the authors apply an explicit semantic analysis approach that infers more meaningful entities. Once entities or keywords are identified, the edges between them are created based on the moving window approach: the words are not separated by more than k-2 words, for instance 20 words in an example presented by the authors. Once this step is completed; the authors suggest further filtering of the concept networks based on a threshold that is fine-tuned to the dataset. 2.4.
Most of the approaches discussed above use co-occurrence of words to define relationship between
the nodes in text networks. However, text analysis has been combined with network analysis in
learning analytics methodologies in other ways as well. Some approaches combine text analysis with
network analysis to delineate groups of people that shared discourse of particular kind, then
analysing relationships between these individuals. We provide three prominent examples that reflect
the techniques. Hecking and colleagues [26] apply NLP to semantically identify conversations
exclusively focused on the subject matter. Then, the relationships between learners who exchanges
only this subject matter content are constructed and analysed via network analysis. Dascalu and
colleagues [27] apply the so-called coherence network analysis – an NLP-based approaches that
creates links between people based on the similarity between their text. Hecking and colleagues [
Across all the above examples we can observe that the more scalable approaches are often less theoretically grounded. In many of these studies analytical decisions are based on convenience and require contextual decisions that impede generalisability. As a result, interpreting text networks, i.e. addressing the question of ‘what these networks represent’, is non-trivial, and its external validity is limited. That is not to say that the problems like these characterise all the examples. Theory-grounded approaches to integrating text into networks include bipartite networks of learners and constructs they contributed to where the presence of text and ties are theoretically justified, as well as graphs derived from concept maps constructed by learners themselves [29]. In the first example, a network representation is an operationalization of an emergent discourse-mediated community, in the second example - it is student knowledge representation. However, theory-based interpretations of text networks built on co-occurrence ties, which is a more commonly used approach, are difficult to infer.
This brief review highlights that, so far, the potential of text networks in learning analytics to contribute to theory or generalize beyond specific examples, is limited, despite their pragmatic utility in deriving insight for specific cases. Defining ties between words or phrases through co-occurrence is a practical and convenient decision that requires little theoretical knowledge of what is being represented. However, such a decision can lead to oversimplifying the patterns captured through the network. Co-occurrence, particularly when it comes to representing networks of text, is a crude tie definition. Its power to derive an insight comes at a cost: the noise in the network that treats multiple types of linguistic relationships similarly [30]. When further coarse graining takes place, for instance, if the concepts used in a network as nodes are derived computationally using machine learning algorithms, the nuances in meaning that individuals assign to perceived language, are further washed out.
Cognitive science has evolved to include network approaches with rigorous and theory-grounded
network definitions, offering alternative inspiration to researchers in learning analytics [31].
Cognitive networks are conceptualized as the mental reflection of language and associative
knowledge in the human mind [32]. Accessing cognitive networks means reading people’s minds,
accessing people’s perceptions as associations of ideas about environments, opinions, and emotions.
Overwhelming empirical research [
Representing human associative knowledge as networks is advantageous over black-box machine learning tools [40]. Firstly, it facilitates generalizability - metrics of network representations, operationalized in ways that reflect human associations or semantic memory, allows to use metrics and null models from network science (see also [41]). For instance, network distance out matches semantic latent analysis in predicting similarity rates [42], whereas network growth models provide evidence for the preferential acquisition hypothesis in word learning, i.e. a tendency for children to acquire first the most semantically prominent concepts in the language they are exposed to [43], [44]. Secondly, when the interpretation of these cognitive structures is consistent, the analysis can power cognitive and psycholinguistic theories through data. For instance, checking which concepts were associated with a target idea provides contextual information about how that idea was semantically framed by a given text. This reconstruction of contextual information from associations is formally described by semantic frame theory [45] and operationalized by cognitive networks, where semantic frames become network neighborhoods or communities of tightly related concepts (see also [46]).
Here, to offer an alternative to approaches used in learning analytics, we describe a specific
framework to operationalizing cognitive networks. This framework of forma mentis networks (FMN,
from forma mentis, Latin for “mindset”) can capture, reconstruct, and explore perceptions in
individuals or groups. FMNs combine artificial intelligence, cognitive psychology and complex
systems to explore both explicit/conscious and implicit/subconscious knowledge and emotional
perception of individuals or groups of individuals toward a given topic (see [
Behavioral forma mentis networks (BFMNs) represent knowledge as a web of concepts
interconnected by memory links and rated in terms of sentiment/valence, i.e., “positive”, “negative”
or “neutral”. From a methodological point of view, this means that a link between the concepts is
based on free associations [49] for establishing conceptual links: participants are given cue words, for
example “bird”, to which they might respond with the word “dove”. These two concepts, “bird” and
“dove” are then associated and linked with each other. Positive or negative valence between the
concepts is elicited from the individuals, embedded in the network as an edge attribute. Such tie
definitions have strong theoretical roots. Free associations represent conceptual knowledge about the
external world as embedded in the so-called semantic memory [50] and are consequently powerful
proxies for predicting language learning [34], creativity levels [37] and even personality traits [51].
FMNs rely on such powerful psycholinguistic tools but include an emotional aspect as well, quantified
via word valence, e.g., how positively, negatively, or neutrally a given concept is perceived [
An example is reported in Figure 1 (reproduced from [47]), which features the FMNs around “mathematics” as reconstructed by 159 high-schoolers and 59 STEM researchers. Importantly, participants provided free associations and valence norms, tagging nodes/concepts as positive, negative or neutral. Concepts tagged as negative and linked mostly with other negative concepts were found to elicit higher levels of anxiety in an external dataset of affective psycholinguistic norms [52]. Hence, words mostly surrounded by negative associates in FMNs were also found to correspond to anxiety-eliciting concepts and were therefore used to identify signals of STEM anxiety [47], test anxiety and implicit negative biases such as stereotype threats [48] present in the students’ perception and absent in researchers’.
Textual forma mentis networks (TFMNs) adopt natural language processing to build links between concepts. Language can describe experiences (“I moved to another town”, “He passed away”, etc.) and inner emotional states (“I feel relaxed”, “I feel like I am slowly healing”). Such language can be communicated through text, e.g., in social media posts. In essence, the premise for TFMN is similar to that used in text networks in learning analytics. The difference, however, lies in the way network ties are operationalized in TFMN. TFMNs are sensitive to syntactic and semantic associations of words in language, linking two words that follow each other within the unit of analysis and combining this with the automatically detected sentiment within the concepts to denote valence.
From a methodological perspective, TFMNs can identify syntactic relationships in text with the help of NLP and AI. Stella [53] implemented TFMNs based on sentence parsing from the Stanford NLP universal parser [54], as implemented in Mathematica 11.3 through the TextStructure routine. Powered by a recurrent neural network architecture, the dependency parser is trained to identify grammatical dependencies between words in sentences. The parser scans words in linear time with sentence length and, at every step, it maintains: (i) a stack of words being processed so far in the sentence, and (ii) a buffer of words yet to be processed. By transitioning its inner state, the parser is trained to identify words as either syntactically dependent/independent according to their features and the features of both the stack and the buffer. Sequentially, the parser empties the buffer and captures the dependency structure of words in the stack. The parser is taught to apply the correct transitioning through training data, that is an annotated corpus for the English language [55]. Once the syntactic dependencies are identified, TFMNs can be constructed by connecting non-stop word nodes that share a path of syntactic dependencies lower than a given threshold (that is manually selected by the researcher to regulate the density of syntactic links in TFMNs). This refined, condensed structure, is semantically enriched by further linking synonyms, using a dictionary such as WordNet [56]. Finally, words are endowed with sentiment/valence and emotional norms, e.g. “love” is a positive word or inspires “joy” according to psycholinguistic mega-studies (see [46]). Notice that TFMNs are flexible enough to feature different machine learning outcomes or linguistic models under the hood, a feature that is promising in field-specific linguistic models like FLAIR (for fashionrelated language [57]) or spaCy (for news-related language, see https://spacy.io/, accessed 18/05/2022).
Overall, TFMNs follow a nuanced approach to reconstructing the associations between ideas encoded in texts if compared to the networks built on the co-occurrence of ideas. TFMNs capture semantic, syntactic, and emotional structures underpinning the associations of concepts, without the need to interview individuals. Crucially, whereas in BFMNs there can be different affective perceptions for the same word across groups, TFMNs rely on external data for producing the valence labels attributed to words (e.g. the EmoLex dataset [58]). This means that in two different BFMNs, the same word “mathematics” might be perceived as a negative concept in one network and as a positive concept in another one (see also Figure 1). This difference is due to the fact that in the behavioral data behind those networks, individuals rated “mathematics” with lower valence scores in one case compared to the other. In all TFMNs based on the EmoLex dataset, “mathematics” would always be represented as a neutral concept. This is due to a limitation in the way TFMNs are constructed: TFMNs can only access textual data, and so syntactic relationships, but they do not consider meta-data about valence like behavioral forma mentis networks do. As discussed also in [53] and in [59], also TFMNs can portray the same concept along difference valence connotations but only through contextual information: In a given text, “mathematics” could be syntactically linked with mostly negative concepts and thus acquire a negative affective connotation within its own semantic frame/network neighborhood. Consequently, TFMNs put even more emphasis over the importance of going beyond node-level quantifications of valence and reconstruct affective perceptions of concepts by checking how they are interconnected with each other. In other words, whereas BFMNs already present variability at node level (through valence scores), TFMNs should be analyzed always in terms of their network structure and how it relates with the identified valence/emotional labels [59].
Our reflection presents concerns about the rigor, theoretical grounding, generalizability, and scalability of approaches to text networks in learning analytics. First, prominent approaches used for text networks in learning analytics can improve their use of theory. Text networks based on cooccurrence are not theory grounded. Epistemic networks are also less theory grounded, particularly in the instances when automated detection of codes is in place. Second, the generalizability of text networks is limited, due to varying and convenience-based decisions around the unit of analysis and operationalizations of the networks. Analytical choices in co-occurrence networks enable scalability but result in the presence of the noise and lack of sensitivity to the context within the text, i.e., it is not evident to the experimenter whether a co-occurrence expresses a syntactic, semantic, or phonological association between words.
We presented an approach to text networks rooted in cognitive network science, Forma Mentis Networks (FMNs) to demonstrate how the issues currently present in many instances of text networks can be overcome. FMNs are theory-based and can be applied to data sources that are either elicited from individuals or collected from written text, where network ties can be interpreted as associations or as sequences. FMNs are based on previous research in cognitive networks and allow representing knowledge structures. Yet, they also enable scalability - as automated approaches are used to derive networks [54]. FMNs contain clusters of nodes with similar semantic characteristics that can be compared with the models of mental lexicon. The tight interrelationship between theory, a network representation, and analytical techniques that enable generalizability (comparison to external models of language) offer an example of a methodology that is both scalable and informative to theory and practice. We hope that this critical and reflective argument opens for a discussion about other possibilities of analyzing text networks in the learning and teaching settings.
The authors acknowledge the feedback of the workshop participants at NetSciLA22 that was integrated into this submission, as well as thank anonymous reviewers for their feedback. [20] D. Whitelock, A. Twiner, J. T. Richardson, D. Field, and S. Pulman, “What does a ‘good’essay look like? Rainbow diagrams representing essay quality,” in International Conference on Technology Enhanced Assessment, 2017, pp. 1–12. [21] E. Yun and Y. Park, “Extraction of scientific semantic networks from science textbooks and comparison with science teachers’ spoken language by text network analysis,” International Journal of Science Education, vol. 40, no. 17, pp. 2118–2136, 2018. [22] S. S. Fougt, A. Siebert-Evenstone, B. Eagan, S. Tabatabai, and M. Misfeldt, “Epistemic network analysis of students’ longer written assignments as formative/summative evaluation,” in Proceedings of the 8th international conference on learning analytics and knowledge, 2018, pp. 126–130. [23] R. Ferreira, V. Kovanović, D. Gašević, and V. Rolim, “Towards combined network and text analytics of student discourse in online discussions,” in International conference on artificial intelligence in education, 2018, pp. 111–126. [24] M. Scardamalia and C. Bereiter, “Computer Support for Knowledge Building Communities,” in CSCL: Theory and Practice of an Emerging Paradigm, T. Koschmann, Ed. Malwah, New Jersey: Lawrence Erlbaum Associates Inc. Publishers, 1996. [25] Y. Taskin, T. Hecking, and H. U. Hoppe, “ESA-T2N: a novel approach to network-text analysis,” in International conference on complex networks and their applications, 2019, pp. 129– 139. [26] T. Hecking, I. A. Chounta, and H. U. Hoppe, “Role modelling in MOOC discussion forums,”
Journal of Learning Analytics, vol. 4, no. 1, pp. 85–116, 2017. [27] M. Dascalu, S. Trausan-Matu, P. Dessus, and D. S. McNamara, “Discourse Cohesion: A Signature of Collaboration,” in Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, Poughkeepsie, New York, 2015, pp. 350–354. doi: 10.1145/2723576.2723578. [28] U. Hoppe, “Computational Methods for the Analysis of Learning and Knowledge Building Communities,” in Handbook of Learning Analytics, First., C. Lang, G. Siemens, A. Wise, and D.
Gasevic, Eds. Society for Learning Analytics Research (SoLAR), 2017, pp. 23–33. [29] I. Koponen and M. Nousiainen, “Koponen, I. T., & Nousiainen, M. (2018). Concept networks of students’ knowledge of relationships between physics concepts: finding key concepts and their epistemic support.,” Applied network science, vol. 3, no. 1, pp. 1–21, 2018. [30] A. Ninio, “Syntactic networks, do they contribute valid information on syntactic development in children?. Comment on" Approaching human language with complex networks" by J. Cong and H. Liu,” Physics of life reviews, vol. 11, no. 4, pp. 632–634, 2014. [31] C. Siew, “Investigating cognitive network models of learners’ knowledge representations,”
Journal of Learning Analytics, vol. 9, no. 1, pp. 120–129, 2022. [32] C. S. Siew, D. U. Wulff, N. M. Beckage, and Y. N. Kenett, “Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics,” Complexity, vol. 2019, 2019. [33] C. S. Siew, “Using network science to analyze concept maps of psychology undergraduates,”
Applied Cognitive Psychology, vol. 33, no. 4, pp. 662–668, 2019. [34] T. T. Hills, M. Maouene, J. Maouene, A. Sheya, and L. Smith, “Longitudinal analysis of early semantic networks: Preferential attachment or preferential acquisition?,” Psychological science, vol. 20, no. 6, pp. 729–739, 2009. [35] N. Castro and M. Stella, “The multiplex structure of the mental lexicon influences picture naming in people with aphasia,” Journal of Complex Networks, vol. 7, no. 6, pp. 913–931, 2019. [36] D. R. Amancio, “A complex network approach to stylometry,” PloS one, vol. 10, no. 8, p.
e0136076, 2015. [37] Y. N. Kenett, D. Anaki, and M. Faust, “Investigating the structure of semantic networks in low and high creative persons,” Frontiers in human neuroscience, vol. 8, p. 407, 2014. [38] D. M. Lydon-Staley, D. Zhou, A. S. Blevins, P. Zurn, and D. S. Bassett, “Hunters, busybodies and the knowledge network building associated with deprivation curiosity,” Nature human behaviour, vol. 5, no. 3, pp. 327–336, 2021. [39] I. T. Koponen and M. Pehkonen, “Coherent knowledge structures of physics represented as concept networks in teacher education,” Science & Education, vol. 19, no. 3, pp. 259–282, 2010. [40] C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” Nature Machine Intelligence, vol. 1, no. 5, pp. 206–215, 2019. [41] A. A. Kumar, “Semantic memory: A review of methods, models, and current challenges,”
Psychonomic Bulletin & Review, vol. 28, no. 1, pp. 40–80, 2021. [42] Y. N. Kenett, E. Levi, D. Anaki, and M. Faust, “The semantic distance task: Quantifying semantic distance with semantic network path length.,” Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 43, no. 9, p. 1470, 2017. [43] T. T. Hills, M. Maouene, J. Maouene, A. Sheya, and L. Smith, “Longitudinal analysis of early semantic networks: Preferential attachment or preferential acquisition?,” Psychological science, vol. 20, no. 6, pp. 729–739, 2009. [44] E. A. Karuza, “The value of statistical learning to cognitive network science,” Topics in
Cognitive Science, vol. 14, no. 1, pp. 78–92, 2022. [45] C. J. Fillmore and C. F. Baker, “Frame semantics for text understanding,” in Proceedings of
WordNet and Other Lexical Resources Workshop, NAACL, 2001, vol. 6. [46] A. Semeraro, S. Vilella, G. Ruffo, and M. Stella, “Writing about COVID-19 vaccines: Emotional profiling unravels how mainstream and alternative press framed AstraZeneca, Pfizer and vaccination campaigns,” arXiv preprint arXiv:2201.07538, 2022. [47] M. Stella, S. De Nigris, A. Aloric, and C. S. Siew, “Forma mentis networks quantify crucial differences in STEM perception between students and experts,” PloS one, vol. 14, no. 10, p. e0222870, 2019. [48] M. Stella and A. Zaytseva, “Forma mentis networks map how nursing and engineering students enhance their mindsets about innovation and health during professional growth,” PeerJ Computer Science, vol. 6, p. e255, 2020. [49] S. De Deyne, D. J. Navarro, and G. Storms, “Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations,” Behavior research methods, vol. 45, no. 2, pp. 480–498, 2013. [50] M. Steyvers and J. B. Tenenbaum, “The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth,” Cognitive science, vol. 29, no. 1, pp. 41–78, 2005. [51] A. P. Christensen, Y. N. Kenett, K. N. Cotter, R. E. Beaty, and P. J. Silvia, “Remotely close associations: Openness to experience and semantic memory structure,” European Journal of Personality, vol. 32, no. 4, pp. 480–492, 2018. [52] M. Montefinese, E. Ambrosini, B. Fairfield, and N. Mammarella, “The adaptation of the affective norms for English words (ANEW) for Italian,” Behavior research methods, vol. 46, no. 3, pp. 887–903, 2014. [53] M. Stella, “Text-mining forma mentis networks reconstruct public perception of the STEM gender gap in social media,” PeerJ Computer Science, vol. 6, p. e295, 2020. [54] D. Chen and C. D. Manning, “A fast and accurate dependency parser using neural networks,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 740–750. [55] N. Silveira et al., “A gold standard dependency corpus for English,” in Proceedings of the Ninth
International Conference on Language Resources and Evaluation (LREC’14), 2014, pp. 2897–2904. [56] G. A. Miller, “WordNet: a lexical database for English,” Communications of the ACM, vol. 38, no. 11, pp. 39–41, 1995. [57] A. Akbik, T. Bergmann, D. Blythe, K. Rasul, S. Schweter, and R. Vollgraf, “FLAIR: An easy-touse framework for state-of-the-art NLP,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 54–59. [58] S. M. Mohammad and P. D. Turney, “Crowdsourcing a word–emotion association lexicon,”
Computational intelligence, vol. 29, no. 3, pp. 436–465, 2013. [59] M. Stella, M. S. Vitevitch, and F. Botta, “Cognitive Networks Extract Insights on COVID-19 Vaccines from English and Italian Popular Tweets: Anticipation, Logistics, Conspiracy and Loss of Trust,” Big Data and Cognitive Computing, vol. 6, no. 2, p. 52, 2022.