<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>IRCDL</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Research and Teaching Public Communication of Science and Technology on Digital Data</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Emanuele Di Buccio</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Federico Neresini</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Information Engineering</institution>
          ,
          <addr-line>Univesity of Padova</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Philosophy, Sociology, Education and Applied Psychology, Univesity of Padova</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Department of Statistical Sciences, University of Padova</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>20</volume>
      <fpage>22</fpage>
      <lpage>23</lpage>
      <abstract>
        <p>In recent decades, there has been a growing interest among Social Science researchers in computational approaches; Computational Social Science and Digital Sociology are examples of these research directions. An interdisciplinary research field that can be framed within Social Science is Public Communication of Science and Technology (PCST), which examines how science and technology can afect contemporary society and how society can afect science and technology. The digitization of traditional media and the proliferation of other information channels, such as Social Media, provide new opportunities for PCST. This paper discusses the issues that need to be addressed to support PCST scholars, possible solutions to address them, and the integration of these solutions into a single platform that is being used to support research and teaching. Concerning teaching, the paper presents an example of how the platform can be used in the context of a university course.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Digital Social Science</kwd>
        <kwd>Research Platform</kwd>
        <kwd>Public Communication of Science and Technology</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Social Science is a broad field of research that includes multiple disciplines, such as Sociology and
Political Science. In recent decades, there has been a growing interest in adopting computational
approaches to deal with the increasing amount of digitized content available [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ]. There is a
large body of work that involves the adoption of Machine Learning (ML), and more generally
AI-based or AI-inspired methodologies to support the research tasks of social scientists [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], to
propose new interdisciplinary methodologies, or to rethink and possibly automate previous
theories and approaches from a new perspective [
        <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
        ].
      </p>
      <p>
        This work will focus on an interdisciplinary field called Public Communication of Science
and Technology (PCST) [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. This field includes the “practice to make specialized knowledge
available for the public” [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]; science communication “is used to inform, engage, persuade, change
behaviors, and support better decision making [. . . ] aims to lift the social, environmental and
economic standing of a nation’s people [. . . ] It may also support the participation of citizens in
setting the agenda for scientific research.” [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Communication plays a crucial role in scientific
research, e.g., to attract attention and to enhance its legitimacy in the eyes of its stakeholders
and potential supporters [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9</xref>
        ]. The public debate on science-related issues may begin when the
debate among experts (scientists) is still “ongoing” – see, for example, the case of cloning. Even
when there is a consensus among scientists, scientific issues become controversial when they
become the subject of public debate. The mass media – both “old” media, such as newspapers,
and “new” media, such as social networks – play an essential role in the public perception
of scientific and technological issues and innovations. For example, they constantly propose
diferent (interpretative) frames that may influence how the public perceives a particular issue.
If, therefore, PCST activities take place mainly in the media arena, this means that they are
characterized by the dual role of the mass media: on the one hand, they constitute a privileged
space within which relationships between science and society occur; on the other hand, the
media themselves contribute to fueling and shaping these relationships. For this reason, the
analysis of media communication on science and technology represents an excellent opportunity
for the social sciences.
      </p>
      <p>
        This paper will consider research and teaching on PCST. We will reconsider the analysis
on user needs conducted in the context of Digital Humanities [
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ] and Social Science
Research [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and report on issues that need to be addressed to support these activities, possible
directions to address them, and our past and current eforts in pursuing these directions. To
show how we can adopt some of the proposed solutions for teaching, we will discuss a recent
activity we carried out during a course for the Master’s Degree in Communication Strategies.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Issues in PCST using Digital Data</title>
      <p>
        Public Communication of Science and Technology encompasses several activities. In this paper,
we will focus on media monitoring, which aims to follow the discourse – in the case of PCST,
the discourse on science and technology – on one or more media channels. One of the channels
traditionally followed is newspapers — in this sense, we can think of newspapers as “old” media
compared to more recent channels such as social media platforms. Data digitization allows PCST
scholars to test their research hypotheses on both “new” articles and historical newspaper data.
Several research projects have focused on designing and developing digital libraries to preserve
and provide access to historical newspaper archives; recent projects include NewsEye [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]
and Impresso [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Research in PCST through newspapers is still relevant today, for instance,
because it ofers the possibility of investigating research hypotheses that require longitudinal
studies. For example, suppose we wanted to follow the discourse of computing technologies or
AI over several decades, starting in the 1960s. Social media are too recent to be a data source
for such research questions. However, social media is an invaluable and necessary source for
other research questions, such as studying the discourse on COVID-19 or Generative AI, newer
viewpoints, or interactions specific to new platforms. Other media channels, such as vlogs or
podcasts, are now available and can be used as data sources.
      </p>
      <p>Therefore, PCST can primarily benefit from working with digitized data and, as we will
discuss later, using computational approaches to uncover how the media present and “frame”
issues, such as those related to science and technology. To achieve this goal, however, several
issues must be addressed and are discussed in the remainder of this section.</p>
      <sec id="sec-2-1">
        <title>2.1. Datasets and continuous media monitoring</title>
        <p>
          Content analysis is a well-established practice in social science research. However, most previous
studies have been based on samples, e.g. a subset of articles on a particular topic. While this
is acceptable for some research questions, other questions require analysis of or comparison
with the “entire population.” For example, when studying the presence of articles on science
and technology over time, working only with articles relevant to science and technology and
looking at absolute frequencies would have led to an incorrect conclusion: that media pay
growing attention to science and technology. However, it is not the case because the relative
frequency is almost constant over time in the last decade [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. Another reason for not working
with samples is the definition of the object of interest. If we are interested in following the
discourse of science and technology in newspapers, why not focus only on the “Science” and
“Technology” sections? The reason is that we will miss part of the discourse: what about the
debate on these issues within articles mainly focused on sport or business?
        </p>
        <p>In the case of newspapers, studies are rarely conducted on all the newspapers available online:
a subset of them is selected to include those that are representative of diferent spins. Even
when a subset of newspapers (and sources in general) is selected, the size of the data, which
is longitudinal in nature, may require a scalable platform to handle it. Media monitoring and
approaches dedicated to handling news content are not new to IAR and NLP: the topic has been
the focus of evaluation campaigns, workshops,1 and projects; dedicated datasets2 have been
created. Therefore, PCST can benefit from these methodologies and resources.</p>
        <p>
          Several platforms allow experts in other disciplines with limited programming skills to work
with data; examples include Knime3, Orange Data Mining,4, and CorText [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. Knime and Orange
allow workflows to be implemented via “visual programming”, specifically by connecting blocks
— nodes in Knime and widgets in Orange; diferent types of blocks exist, e.g., for data collection,
preprocessing, and analysis. CorText allows workflows to be implemented using several existing
functionalities for data collection, subsetting, preprocessing, and analysis. While continuous
data monitoring can be implemented using more advanced or custom functionality in Knime or
by augmenting CorText with other libraries, these solutions may not be easily implemented by
PCST scholars, that can benefit from an integrated environment, complex search functionalities,
and (potentially very large) dataset export capabilities.
        </p>
        <p>A diferent approach is taken by systems such as NOAM [ 16], The European Media
Monitor [17], NewsEye, or Impresso. These systems aim to provide an integrated and ready-to-use
environment. However, none of them meet all the requirements when it comes to continuous
and ongoing monitoring (and not just archives), IAR, and support for the strategies PCST
scholars use to conduct research (see section 2.3) or within teaching activities (see section 3.2).</p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Heterogeneous sources access and processing</title>
        <p>A common methodology in PCST, given a research hypothesis, is to conduct a comparative
study between diferent sources/channels. For example, we can study the same phenomenon,
1See, for example, https://research.signal-ai.com/newsir18/ and https://research.idi.ntnu.no/NewsTech/INRA/
2See, for example, https://catalog.ldc.upenn.edu/LDC2008T19 and https://trec-core.github.io/2018/
3https://www.knime.com/
4https://orangedatamining.com/
e.g. the public debate on AI, in diferent countries and use media channels such as newspapers
and social media as proxies for public opinion. This requires the construction of corpora that
are aligned in terms of the temporal dimension in diferent languages, from diferent channels,
and possibly in diferent modalities. Podcasts, for example, can be a relevant source to study
nowadays since there are many of them focused on news, and besides the content presented in
the episodes, one could also look at the way the content is delivered, e.g., as done in [18], by
considering vocal and conversational properties when predicting seriousness and energy.</p>
        <p>Even when libraries are available to collect and preprocess diferent types of data, such as
those mentioned above, access to the channels can be a problem. Newspapers such as The
Guardian5 or The New York Times6 provide Web APIs. However, services for collecting data
from (some) social media that used to be freely available for research purposes are no longer
available or, if available, require substantial fees to download large amounts of data. This can
severely limit research on these channels.</p>
        <p>In addition to access to sources, another aspect to consider is content heterogeneity. Even
when considering the same modality, e.g., text, diferent channels may require diferent methods.
A well-known example is the case of topic extraction methods, such as Topic Modeling (TM)
algorithms, whose efectiveness may be afected by document length [ 19], which may vary in
datasets consisting of microblog or forum posts such as Reddit, or when diferent sources are
considered simultaneously [20].</p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Workflow support</title>
        <p>When dealing with experts in other fields, such as Humanities or Social Sciences, one
central issue is supporting their workflows . Those experts alternate quantitative and qualitative
approaches to investigate their research questions or, more in general, to accomplish a task.</p>
        <p>
          A discussion on this aspect is reported in [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. Even if the contribution is in the context of
the NewsEye project and about the study of historical newspapers, the authors provided an
abstraction of the problem and proposed a workflow that can adopted as a conceptual tool. The
authors discuss how an interdisciplinary digital hermeneutics workflow is necessary to pursue
the direction of interdisciplinary research that does not consider only the distinct points of view
– the one by humanists and the one by computer scientists – but a joined view. The goal is to
move us away from “supporting their workflow” and towards an interdisciplinary approach. For
instance, it is not always possible to frame a task in “simpler” subtasks and then later translate
them in pipelines: this is not how humanists (and in our case PCST scholars) proceed. For
this reason, [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] proposed a workflow that considers three main aspects: (a) data, (b) iterative
qualitative analytical steps over the data, and (c) critical reflection on data, algorithms, and
tools.
        </p>
        <p>
          As for (a), besides the need to focus on specific subcorpora, another critical aspect is the
curation of the data. In the event of historical newspapers as in [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ], tasks related to this point
included extracting content and metadata from scans or images via OCR technologies. Based on
our experience with the more “recent” online newspapers, we can add that automatic techniques
robust to diverse templates and structures of the pages are required when scaling on the number
5https://open-platform.theguardian.com
6https://developer.nytimes.com
of sources. Another point stressed in [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] is the importance of metadata to get the context of
the themes under investigation, which is crucial for deriving meaning from the data.
        </p>
        <p>
          As for (b), a key aspect is the iterative approach and the high level of interaction required
with the data. In this case, search can be a specific tactic within a more complex strategy to
accomplish a task. In [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ], the authors discuss some relevant tasks when working with historical
newspapers and useful digital interface functionalities to accomplish such tasks:
1. filtering and searching by full-text queries and metadata such as time or newspaper;
2. identification and disambiguation of named entities;
3. identify the first occurrence of words or expressions;
4. study the change in meaning of words over time;
5. extraction of themes (topic in TM), interaction with theme descriptions (labels) for their
interpretation and possible refinement, access to a representation of a document based
on themes, visualization of the prominence of themes over time;
6. advanced search features such as relying on Boolean or regular expressions.
In addition, [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] mentioned the importance of improving search beyond keywords since some
concepts are complex to express by a set of keywords, even if the suggestion/extraction of new
relevant keywords – such as named entities extracted from the subcorpora – could help.
        </p>
        <p>
          The last point (c) is crucial for computer scientists because it requires “openness and
transparency of methods and tools” [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]; this is important both for reproducibility and to make
explicit the assumptions underlying methods and algorithms and the role these assumptions
play in investigating the experts’ research questions.
        </p>
        <p>We observed analogous needs when interacting with PCST scientists [21, 22]; they require:
• better support for IAR, going beyond keyword-based search;
• ways to easily incorporate new and possibly heterogeneous sources for new perspectives
on the public perception of science and technology issues;
• to switch from one (quantitative) strategy to another, to return to a more qualitative
analysis, and then to perform further interactions;
• to compute a set of consolidated or new indicators, e.g., the “risk indicator” [23], on the
subcorpora identified after several iterations.</p>
        <p>Digital platforms to support research and teaching in these areas should be able to provide
these functionalities in an integrated environment or at least facilitate the “implementation” of
complex workflows that are not necessarily linear, but that can support diferent strategies and
the alternation between quantitative and qualitative approaches to data analysis.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Towards Supporting Research and Teaching in PCST</title>
      <sec id="sec-3-1">
        <title>3.1. Research</title>
        <p>In Section 2, we identified three main issues: (i) datasets and continuous monitoring; (ii)
heterogeneous sources access and processing; (iii) lack of supporting workflows for PCST scholars.
In this section, we will describe how we are currently addressing them in an interdisciplinary
project called TIPS7 (Technoscientific Issues in the Public Sphere).</p>
        <p>The first issue, i.e. not to limit the research to samples and to carry out continuous monitoring,
has been addressed by designing and developing a modular software platform, presented
in [21, 22], that stores and provides access to articles collected from fifteen newspapers in
diferent languages; for three of them, archives are available that go back to the 1980s, obtained
by complementing test collections, such as The New York Times Annotated Corpus, or collected
through the available Web API, as for The Guardian. All newspapers are still being monitored
so that we can work on more recent issues. Continuous monitoring required engineering efort
to design and implement a robust architecture. We have integrated search, PoS tagging, named
entity extraction, and topic modeling into a single platform that meets all of the requirements
discussed in the section 2.1 for continuous and ongoing monitoring, IAR, and support for the
strategies used by PCST researchers to conduct their research. Access to the platform requires
authentication; credentials may be requested for research purposes. To support reproducibility,
users can download metadata of the documents used for their analysis; the metadata includes
the article’s URL, which allows access to the full content via the original source.</p>
        <p>Regarding the second issue, i.e. heterogeneous source access and processing, we designed
the platform to work with arbitrary documents and diferent languages. The platform already
stores and provides access to aligned corpora in diferent languages, which allowed us to
perform comparative studies between diferent countries. Even if we do not monitor social
media platforms, existing datasets can be easily included and processed by the existing pipeline.
How to replace channels like the Twitter API is an open question and a solution has not been
found yet. As for the robustness of the algorithms when working with heterogeneous data,
such as [20], we have built temporally aligned test collections for diferent topics — e.g., DNA
and AI — from 2010 to 2022 using data collected from social media and the news; they will be
used to conduct experimental evaluations. The involvement of the PCST scholars provides a
unique opportunity to gain insights into the efectiveness of these approaches on “real” tasks
through qualitative evaluation.</p>
        <p>The third issue, i.e., supporting workflows for PCST scholars, is the central aspect we are
working on. The current functionalities already support filtering and search, named entity
extraction, identifying the first occurrence of words and expressions, and advanced search
features like regex or boolean constraints. A dedicated functionality in the platform allows the
extraction of the top-named entities and nouns for a given query, thus helping users express
their needs better. Additionally, users can work on specific subcorpora, obtained by boolean
queries, directly within the platform, thus utilizing all the functionalities and indicators available
without relying on other platforms or libraries. Some workflows are already supported, e.g.,
search → identify a subcorpus
→ topic extraction on the subcorpus
→ topic description and top docs per topic analysis
→ analysis of the evolution of topics over time
→ identify a subcorpus using a subset of the topics and re-extract them
Even if the platform supports multiple iterations, the above workflow is limited to search and
TM. More articulated workflows can be integrated into the platform, e.g., that proposed in [ 24]
to study the case of energy transition in Italian newspapers; that workflow involves additional
techniques such as named entity recognition to identify prominent actors, and graph-based
representations obtained from the articles’ content to study actors relationships. Section 3.2
will present another possible workflow for PCST scholars and students.</p>
        <p>Dealing with large amounts of data in terms of information access and automatic extraction
of valuable and usable representations is not the only reason to introduce experts in PCST to
computational approaches. Another reason is the increasing attention some of these Computer
Science disciplines are receiving in the public sphere today. AI is becoming a prominent topic in
the media and for political institutions, thanks to the progress achieved through new (computer)
architectures and models and their widespread potential applications. Emerging technologies,
such as AI-based chat-bots, are becoming controversial for their potential future impact on
society, and institutional organizations have proposed specific regulations. 8 Sociologists and
communication experts may be directly afected by these emerging technologies, and their
role may be critical in discussing or communicating the implications for society. For example,
when considering Generative AI, one concern is whether automatically generated content
will become dominant. How might that afect society? In addition to the impact on political
orientations or public perception, could other aspects be afected? These types of questions are
relevant to PCST research activities that focus on the impact of digital technology on society
and social interactions. These questions can also be the subject of teaching activities for future
communication and sociology professionals, as we will discuss in the next section.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Teaching</title>
        <p>
          The last remark on section 3.1 allows us to introduce another aspect, i.e. the introduction of
students of social and communication sciences to topics in IAR, NLP and ML. This need has been
the rationale for interdisciplinary courses such as those in Digital Humanties and Computational
Social Science, which have been ofered for several years. As discussed in [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] in the context
of research on historical newspapers, “historians need to acquire new skills, especially in the
practice of (digital) hermeneutics, which refers to the interpretation and understanding of large,
digitized or digitally born data sources.” The same is true for students (and scholars) in PCST.
To promote an appropriate level of understanding of IAR, NLP, and ML topics, and to foster
debate among the “future” communication professionals, it is imperative to introduce students
to these concepts and also to some of the current and possible future implications. Following this
direction, in this section, we will present a teaching activity we carried out on text classification
to identify articles on emerging technologies; moreover, we will discuss other activities we plan
to carry out to complement the first one.
        </p>
        <p>As an example of teaching activities that might benefit students in Sociology and
Communication, we report on recent experience in the course of Digital Sociology in the Master Degree
of Communication Strategies at the University of Padova. The objective of the course is to
introduce the students to epistemological and methodological issues concerning Digital Social
Research, to digitalizing traditional methods (e.g., web surveys), data-driven social research, and
making social research on Social Media and through Social Media, and digitalized newspapers.</p>
        <p>As part of the course, the students were presented with how to represent unstructured data,
such as newspaper articles, and ML techniques to analyze them, more specifically, supervised
text classification on a specific object of study: emerging technologies. The task was framed
8https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/
eu-ai-act-first-regulation-on-artificial-intelligence
as a binary classification problem where the goal was to determine whether a document was
about emerging technologies. In the first lecture on this activity, the students were introduced
to the demarcation problem, i.e., to the problem of determining the object of interest for the
study; the goal was to identify a set of criteria then used to classify a document concerning
relevance to emerging technologies. Even if the lecturers determined some criteria before the
beginning of the activity, those criteria were not presented to the students, who were left to
come up with the criteria and agree on those that must be adopted. The result of this activity
was the following criteria: (i) a technology framed as "new" is mentioned; (ii) some impacts, i.e.,
changes, on society are described; (iii) a technology related to scientific research is mentioned.</p>
        <p>After determining the criteria, we built a dataset using articles from six English newspapers9
published from January 1, 2016, to November 7, 2023. The articles were retrieved by merging
the results from the following queries:
• "emerging technology" OR "emerging technologies"
• chatgpt
• "fusion energy"
• "genome editing"
• "neuralink"
• "self driving car" OR "autonomous car" OR "driverless car" OR "robotic car" OR "google
car" (and the corresponding version with "cars")
The expression between double quotes is interpreted as phrase queries (the constituting words
must occur near each other as in the string). We considered candidate non-relevant documents
those not returned as results for those queries. We set as an additional filter that articles must
be relevant to Science and Technology issues, according to a previously developed classifier —
see [21] for details on the manually labeled dataset and [22] for the approach used.</p>
        <p>Then, we extracted a random sample of 658 documents that covered the diverse queries and
(possibly) non-relevant documents; the sample was explicitly created, maintaining a balance
between the two classes. The sample was then delivered to the students, who manually labeled
14 documents, each using the criteria. A total of 46 students were involved in the activity. Along
with labeling relevant and non-relevant documents, students were asked to perform a quality
check to identify duplicates or documents with incomplete text due to our extraction procedure.
After labeling the assigned 14 documents, students were asked to label the documents from
another student. After the labeling was performed, the students were divided into groups. They
were asked to discuss the given labels and to agree on each document label, paying particular
attention to problematic cases. The results were then provided to the lecturer. Some documents
were removed because of the quality check; the resulting dataset consists of 642 documents.</p>
        <p>In the following lecture, the students were introduced to fundamental notions on Computer
Science, such as the notion of “algorithm”, and on ML learning, focusing on supervised text
classification. The experience of labeling was instrumental in this subsequent lecture. Part of
the lecture was devoted to presenting the efectiveness of some classifiers trained on the labeled
dataset produced by the students. Even if the resulting dataset was small, we trained several
classifiers using 5-fold cross-validation as a proof of concept. We used the JSAT Library [ 25],
9Mirror, The Guardian, The Telegraph, The New York Times, The Times of London, and The Financial Times
the one currently adopted “in production” in TIPS. The tested classifiers were Multinomial
Naive Bayes (MNB), Logistic Regression with Coordinate Descent Methods (LRDCD) [26], and
Stacking [27] of these two classifiers; the focus on these approaches was motivated by the
previous encouraging results observed for classifying Science and Technology articles [22].</p>
        <p>The subsequent lecture was devoted to current and possible future implications of ML
approaches and technologies, e.g., approaches and technologies using behavioral data and
Large Language Models, on society. The participation of the students and the constructive
interactions were perceived as an indicator of a positive and valuable experience, and the main
methodological aspects related to IAR, NLP, and ML were acquired by the students.</p>
        <p>The students were then given a new sample to label and, later, the predictions based on the
most efective classifier (the one using Stacking) to check the predictions’ correctness. The
labeling procedure followed the same approach adopted in the previous phase: labeling 14
documents and then discussing the labeling within the group. In the second phase, we got fewer
labeled documents (383). As for comprehensiveness, Table 1 reports the results of the three
classifiers on the full dataset; article URLs and labels of the adopted dataset are available in [ 28].
The obtained results suggest additional work should be done, e.g., increasing the size of the
labeled set, before using the classifier for research purposes.</p>
        <p>Text classification is one of many topics that can benefit teaching activities. The availability
of classified and indexed longitudinal corpora allows us to present methodological aspects, such
as the importance of working with the entire population to monitor some phenomena — see
the example of the erroneous interpretation related to absolute frequency reported in Section 2.
Another activity might be to show some representations obtained from the classified data.
For example, we might ask if the perception of risk when discussing emerging technologies
has changed over time. Can NLP and ML help? We might extract temporal word embedding
representations and compare the distance among the embedding representation of the terms
“risk” and “emerging technology” over time.</p>
        <p>As a proof of concept and a basis for a possible activity to carry out for the next edition of
the course, we considered all the documents answering the queries reported above and merged
the obtained results; 80494 documents constitute the resulting subcorpus. We preprocessed
them, replacing all the terms in the query constituted by more than one token, e.g., emerging
technology, with a string to denote the entire expression; we then transform the text using
lemmas instead of the original words — we used Stanza [29] for the extraction of the lemmas.
Then, we trained a model using 100 dimensions to represent each word, 5 static iterations, and
5 dynamic iterations as suggested in [30]. We then computed the distance (cosine similarity)
between “risk” and “emerging technology” over time; the trend is reported in Figure 1, specifically
the line connecting points depicted by circles. The other line (points depicted as squares) refers
to the similarity between the term “’risk” and the term “emerging technology”, when in the
preprocessing step, all the terms used in the query – “chatgpt”, “fusion energy”, “genome editing”,
“neuralink” and the diferent variants of “autonomous car” – where replaced by “emerging
technology”; the basic idea underlying the second approach was to have a measure of the
relationship among risk and emerging technologies when considering all the technologies
of the case study. Both cases show a peak in 2023. One can then look at the words closest
to “emerging technology” to interpret the results; the top 20 per year are reported in Table 2
when not “collapsing” the diverse queries. In the case of “emerging technologies collapsed”, we
observed words such as “AI”, “generative”, “vehicle”, “robot”, “automation”, “autopilot”; those
words, along with “cybersecurity”, might suggest a possible interpretation of the reasons for
a peak of the closeness to risk. A more fine-grained analysis based on the actual documents
from that year must then be adopted to confirm the result, and that requires advanced search
functionalities to retrieve documents relevant to the task. This is an example of workflow
mentioned in Section 2 and that we aim to support.</p>
        <p>Another example, relying on TM algorithms, might help to introduce the discussion on
controversial issues rooted or related to research on IAR, ML, or AI in general. These
indications by PCST scholars might lead to novel research problems to address and result in novel
algorithms and paradigms. For example, we considered two newspapers: The New York Times
and The Guardian. We used the articles available in TIPS from 1999 to 2022. That resulted
in 4,556,415 documents. Then we extracted all the articles answering the query: (”search
engine” OR ”information retrieval” OR ”machine learning” OR ”artificial
intelligence”) in the time interval 1999-2022; 1999 was selected as the starting year because
the number of articles in The Guardian seems to be small before that date and we wanted to
align the two corpora. We then extracted 30 topics using LDA10 with 500 iterations and the
stop-word list provided by the library. Table 3.2 reports a subset of the topics. For instance,
the top documents from topic 8 suggest several concerns related to social media platforms and
content, such as misinformation. Topic 14 is focused on preoccupations associated with climate
change and the environment. Still, top documents also include how these issues can benefit
from AI research results and advancement in the field. Topic 25 concerns recent advances
in large language models, and the discussion includes the impact that might have on society.
Topic 29 includes a discussion on what AI can bring to Art, but also concerns on the problem
of copyright infringement due to some IR and AI technologies; these concerns, for instance,
resulted in publishers asking governments to protect their work which is “ingested” by AI-based
technologies. Those mentioned in the last paragraphs are only a few examples of the relationship
between science, technology, and society.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Final remarks</title>
      <p>This paper discussed how PCST can benefit from working on Digital Data. We explicitly
discussed some issues that need to be addressed, relying on our experience on an ongoing
interdisciplinary research project called TIPS and previous works on Historical Newspapers
Archives. The solutions to some of these issues are already integrated into the TIPS platform.
Section 3 discussed how our efort is helpful for research and teaching activities. Besides the
specific activity considered in the paper, we should also point out that the platform has been
actively used for several years by Bachelor’s, Master’s, and Ph.D. students for their theses, which
generally focus on specific issues concerning technologies and their impact on society.</p>
      <p>A large body of work still needs to be done to provide better support, e.g., implementing
other workflows like that described in Section 3.2 or proposed in [ 24] directly into the platform.</p>
      <p>Moreover, some platforms discussed in the paper are useful for research activities; are they
10https://mimno.infosci.cornell.edu/jsLDA/</p>
      <p>Top words
apple mobile android gt&amp;gt app phone iphone apps google phones
students university science education school professor research universities computer online
human intelligence computer artificial machine machines humans computers language learning
game games video virtual players play player real reality film
internet software your users use information web data system computer
company said companies percent market business billion investors chief money
your home like technology devices voice into smart amazon assistant
facebook social users media twitter news content tech youtube company
data how used such says could research information about use
said online advertising internet ads service companies business web site
health patients medical cancer care nhs said doctors patient disease
robots robot space human robotics weapons artificial intelligence military robotic
energy climate water species said food carbon could change global
said data privacy about information public government law had use
brain science human his life scientists scientific mind book consciousness
google google’s microsoft said companies company amazon search european tech
said technology software like research computer company a.i companies data
jobs workers work job economy automation economic report skills employees
cars car vehicles self-driving autonomous uber travel tesla vehicle driving
search web site sites information engine online pages your find
said covid vaccine health coronavirus were pandemic virus cases had
google search engine yahoo google’s microsoft users company results internet
chatgpt said technology about artificial intelligence google musk use privacy
says digital technology business media social guardian such director marketing
china chinese government said united states american companies technology china’s
even about future technology power way might just too much
music books art book library digital copyright work artists into
efective also for educational activities? How can we improve them to support teaching better
and allow practice on some methodologies/research issues? Should we provide the users with
novel search or analysis primitives for better support?</p>
      <p>The active participation by experts is a unique opportunity, especially for the evaluation of
the efectiveness of IAR, NLP and ML. For this reason, we plan to extend the platform to gather
more feedback from the users, e.g., allowing them to specify additional annotations during the
labeling procedure, such as notes on why the document was perceived as relevant.</p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgments</title>
      <p>The authors would like to thank all the students of the Digital Sociology course in the Master
of Communication Strategies at the University of Padova. Without their commitment and
participation, the teaching activity reported in this paper would not have been possible.
ager, 2016. URL: https://docs.cortext.net.
[16] I. Flaounas, O. Ali, M. Turchi, T. Snowsill, F. Nicart, T. De Bie, N. Cristianini, Noam:
News outlets analysis and monitoring system, in: Proceedings of the 2011 ACM SIGMOD
International Conference on Management of Data, SIGMOD ’11, Association for Computing
Machinery, New York, NY, USA, 2011, p. 1275–1278. doi:10.1145/1989323.1989474.
[17] R. Steinberger, B. Pouliquen, E. van der Goot, An introduction to the Europe Media Monitor
family of applications, in: Proceedings of the SIGIR 2009 Workshop on Information Access
in a Multilingual World, volume 43, 2009. arXiv:1309.5290.
[18] L. Yang, Y. Wang, D. Dunne, M. Sobolev, M. Naaman, D. Estrin, More than just words,
ACM, 2019, pp. 276–284. URL: https://dl.acm.org/doi/10.1145/3289600.3290993. doi:10.
1145/3289600.3290993.
[19] J. Qiang, Z. Qian, Y. Li, Y. Yuan, X. Wu, Short Text Topic Modeling Techniques, Applications,
and Performance: A Survey, IEEE Transactions on Knowledge and Data Engineering 34
(2022) 1427–1445. doi:10.1109/TKDE.2020.2992485. arXiv:1904.07695.
[20] J. Qiang, P. Chen, W. Ding, T. Wang, F. Xie, X. Wu, Heterogeneous-Length Text Topic
Modeling for Reader-Aware Multi-Document Summarization, ACM Transactions on
Knowledge Discovery from Data 13 (2019) 1–21. doi:10.1145/3333030.
[21] A. Cammozzo, E. Di Buccio, F. Neresini, Monitoring technoscientific issues in the
news, in: ECML PKDD 2020 Workshops - Workshops of the European Conference
on Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2020):
SoGood 2020, PDFL 2020, MLCS 2020, NFMCP 2020, DINA 2020, EDML 2020, XKDD 2020
and INRA 2020, Ghent, Belgium, September 14-18, 2020, Proceedings, volume 1323
of Communications in Computer and Information Science, Springer, 2020, pp. 536–553.
doi:10.1007/978-3-030-65965-3\_37.
[22] E. Di Buccio, A. Cammozzo, F. Neresini, A. Zanatta, TIPS: search and analytics for social
science research, in: L. Tamine, E. Amigó, J. Mothe (Eds.), Proceedings of the 2nd Joint
Conference of the Information Retrieval Communities in Europe (CIRCLE 2022), Samatan,
Gers, France, July 4-7, 2022, volume 3178 of CEUR Workshop Proceedings, CEUR-WS.org,
2022. URL: https://ceur-ws.org/Vol-3178/CIRCLE_2022_paper_33.pdf.
[23] E. Di Buccio, A. Lorenzet, M. Melucci, F. Neresini, Unveiling latent states behind social
indicators, in: R. Gavaldà, I. Zliobaite, J. Gama (Eds.), Proceedings of the First Workshop on
Data Science for Social Good co-located with European Conference on Machine Learning
and Principles and Practice of Knowledge Dicovery in Databases, SoGood@ECML-PKDD
2016, Riva del Garda, Italy, September 19, 2016, volume 1831 of CEUR Workshop Proceedings,
CEUR-WS.org, 2016. URL: https://ceur-ws.org/Vol-1831/paper_6.pdf.
[24] F. Neresini, P. Giardullo, E. D. Buccio, A. Cammozzo, Exploring socio-technical future
scenarios in the media: the energy transition case in italian daily newspapers, Quality and
Quantity 54 (2020) 147–168. doi:10.1007/s11135-019-00947-w.
[25] E. Raf, Jsat: Java statistical analysis tool, a library for machine learning, Journal of</p>
      <p>Machine Learning Research 18 (2017) 1–5. URL: http://jmlr.org/papers/v18/16-131.html.
[26] H.-F. Yu, F.-L. Huang, C.-J. Lin, Dual coordinate descent methods for logistic regression
and maximum entropy models, Machine Learning 85 (2011) 41–75. URL: https://doi.org/
10.1007/s10994-010-5221-8. doi:10.1007/s10994-010-5221-8.
[27] D. H. Wolpert, Stacked generalization, Neural Networks 5 (1992) 241–259. doi:10.1016/</p>
      <p>S0893-6080(05)80023-1.
[28] E. Di Buccio, F. Neresini, Data from: Research and Teaching Public Communication of</p>
      <p>Science and Technology on Digital Data, 2024. doi:10.5281/zenodo.10616684.
[29] P. Qi, Y. Zhang, Y. Zhang, J. Bolton, C. D. Manning, Stanza: A Python natural language
processing toolkit for many human languages, in: Proceedings of the 58th Annual Meeting
of the Association for Computational Linguistics: System Demonstrations, 2020. URL:
https://nlp.stanford.edu/pubs/qi2020stanza.pdf.
[30] V. Di Carlo, F. Bianchi, M. Palmonari, Training temporal word embeddings with a
compass, in: 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative
Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI
Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, 2019, pp. 6326–6334.
doi:10.1609/aaai.v33i01.33016326. arXiv:1906.02376.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>N.</given-names>
            <surname>Marres</surname>
          </string-name>
          ,
          <article-title>Digital sociology : the reinvention of social research / Noortje Marres</article-title>
          , Polity Press, Cambridge Malden,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>G. A.</given-names>
            <surname>Veltri</surname>
          </string-name>
          , Digital social research / Giuseppe A.
          <string-name>
            <surname>Veltri</surname>
          </string-name>
          , Polity, Cambridge (UK)
          <article-title>Medford (MA</article-title>
          , USA),
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>P.</given-names>
            <surname>DiMaggio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Nag</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Blei</surname>
          </string-name>
          ,
          <article-title>Exploiting afinities between topic modeling and the sociological perspective on culture: Application to newspaper coverage of U.S. government arts funding</article-title>
          ,
          <source>Poetics</source>
          <volume>41</volume>
          (
          <year>2013</year>
          )
          <fpage>570</fpage>
          -
          <lpage>606</lpage>
          . arXiv:
          <fpage>9605103</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>D.</given-names>
            <surname>Odijk</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Burscher</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Vliegenthart</surname>
          </string-name>
          , M. de Rijke,
          <article-title>Automatic thematic content analysis: Finding frames in news</article-title>
          , in: A.
          <string-name>
            <surname>Jatowt</surname>
            ,
            <given-names>E.-P.</given-names>
          </string-name>
          <string-name>
            <surname>Lim</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <string-name>
            <surname>Ding</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Miura</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Tezuka</surname>
            , G. Dias,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Tanaka</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Flanagin</surname>
            ,
            <given-names>B. T.</given-names>
          </string-name>
          Dai (Eds.),
          <source>Social Informatics</source>
          , Springer International Publishing, Cham,
          <year>2013</year>
          , pp.
          <fpage>333</fpage>
          -
          <lpage>345</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bucchi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Trench</surname>
          </string-name>
          ,
          <article-title>Science communication research: themes and challenges</article-title>
          , Routledge, New York,
          <year>2014</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>P.</given-names>
            <surname>Catapano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Fayard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. V.</given-names>
            <surname>Lewenstein</surname>
          </string-name>
          ,
          <source>The Public Communication of Science and Technology and International Networking</source>
          , Springer Netherlands, Dordrecht,
          <year>2003</year>
          , pp.
          <fpage>31</fpage>
          -
          <lpage>42</lpage>
          . doi:
          <volume>10</volume>
          .1007/
          <fpage>978</fpage>
          -94-017-0801-
          <issue>2</issue>
          _
          <fpage>3</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P.</given-names>
            <surname>Broks</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Gascoigne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Leach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. V.</given-names>
            <surname>Lewenstein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Massarani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Riedlinger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Schiele</surname>
          </string-name>
          ,
          <article-title>Communicating science: a global perspective</article-title>
          , ANU Press,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Pansegrau</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Shukla</surname>
          </string-name>
          ,
          <article-title>The Cultural Authority of Science: Comparing across Europe, Asia, Africa and the Americas</article-title>
          ,
          <source>Routledge Studies in Science, Technology and Society</source>
          , Taylor &amp; Francis,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>P.</given-names>
            <surname>Magaudda</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Neresini</surname>
          </string-name>
          ,
          <article-title>Gli studi sociali sulla scienza e la tecnologia, Manuali</article-title>
          . Sociologia, Il Mulino,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>E.</given-names>
            <surname>Pfanzelter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Oberbichler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Marjanen</surname>
          </string-name>
          , P.-C. Langlais,
          <string-name>
            <given-names>S.</given-names>
            <surname>Hechl</surname>
          </string-name>
          ,
          <article-title>Digital interfaces of historical newspapers: opportunities, restrictions and recommendations</article-title>
          ,
          <source>Journal of Data Mining &amp; Digital Humanities HistoInformatics</source>
          (
          <year>2021</year>
          ). URL: https://jdmdh.episciences.org/ 6121. doi:
          <volume>10</volume>
          .46298/jdmdh.6121.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>S.</given-names>
            <surname>Oberbichler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Boroş</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Doucet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Marjanen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Pfanzelter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Rautiainen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Toivonen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Tolonen</surname>
          </string-name>
          ,
          <article-title>Integrated interdisciplinary workflows for research on historical newspapers: Perspectives from humanities scholars, computer scientists, and librarians</article-title>
          ,
          <source>Journal of the Association for Information Science and Technology</source>
          <volume>73</volume>
          (
          <year>2022</year>
          )
          <fpage>225</fpage>
          -
          <lpage>239</lpage>
          . doi:
          <volume>10</volume>
          .1002/ asi.24565.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>A.</given-names>
            <surname>Doucet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gasteiner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Granroth-Wilding</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Kaiser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Kaukonen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Labahn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.- P.</given-names>
            <surname>Moreux</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Muehlberger</surname>
          </string-name>
          , E. Pfanzelter,
          <string-name>
            <surname>M.-E. Therenty</surname>
          </string-name>
          , et al.,
          <article-title>Newseye: A digital investigator for historical newspapers</article-title>
          ,
          <source>in: 15th Annual International Conference of the Alliance of Digital Humanities Organizations, DH</source>
          <year>2020</year>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>Impresso</given-names>
            <surname>Project</surname>
          </string-name>
          ,
          <year>2023</year>
          . URL: https://impresso-project.
          <source>ch/.</source>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>F.</given-names>
            <surname>Neresini</surname>
          </string-name>
          ,
          <article-title>Old media and new opportunities for a computational social science on PCST</article-title>
          ,
          <source>Journal of Communication</source>
          <volume>16</volume>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>P.</given-names>
            <surname>Breucker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.-P.</given-names>
            <surname>Cointet</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. Hannud</given-names>
            <surname>Abdo</surname>
          </string-name>
          , G. Orsal, C. de Quatrebarbes, T.-K. Duong,
          <string-name>
            <given-names>C.</given-names>
            <surname>Martinez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. P.</given-names>
            <surname>Ospina Delgado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. D.</given-names>
            <surname>Medina Zuluaga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. F.</given-names>
            <surname>Gómez Peña</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. A.</given-names>
            <surname>Sánchez Castaño</surname>
          </string-name>
          , J. Marques da Costa,
          <string-name>
            <given-names>H.</given-names>
            <surname>Laglil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Villard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Barbier</surname>
          </string-name>
          ,
          <string-name>
            <surname>Cortext</surname>
          </string-name>
          man-
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>