Online consultation platforms have improved the possibilities for citizens to have an input on public decision making. However, and especially at large scale, identification of the topics discussed and entities evoked has been identified as dificult for both citizens and platform administrators. In this paper, we leverage topic modeling, Named Entity Recognition and Linking and Semantic Textual Similarity to build a knowledge graph representing the diferent contributions to the citizen consultation in French language. The generated graph links the diferent proposals to topics identified in the consultation and to relevant DBpedia resources. The model proposed for representation of citizen consultations as knowledge graphs simplifies the retrieval of proposals focused on specific topics or mentioning a given entity. It also allows us to improve contextualization of important words in proposals by linking them to short definitions extracted from Wikipedia.
Citizen participation has become increasingly popular in recent decades [1, 2] to involve citizens in public decision-making [3]. More specifically, in comparison with their ofline counterparts, online participatory platforms allow to gather a larger, more diverse audience, therefore helping to produce results more representative of the consulted population’s opinions. Online participatory platforms have been used in a wide variety of use cases, such as participatory budgeting [4, 5], citizen consultation [6, 7], or gathering and release of open data produced by citizens [8]. These platforms have addressed a wide range of issues, such as public space planning [9], digital transformation of countries [10], and climate change mitigation and adaptation [11, 12].
LGOBE (R. Angarita)
Consultations are a means of allowing citizens to ofer input on a given topic to guide the actions taken by a given institution. Even though institutions are not necessarily constrained by consultation outcomes, consultations are often regarded as an important means of improving citizens’ impact on public policies. Online consultation platforms typically allow users to express their ideas using text and sometimes other media. These proposals are then either directly sent to the institution or shared publicly to solicit feedback from citizens using an upvote/downvote system or written comments.
Online citizen participation platforms have been used in various countries [11] and frequently used consultation techniques. Examples of online citizen consultations at the local scale [13], which are the most common, generally focus on urban planning or local policies. An example of a large-scale online consultation can be found in the République Numérique (Digital Republic, RepNum for short in the following) consultation. This consultation aimed at letting citizens and other stakeholders publish opinions and amendments to a draft bill focused on the digital transformation of the French administration, open data, internet access, data protection, and open science. By the end of the consultation, 692 proposals had been submitted on the platform.
One of the challenges of using public consultations is managing the high volume of information that they generate, especially in large-scale consultations [14, 15]. This information overload can be overwhelming for both citizens and platform administrators. Citizens may have dificulty finding contributions related to their areas of interest or expertise, while platform administrators may struggle to read and summarize all of the contributions. This challenge is an open one that has been studied by researchers and experts in the field. In addition, this issue is exacerbated by the fact that most participatory platforms rely on simple methods for information retrieval, such as text search without relevance assessment or complex criteria. However, using preexisting resources from a knowledge graph has been proven to be an efective tool for information retrieval, as demonstrated by previous research [16, 17]. Therefore, this approach can be used to address the issue of information overload faced by stakeholders in participatory platforms.
Other issues have also been identified such as the dificulty of reaching a consensus in a collaborative environment and the absence of efective communication between institutions and citizens [18].
In comparison with existing search systems in public consultation platforms, the use of a knowledge graph facilitates collaboration between users to improve the quality of debates and can lead to an accurate decision-making support tool. Moreover, graph-based algorithms, recommender systems, and semantic reasoning based on users’ votes and comments can enable advanced data analytic and reasoning capabilities to identify the most relevant proposals by topic and cross topics [19, 20].
In this work, we propose to build a knowledge graph of public consultations to address the challenges discussed above and enhance such participatory processes. Its contributions are the following: i) the enrichment of online public consultation data with linked data to improve the contextualization of texts, facilitate its integration with other linked data sources including other consultations, increase transparency and accountability, and enhance decision making; ii) the analysis of proposals to infer their topics for better organization and understanding of the information contained within the consultations. By categorizing consultations based on topics, it can be easier to identify trends and patterns in public opinion and to compare and contrast diferent viewpoints; and iii) we provide recommendations for future work based on the specific characteristics of the public consultations of our datasets.
The rest of the paper is structured as follows. Section 2 gives an overview of some of the relevant related works. In section 3, we describe the approach adopted to build the online public consultation knowledge graph. Section 4 discusses the proposed solution and dresses perspectives for future work. Code used in the production of this paper is available at https: //github.com/WilliamAboucaya/repnum-kg.
Knowledge Graphs (KGs) represent facts in the form of nodes and relationships [21, 22]. They use a graph-based data model to capture knowledge in scenarios that involve integrating, managing and extracting data from diverse sources [23]. The two main graph models are RDF [24] and Property Graphs [25]. RDF is a W3C standard RDF for data interchange on the Web. Nodes are resources with a unique identifier on the Web. Edges link the nodes by building subject–predicate–object expressions. Property graphs are directed labeled multigraphs in which nodes and edges can have properties in the form of key-value pairs. Nodes represent entities and edges represent relationships between those entities. Both RDF and Property Graphs have their advantages and disadvantages, depending on the use case [26, 27].
The creation of a KG from text involves the use of Natural Language Processing (NLP) techniques, including Named-Entity Recognition (NER) [28] and Named-Entity Linking (NEL) [29]. NER focuses on identifying and categorizing entities within text, such as people, organizations, and locations, among others. NEL, on the other hand, connects the recognized entities to entities within a knowledge base. Several methods for constructing domain-specific KGs have been developed across a range of domains, including fashion, medicine, and more [30, 31, 32, 33].
Barroca et. al. [30] applied Named Entity Recognition (NER) and Named Entity Linking (NEL) techniques on product textual descriptions to enhance a fashion-related KG. The process involved identifying entities, such as materials used in the product, within the descriptions and linking them to their corresponding, unique nodes in the KG. Rincon-Yanez et. al. [31] proposed a methodology for creating KGs related to novels. This methodology involves using NLP techniques such as NER and NEL to generate simple triples as a starting point, and then enhancing them using external open knowledge sources, such as DBpedia, to build a comprehensive KG. This methodology was tested on a novel, resulting in the extraction of characters and their relationships. In a similar vein, Alpizar-Chacon and Sosnovsky [34] proposed a solution which automatically labels book’s index item in relation to the main subject of a textbook area. These index concepts can be associated with the central themes of the book, common concepts, topics related to but distinct from the book’s subject area, or unrelated to the book’s subject area altogether.
Reklos and Meroño-Peñuel [32] developed an approach for identifying causal relationships in medical publications. Their approach involves detecting causal sentences, extracting entities, and using this information to build a KG that represents the causal relationships. For example, a node representing the cause of an illness would be connected to a node representing the illness.
Another relevant application of KGs is the analysis of parliamentary debates, as demonstrated by Tamper et. al. [33]. They proposed a method for constructing and enhancing KGs of plenary debates in the Parliament of Finland. The methodology involves preprocessing the text, including lemmatization to normalize the text and subject indexing to describe the text succinctly, due to the particularities of the Finnish language. NER and NEL techniques are then applied to extract named entities and link them to the KG.
NLP techniques have been utilized in various ways to improve public consultations [35, 36, 37]. Weng et. al. [35] aimed to aggregate individual contributions into common narratives for improved collaboration by extracting sentences in the form of subject + verb + object and verb + object. Similarly, the New Zealand government categorized proposals in a public consultation using NLP, as per [36]. Another NLP application in a public consultation was done in collaboration with the Italian Ministry of Education, where key concepts and frequently used verbs were extracted from the answers, as demonstrated by Caselli et. al. [37].
In [38], Cantador et. al.proposed an approach to identify controversial proposals in a consultation by utilizing external knowledge extracted from open government data collections. Moreover, for building the thematic taxonomy, authors manually set categories and assign a platform’s pre-existing tags to appropriate categories. Despite the progress made using NLP techniques on public consultations, to the best of our knowledge, our proposed methodology for constructing online public consultation KGs based solely on the textual content submitted by contributors is a novel approach.
Figure 1 shows the diferent stages of our methodology for building the consultation KG. We detail these stages in the rest of this section by building a KG for the RepNum consultation. This consultation took place from September 26th, 2015 to October 18th, 2015, and received 692 proposals, including the 30 original articles of the bill generated by the organizing ministry. The consultation data is publicly available as open data through the following link: https://www.data. gouv.fr/fr/datasets/consultation-sur-le-projet-de-loi-republique-numerique. It includes the contributions of 21,464 users, almost 150,000 votes, and 6,000 arguments – which are comments identified as for or against a proposal.
We use Latent Dirichlet Allocation (LDA) [39] on the proposals submitted to the RepNum consultation to identify the diferent topics. Each proposal is represented as a document, and LDA is an algorithm for topic modeling in a set of documents where each topic is associated with the diferent N-grams 1 based on their number of occurrences in the diferent documents. Each proposal is then associated with all relevant topics based on a truth value score. Using this algorithm, we need to choose the number of topics to identify. We applied LDA for ∈ [4, 15] and assessed the suitability of each version of the topic modeling based on the consistency of the 20 most frequent words for each topic. We consider = 5 to be the parameter giving the most coherent results for the RepNum citizen consultation. Figure 2 shows the five main topics identified and their most frequent words. We use these topics, in addition to the proposal’s title, author, and content, to further characterize the diferent proposals.
We use a French-language adaptation of DBpedia Spotlight [40, 41] (DBS for short in the following) to identify entities in our proposals. DBS extracts important words from a given text and links them to resources in DBpedia [42]. We have set the “confidence” parameter to 0.6 and left everything else to default. Our observations about DBS concern the French model. Linking 1A sequence of 1 or more words usually written consecutively. For example, both “text” and “knowledge graph” are N-grams in the context of this workshop. the most important elements in the proposals to DBpedia resources allows us to associate these elements with definitions from Wikipedia and connect proposals based on the resources they mention. To eliminate certain common false associations identified in preliminary tests, we iflter the obtained results: • DBS tends to identify years and dates as important elements of the proposals and link them with the aggregation Wikipedia page for events happening at this given date - e.g., https: //en.wikipedia.org/wiki/December_13-. In the context of an online consultation, such elements are often mentioned in contexts where this linking is not relevant. Therefore, we exclude these resources before performing NEL. • We also observe that the model tends to identify certain N-grams as film titles. This has almost only led to incorrect identification of common N-grams as film titles in the context of the RepNum consultation. Consequently, we have filtered non-documentary iflms out of the resource graph with which we associate our entities. The choice to keep documentary films is made to give citizens the possibility to add references to support their proposals.
Then, we identified an additional issue: acronyms with multiple definitions are often associated with the wrong resource. For example, the acronym “CADA” is used in a proposal to mention the “Administrative Documents Access Commission” (in French, Commission d’Accès aux Documents Administratifs). However, this entity is linked to “Shelter for Asylum Seekers” (in French, Centre d’Accueil de Demandeurs d’Asile), even though “administrative documents” is mentioned further in the proposal. This issue has been confirmed with similar results by using DBS on another online consultation called RUA (Open data available at https://www.data.gouv.fr/fr/datasets/consultation-vers-un-revenu-universel-dactivite-1/). We solve this issue by applying the following process to each acronym identified as important in a proposal.
• If DBpedia has a node with a name identical to the acronym: – If this node is a specific resource (e.g., a Wikipedia page) or redirects to it, we associate the acronym with this resource. – Else, if the node is a disambiguation node for multiple resources, we gather the abstract for each resource. Then, using the SBERT [43, 44] model for Semantic Textual Similarity mining, we apply cosine similarity to each abstract with the proposal containing the acronym. Finally, we associate the acronym with the resource whose abstract has the highest cosine similarity with the proposal. • Else, using the Wikipedia API, we search for the acronym and gather the 5 best results.
Then, similarly to the previous dash, we use SBERT’s cosine similarity to identify the Wikipedia page whose abstract is the most similar to the proposal. Finally, we link the acronym with the DBpedia resource associated with the identified Wikipedia page.
The methods NER and NEL mentioned earlier generate a set of proposals where certain N-grams are associated with a DBpedia resource. The obtained results are presented in Table 1. A considerable proportion of the proposals (16.2 %) are not associated with any entity, likely because these proposals are shorter than others. Specifically, while the median number of tokens2 for proposals with zero annotations is 80, it is 188 for proposals with one or more annotations. Moreover, more than one-third of the proposals (36.6 %) are associated with at least five entities. Also, the number of entities linked to at least one proposal is greater than the number of proposals. However, although the number of entities connected to five or more proposals is significantly lower, almost half (47.8 %) of the annotations link proposals to these entities. We store the generated KG using the JSON-LD format [45]. As JSON-LD is an RDF serialization, querying the graph using SPARQL is possible. Figure 3 depicts a general consultation KG built based on this model.
The model we propose facilitates the issuance of new queries in consultations by both citizens and platform administrators. For instance, one can retrieve all proposals mentioning a specific entity while focusing on a particular topic, such as “all the proposals mentioning the National Commission on Informatics and Liberty (CNIL)” with an LDA-score greater than 0.6 for the “Data Protection” topic. Furthermore, the model allows for summarizing contributions by identifying the topics and entities mentioned in a consultation or linking entities to specific groups of users. By using this method, platform administrators can enhance the transparency of their reports on citizen consultations by adding graph-based data, in addition to responses to a selection of proposals3, either chosen randomly or based on arbitrary criteria. Lastly, our work can be generalized to most online participatory platforms to connect contributions from various online consultations. 2In NLP vocabulary, a token refers to a part of a string. In this study, a token is either a number, a word, or a punctuation mark. 3For instance, see https://www.republique-numerique.fr/project/projet-de-loi-numerique/step/reponses
Online public consultations allow citizens to create proposals and participate in public debates and decision-making. However, these consultations cover a wide range of topics and are presented in text format, making it challenging for both participatory platform administrators and institutions to manage and extract meaningful insights from them. Additionally, citizens may find it dificult to locate relevant information, identify proposals within a specific topic, or gain an overview of the relationship between existing proposals in a given consultation. To address these challenges, our work proposes a methodology for automatically constructing a consultation knowledge graph that can mitigate these issues. We achieve this by utilizing Latent Dirichlet Allocation (LDA) to identify the most relevant topics in a consultation, and Named Entity Recognition (NER) and Named Entity Linking (NEL) using DBpedia Spotlight to link the diferent proposals to resources in DBpedia, resulting in a comprehensive and interconnected representation of the consultation data.
In the future, we plan to enhance the consultation knowledge graph by incorporating
additional data sources, such as the French National Assembly (https://data.assemblee-nationale.fr),
to facilitate the creation of more informed and comprehensive proposals. We also intend to
improve our topic modeling methodology by utilizing automated labeling of topics based on
Wikipedia articles [46], which could aid in identifying common topics across diferent
consultations. Additionally, we aim to evaluate the ability of DBpedia Spotlight to identify the most
important words in proposals. To do so, we plan to perform a quantitative assessment of the
relevance of the annotated words and their associated annotations using independent testers.
[1] T. Bovaird, Beyond engagement and participation: User and community coproduction of
public services, Public Administration Review 67 (2007) 846–860.
[2] A. Lupia, Shortcuts versus encyclopedias: Information and voting behavior in california
insurance reform elections, American Political Science Review 88 (1994) 63–76.
[3] H. Baum, Citizen participation, in: N. J. Smelser, P. B. Baltes (Eds.), International
Encyclopedia of the Social & Behavioral Sciences, Pergamon, Oxford, 2001, pp. 1840 – 1846.
[4] J. Holston, V. Issarny, C. Parra, Engineering software assemblies for participatory
democracy: The participatory budgeting use case, in: 2016 IEEE/ACM 38th International
Conference on Software Engineering Companion (ICSE-C), Institute of Electrical and Electronics
Engineers, Austin, TX, USA, 2016, pp. 573–582.
[5] C. Parra, C. Rohaut, M. Maeckelbergh, V. Issarny, J. Holston, Expanding the design space
of ict for participatory budgeting, in: Proceedings of the 8th International Conference on
Communities and Technologies, C&T ’17, Association for Computing Machinery, New
York, NY, USA, 2017, p. 213–221.
[6] J. K. Ward, F. Cafiero, R. Fretigny, J. Colgrove, V. Seror, France’s citizen consultation on
vaccination and the challenges of participatory democracy in health, Social Science &
Medicine 220 (
Technologie de l’information, culture & société (2018).
[11] V. Pina, L. Torres, S. Royo, Comparing online with ofline citizen engagement for climate
change: Findings from austria, germany and spain, Government Information Quarterly 34
(2017) 26–36. Open Innovation in the Public Sector.
[12] E. Glaas, M. Hjerpe, M. Karlson, T.-S. Neset, Visualization for citizen participation: User
perceptions on a mainstreamed online participatory tool and its usefulness for climate
change planning, Sustainability 12 (
Policy for the Information Society (2009) 10–09. [16] R. Reinanda, E. Meij, M. de Rijke, Knowledge graphs: An information retrieval perspective,
Foundations and Trends® in Information Retrieval 14 (