A Gamified Crowdsourcing Framework for Data-Driven Co-Creation of Policy Making and Social Foresight Andrea Tocchetti, Marco Brambilla Politecnico of Milano, Department of Electronics, Information and Bioengineering Via Ponzio 34/5, 20133 Milano Italy andrea.tocchetti@polimi.it, marco.brambilla@polimi.it Abstract Over the last decades, communication between governments and citizens has become a remarkable problem. Governments’ decisions do not always match the visions of the citizens about the future. Achieving such alignment requires cooperation between communities and public institutions. Therefore, it’s essen- tial to innovate governance and policymaking, developing new ways to harness the potential of public engagement and participatory foresight in complex gover- nance decisions. This paper proposes a comprehensive framework that combines crowdsourcing and data analysis to improve the crowd’s collective engagement and contribution in policy-making decisions. Our approach brings together social networking, gamification, and data analysis practices to extract relevant and co- ordinated future visions concerning public policies. The framework is validated through two experiments with citizens and policy-making domain experts. The findings confirm the effectiveness of the framework principles and provide useful feedback for future development. 1 Introduction 1.1 Context and Problem Statement Lately, the communication between governments and citizens has worn thin, leading to a loss of faith and interest in politics. Most of the time, people don’t feel adequately represented by their governments and feel like their actions have no impact on society, leading to reduced interest in politics. People often feel neglected by public powers [1], thus giving up on exercising their participatory rights. Furthermore, the lack of innovation in governance generates a decline in trust and engagement from citizens. The affluence percentages of the most recent European elections held on 26th May 2019 prove such a phenomenon. Indeed, on average, only the 50.6% of the European population expressed their votes. Political alienation is a typical trait of contemporary societies [2], primarily associated with young people. Thus, this problem should be tackled through solutions appealing to younger generations of future voters (and policymakers). The way people take part in political debates has changed over the last years. Although voting is still perceived as the primary approach to participate in public debates, modern forms of participation, like NeurIPS 2020 Crowd Science Workshop: Remoteness, Fairness, and Mechanisms as Challenges of Data Supply by Humans for Automation. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). discussions in online communities and social networks, are spreading quickly among the youngest generations. These trends make it evident that a new way to increase engagement, participation, and contribution in public debates and shorten the distance between decision-makers and citizens is necessary. 1.2 Objective and Contribution This work proposes a framework that combines crowdsourcing and data analysis to improve the crowd’s collective engagement and contribution in policy-making decisions. More precisely, the objectives we aim to cover are: 1. Collect ideas and thoughts from communities through crowdsourcing to brainstorm and co-create solutions to modern and future societal problems; 2. Enhance governments’ foresight capabilities to achieve a meeting point between the govern- ment’s actions and the people’s shared vision of the future; 3. Improve the communication between (local) governments and (local) communities, con- tributing to recover trust in public institutions. Our innovative contribution consists of devising an approach that moves the focus and objective of crowdsourcing and co-design practices from traditional, short-term goals to long-term visionary changes at the societal, economic, and political level. This new focus opens completely new challenges in terms of engagement and participation of workers, as well as in terms of their contribution’s shape and content. These challenges are addressed through a set of principles, illustrated in the following section, to achieve long-term citizen engagement. This work describes the conceptual framework and the implemented digital platform. The proposed solution has been evaluated through in-person and web-based experiments engaging citizens and policy-making experts. 1.3 Paper Organization This paper is organized as follows: Chapter 2 summarizes the essential background on gamification and reports works addressing problems covering engagement and crowdsourcing similar to ours; Chapter 3 presents the principles and the design of the framework; Chapter 4 briefly reports the tools and technologies utilized to develop the prototype; Chapter 5 reports the experiments in details; Chapter 6 discusses the results and the final considerations; finally, Chapter 7 reports some final reflections and the works currently in progress. 2 Related Work 2.1 Gamification & Intrinsic and Extrinsic Motivation In recent years, there has been an increasing interest in motivating people, enhancing engagement, and triggering behavioural changes. One of the most relevant developments in this area is gamification, whose goal is to promote intrinsic motivation towards different activities by exploiting game design techniques. The “Self-Determination Theory” [3] explains that everybody has three innate psychological needs: Competence, Autonomy, and Relatedness. Achieving these needs enhances motivation, mental health, and individual growth. These concepts are strictly related to motivation, which can be further divided into intrinsic and extrinsic motivation. Cerasoli et al. [4] described that both kinds of motivation produce a gain in performance. Still, only the intrinsic one improves the learning outcomes, creativity, self-esteem, and general well-being. Amon Rapp at al. [5] stated that gamification should not rely solely on extrinsic motivators to create long-lasting user engagement. Social aspects can influence the attitude towards a gamification service: enjoyment aspects (intrinsic motivators) drive the actual use, while the utilitarian and social ones (extrinsic motivators) affect attitude, and through that, affect intentions [6]. Hamari et al. [7] demonstrated that a community of people committed to the same goals has a relevant impact on users. Getting exposed to other users’ attitudes and feedback can positively influence the attitude towards using a gamification service. They also discovered that social interaction enhances the social influence and the perception of reciprocal benefits. 2.2 Crowdsourcing & Engagement A good engagement level is essential to achieve a continuous flow of data. Achieving such an objective is one of the reason why the most popular crowdsourcing sources are well-known social platforms. In [13], the authors developed a framework enabling crowdsourcing through social media, collecting people’s thoughts and reactions to the policy-related content they shared. The principles on which their framework was based are the simultaneous utilization of multiple social media, the automated management and continuous monitoring of the content, and the processing of the collected data. In the end, they accomplished a time and cost-efficient technique in an open web environment, improving the level of participation of the citizens in policy debates. In [14] geo-crowdsourcing, social media have been employed to collect data related to local events (crimes, accidents, etc.) to create policies relevant to the citizens’ needs and improve officers’ intervention. Citizens and officers submit an event through the dedicated interface. The submitted content is analyzed, and predictions are made to foretell an event in the city’s considered area. Furthermore, tweets are utilized to analyze topics relevant to citizens’ opinions due to their unstructured nature. Some other works consider improving the communication between the population and governments as their core objective while acknowledging the engagement aspect as one of the main factors to achieve it. In [8–10] Sarah-Kristin Thiel compared the effect of social and reward-based gamification in public participation to understand which types of motivation encourage engagement and whether gamifica- tion could provide incentives to engage. The experiments were carried out on three different versions of the same application. The outcomes revealed that social gamification seems to motivate citizens more towards participation than reward-based gamification. Indeed, they proved that a reward-based system reduces intrinsic motivation. They also asserted that while extrinsic motivation certainly played a role, the intrinsic one is more relevant for encouraging citizens to become active in urban decision-making and public topics. Although gamification was proven to be overall effective, they detected that sometimes gamification does not necessarily increase participation. Bianchini et al. [11] promoted a two-way exchange of proposals between politicians and citizens, who may discuss in a structured and goal-oriented way while satisfying three levels of participation: provision of information, engagement, and empowerment. The application allowed citizens to create and vote for proposals, while the politicians should answer those proposals in a given number of days. The gamification aspects generated an increase in the number of solutions, details, and the total number of activities carried out, leading the users to contribute to building a constructive debate about the problems and initiatives shared within the community. They proved that gamification increased both participation and the quality of the provided content. Sometimes, to encourage engagement and the sharing of ideas funds were instantiated to realize either the most voted proposal or the one considered the best from the public administration, providing an evident form of feedback for the citizens. In Ovar, Portugal [12] a method called “Participatory Budget” allowed citizens to express their support for different budget proposals. The experiment carried out a successful result: about 25% of the city residents voted on proposals for the city budget, allowing the local government to save time and money. 3 Framework Conceptual Definition Our objective is to conceptualize a crowdsourcing framework promoting social and political engage- ment, employing gamification principles to develop intrinsic motivation. The framework covers two primary roles, namely: policy-making experts and citizens, with a particular focus on the latter. The main principles on which the platform is built are: 1. (Gradual) Awareness - Improving the quality of the data requires to increase citizens’ awareness of the proposed topics. While exploring the platform, each member develops his/her awareness of the topics gradually, initially learning and playing alone, then progres- sively increasing the size of their interactions. 2. Crowdsourcing - The framework is built with the primary objective of collecting data from communities of citizens to analyze their thoughts. Data from the platform is mainly collected in the form of text and pictures. 3. Engagement through Gamification - The platform leverages gamification to improve the user experience, making it as enjoyable as possible to achieve a high level of engagement. Even though most of the reported works leverage extrinsic motivations, our objective is a long-lasting engagement achieved through intrinsic motivation. The following subsections detail how practical design choices have covered these principles. 3.1 Awareness Different awareness tools have been devised to bring the participants to the campaign closer to the topic of interest. These tools include small games, informative feeds, and the organization of interactive events. Quiz. Its purpose is to improve the level of awareness and knowledge of the members about a topic. Each question is made of four answers and, eventually, an image. When the question is answered, some information about the question’s topic is provided to the user. News Feed. Articles and news are provided to increase the awareness of citizens about a chosen topic. Providing external content may cause bias, depending on the reports published by the selected sources. Therefore, the latter should be as heterogeneous and reliable as possible to reduce this biasing effect. Event. An event is a public meeting organized by citizens to create an even stronger community. Events should be avoided whenever the topic is a sensitive one, in which case it may generate conflicts among people. Therefore, an administrator should approve events before the organizer would be allowed to invite other members. 3.2 Crowdsourcing Our framework features three crowdsourcing tools to collect contributions from participants, which are meant to create high engagement and at the same time to collect relevant content, supporting co-creation on different levels. Shared View. This element allows citizens to share their point of view within the community. Each shared view is made of an image, at least a keyword, and the assigned subtopic. Citizens express their thoughts on the views shared by the community through three attributes, namely Relevance (whether the shared view is considered inappropriate or relevant for the topic), Need (whether the aspect illustrated by the shared view should be addressed quickly or not), and Radiosity (Whether the shared view is associated to either a "bright" or "gloomy" future) and by assigning a preference to keywords. Change My Mind. Reading opinions and exploring new points of view coming from other citizens may result in self-improvement. “Change My Mind” is a custom game that involves two citizens sharing their opinions on a shared view. One of them is the owner of the shared view; the other is a community member who rated it. The latter is asked to explain what they agree or disagree about the view and whether they would keep or change the rating they gave, in which case they are also asked to provide a new one. The results are shared with the owner of the view, who can then answer in turn. Discussion Groups. A discussion group consists of citizens interested in a set of topics they want to discuss. Different opinions and points of view can contribute to building constructive and exciting discussions. One of the aspects that characterize a discussion group is a weekly poll that encourages group members to propose and vote on topics they would like to discuss. 3.3 Tools’ Positioning Figure 1 represents the positioning of each of the tools with respect to the dimensions of Engagement, Crowdsourcing and Awareness. The evaluations were made considering the context in which the application will be applied, namely the policy-making one. Figure 1: Elements of the framework w.r.t. level of crowdsourcing, awareness and engagement dimensions 4 Implementation To validate our approach, we developed a web-based prototype. Alongside the elements we designed, the most classical ones have been included, namely a chat and a profile. The latter consists of a set of user-selected shared views and achievements displayed to other users visiting the profile, granting the owner of the profile status within the community. Some of the displayed user data can be hidden by configuring privacy settings to address the privacy aspect. The collected data is stored in a relational database and analyzed through a dedicated data analysis pipeline. Aggregated results are provided to policymakers through dedicated online dashboards featuring visualization and exploration capabilities. Figure 2 represents the platform’s section in which the citizens share their views. Figure 2: Screenshot of the Feed section of the prototype in which the citizens shares their views. The prototype’s implementation employed the MVC architectural pattern (Model-View-Controller), Thymeleaf, Bootstrap, and MySQL alongside different libraries: JQuery, Spring Boot, Spring Data JPA, Spring Security, Lombok, MySQL Connector, and HikariCP. 5 Experiments The experiments with citizens and policymaking experts aimed to understand the proposed approach’s weakness and collect feedback to further improve the engagement and crowdsourcing potential. 5.1 Physical Mockup Experiment with Policy Making Experts The physical test held in Fiesole (European University Institute, Villa Schifanoia) involved 15 experts in the socio-political field. Its main objective was to collect feedback on how the platform’s principles would work in a cooperative and interactive environment. The chosen scenario (or topic) for the experiment was “The relationship between humans and Artificial Intelligence in the future”. The process involved five steps (summed up in Figure 3). Figure 3: An overview of the experimental setting, summarizing the interactions expected in the different steps of the experiment. Challenge Awareness. Each participant was given an envelope containing all the sheets useful to understand the chosen scenario. Individual Activity - Case Selection. The second step involved two different sheets for each participant and a common billboard. The first sheet contained four different artificial intelligence application cases, each one printed on two tags. Participants had to choose only one of the proposed cases, write their name on the corresponding tag and stick it on a billboard, on which a graph was represented. The latter involved two different axes. The x-axis ranged between “incremental” and “disruptive”, while the y-axis ranged between “fear” and “hope”. After sticking the tag on the billboard, each participant was asked to attach the corresponding identical tag on the second sheet and write the motivations for the positioning of the label on the billboard. Pair Activity - Discussion. The third step coupled participants with diverging opinions to make them discuss. They exchanged the sheets compiled during the previous step, sharing their thoughts on the topic. After the discussion, each participant compiled another sheet, writing whether they agreed or disagreed about the other participant’s thoughts. Pair Activity - Convergence. After the discussion, a picture-based game, aimed to have each couple converge on a single common vision, was performed (first picture of Figure 4). Each couple was asked to pick one picture representing their vision about the topic, paste it on the provided sheet, and write three meaningful keywords. After all the visions had been generated, they were shared with all the couples who also voted the team they would like to be grouped with for the next activity. Group Activity - Convergence and Sharing. In the last step, new groups were created using the previous voting phase. Each of them was assigned a sheet on which they stuck each vision built in the previous stage. Then, each group discussed and converged to a single vision among the ones they had, for which they were also asked to provide a title, their thoughts, and three new keywords. Afterward, each group was asked to share their vision and thoughts within the “community” (second picture of Figure 4). 5.2 Online Engagement Validation with Citizens The second experiment aimed at understanding whether the platform’s structure and the design of the gamified activities were effective. It engaged about 20 people, aged between 20 and 25, who were Figure 4: Pictures taken during the testing workshop with domain experts, held in Fiesole (FI), Italy asked to perform a set of tasks on the platform’s digital prototype. Each tester carried out the activities without any hint, provided if they could not act after a reasonable amount of time. Before exploring each section, a short description regarding its content, objectives, and functionalities was provided. Each test was performed independently from the others, preventing participants from exchanging opinions or suggestions about the platform’s tasks. Each participant was asked to provide feedback while exploring the platform and compile a questionnaire when the experiment was over. The survey was made of 15 questions, including both qualitative and quantitative ones. The tester was given the feel of being enrolled in a fully functional platform. Two different topics were proposed: “Renewable Energies VS Fossil Fuels” and “Environmental Pollution”. The tester could choose between them without influencing the test. 6 Discussion and Evaluation We derived most of the design principles on which the platform is based on the experiments and the users’ feedback. Results and feedback show that the chosen approaches are effective and achieved a good user engagement level within citizens’ and experts’ communities. A Human-centered approach was applied to collect the experiments’ outcomes, engaging end-users and experts directly and asking them to provide feedback at the end of their experience. The results were collected through an online questionnaire, asking for judgments, comments, and ratings on the platforms’ different elements. The feedback was expressed through qualitative and quantitative questions, as reported in Appendix 1. The first experiment confirmed the usefulness of the “Gradual Awareness” principle in shaping the participants’ thoughts. They stated the experience was appropriately engaging and enjoyable, and the flow of interactions contributed to shaping their ideas, leading to the final vision. During the experiment, an unexpected but exciting behavior stood out: while choosing the most representative image in the “Pair Activity - Convergence" step, some couples chose multiple pictures instead of one, cutting and merging them into a single one. They asserted their vision was too complex to be represented with a single image, and none of those they could choose from was perfectly matching their vision. On the other hand, a few aspects of the experiment were criticized. The dimensions used to classify the scenarios on the graph was the most disapproved and discussed aspect. In particular, “fear” and “hope” were perceived as too extreme and complex to represent their feelings. The second experiment’s feedback pointed out that some of the elements implemented are well designed and useful while it is necessary to rework others. The quiz was perceived as the most intuitive and appreciated activity. In particular, the participants pointed out that the additional information provided was fascinating and improved their overall understanding of the topic. After completing the quiz, few people asked “Where are my points?”, “Isn’t there any recap of the answers I gave? Like the percentage of the correct answers” and “I would like to see a leaderboard when the quiz is over, ranking myself with my friends”. These comments revealed that sometimes a quiz is perceived as a game involving points and leaderboards, leading people to compete. “Discussion groups” was appreciated, although it was identified as the least interesting, probably because most of the features provided are commonly exploited in forums and discussion websites. The most appreciated feature was the possibility of proposing and voting discussion topics. “Change My Mind” was identified as the most controversial activity. Indeed, most of the participants appreciated the design, while others were confused, revealing that there’s still room for improvement. Some participants provided some feedback on how to do so. In particular, they stated it would be better to visualize a list of all the rated views, alongside a preview of its details, having members choose one of them to play with instead of a random one. A participant also suggested introducing a link between the game and the shared view feed, allowing to play a match as soon as the vision has been rated. During the test, the participants were asked to interpret each icon representing the rating attributes (Relevance, Need, and Radiosity) in “Shared View”. The goal was to understand how much each couple of icons was self-explanatory for the associated concept. The most understood icons were the ones related to the “Relevance” attribute, while the others were not considered clear at all. In conclusion, some testers suggested modifying how the attribute ratings are expressed by replacing the sliders with stars. Overall, during the whole testing phase, different additional features were proposed by the participants. They were not informed about the entire design around which the prototype was built. Surprisingly, some of the features excluded from the implementation were suggested: “I would like a section where I can look for events and meetings about a topic” and “I would like a section in which I can read the most recent news about a topic”. Those sentences perfectly exemplify two of the designed elements excluded from the implementation, namely “Event” and “News Feed”. 7 Conclusions In this work, we devised a crowdsourcing framework through which community members can express their opinions and thoughts, sharing their future vision. Ours is a contribution towards enabling crowdsourcing and co-design campaigns to move from traditional, short-term objectives to long- term visionary changes at societal, economic, and political levels, thus opening new challenges in engagement, participation, and design of the crowdsourced content. A gamified approach to developing intrinsic motivation and focus on content creation and community-based elements have been employed. Two experiments were carried out. The first one involved a community of policymaking experts engaged through physical activities, while the second one engaged a group of citizens within a digital prototype of the platform. The experiments confirmed the effectiveness of the applied principles and contributed to improving the most contested features. An improved version of the platform is currently under active development while being tested with small selected communities. The new version includes a redesigned version of the proposed gamified sections. It also supports the creation of shared visions by combining different pictures and an organized and clarified version of the “Change My Mind” activity. Moreover, new co-creation activities have been designed and implemented, improving the content-sharing capabilities of the citizens. Acknowledgements This work is partially funded by the European Commission under the H2020 framework, within the project 822735 TRIGGER (TRends in Global Governance and Europe’s Role). We would like to thank our colleagues Lorenzo Corti, Diletta Di Marco, Roberto Verganti and Stefano Brusadelli for their contributions to the TRIGGER project. References [1] Hamari, J.: Perspectives from behavioral economics to analyzing game design patterns: loss aversion in social games. (2011). [2] Mahnič, N.: Gamification of politics:Start a new game!. Teorija in Praksa. 51. 143-161. (2014). [3] Ryan, R., Deci, E.: Self-Determination Theory and the Facilitation of Intrinsic Motiva- tion, Social Development, and Well-Being. The American psychologist. 55. 68–78. (2000). https://doi.org/10.1037/0003-066X.55.1.68. [4] Cerasoli, C., Nicklin, J., Ford, M.: Intrinsic Motivation and Extrinsic Incentives Jointly Predict Performance: A 40-Year Meta-Analysis. Psychological bulletin. 140. (2014). https://doi.org/10.1037/a0035661. [5] Rapp, A.: A Qualitative Investigation of Gamification: Motivational Factors in Online Gamified Services and Applications. International Journal of Technology and Human Interaction. 11. (2015). https://doi.org/10.4018/ijthi.2015010105. [6] Hamari, J., Koivisto, J.: Why do people use gamification services?. International Journal of Information Management. 35. 419–431. (2015). https://doi.org/10.1016/j.ijinfomgt.2015.04.006. [7] Hamari, J., Koivisto, J.: Social motivations to use gamification: An empirical study of gamifying exercise. ECIS. (2013). [8] Thiel, S.: Reward-based vs. Social Gamification: Exploring Effectiveness of Gamefulness in Public Participation. 1-6. (2016). https://doi.org/10.1145/2971485.2996739. [9] Thiel, S., Fröhlich, P.: Gamification as Motivation to Engage in Location-Based Public Partici- pation?. (2017). https://doi.org/10.1007/978-3-319-47289-8_20. [10] Thiel, S., Ertiö, T.P., Baldauf, M.: Why so serious? The role of gamification on motivation and engagement in e-participation. Interaction Design and Architecture(s). 158-181. (2017). [11] Bianchini, D., Fogli, D.,Ragazzi, D.: Promoting Citizen Participation through Gamification. 1-4. (2016). https://doi.org/10.1145/2971485.2971543. [12] A Political Success Story: Gamification as Civic En- gagement Tool. http://www.gamification.co/2016/01/11/ political-success-story-gamification-civic-engagement-tool. Last accessed June 2019 [13] Spiliotopoulou, L., Charalabidis, Y., N. Loukis, E. and Diamantopoulou, V. (2014), "A frame- work for advanced social media exploitation in government for crowdsourcing", Transforming Government: People, Process and Policy, Vol. 8 No. 4, pp. 545-568. https://doi.org/10.1108/TG- 01-2014-0002 [14] Witanto, Joseph & Lim, Hyotaek & Atiquzzaman, Mohammed. (2018). Smart government framework with geo-crowdsourcing and social media analysis. Future Generation Computer Systems. 89. 10.1016/j.future.2018.06.019. A Questionnaire The questionnaire of 15 questions to which the participant had to answer after the test is reported below. An Italian version was made available too. Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Age . . . . . . . . . Gender . . . . . . . . . 1) Sort the features provided by the platform based on how much interesting they are in your opinion (e.g. Quiz, Shared View, Change My Mind, Discussion Group). ............................................................................................. 2) Sort the features provided by the platform based on how much intuitive they are in your opinion (e.g. Quiz, Shared View, Change My Mind, Discussion Group). ............................................................................................. 3) How would you make those functionalities even more intuitive for new users? ............................................................................................. 4) Which of the features implemented in the platform would you use the most if you would become one of its users? ............................................................................................. 5) Based on the sections implemented in the platform, assign an evaluation on a scale of 1 to 10 for each one of them, basing your rating only on their functionalities. Quiz ............................................................................................. Shared Vision ............................................................................................. Change My Mind ............................................................................................. Discussion Groups ............................................................................................. 6) In “Shared Vision” some icons are used to represent different concepts for each different evaluation. The first one is the “Relevance”, which is represented using a target without any arrows (low Relevance) and a target with an arrow in its center (high Relevance). How clear is this representation on a scale of 1 to 10 in your opinion? ............................................................................................. Propose some icons that could replace the current ones. ............................................................................................. 7) In “Shared Vision” some icons are used to represent different concepts for each different evaluation. The second one is the “Need”, which is represented using two different thermometers, one recalling cold (Secondary) and one recalling hot (Impelling). How clear is this representation on a scale of 1 to 10 in your opinion? ............................................................................................. Propose some icons that could replace the current ones. ............................................................................................. 8) In “Shared Vision” some icons are used to represent different concepts for each different evaluation. The third one is the “Radiosity”, which is represented using a thunderstorm (Gloomy) and a rainbow (Bright). How clear is this representation on a scale of 1 to 10 in your opinion? ............................................................................................. Propose some icons that could replace the current ones. ............................................................................................. 9) In “Shared Vision” there is a functionality which allows each user to suggest a Tag to another user for a specific image, how much is this function relevant on a scale of 1 to 10 in your opinion? ............................................................................................. 10) In “Shared Vision” there is a functionality which allows each user to express their preferences about one or more Tags assigned to an image, how much is this function relevant on a scale of 1 to 10 in your opinion? ............................................................................................. 11) In “Shared Vision” there will be a functionality which allows each user to import and export photos from other Social Medias (e.g. Instagram). Would you use this functionality? (Yes/No, Why?) ............................................................................................. 12) Inside each “Discussion Group” each user can express their own preference on each week about what they would like to discuss about, how much is this function relevant on a scale of 1 to 10 in your opinion? ............................................................................................. 13) Inside the “Profile” it is possible to set which information would be shown to other users, how much is this function relevant on a scale of 1 to 10 in your opinion? ............................................................................................. 14) Which aspects of the platform would you improve? ............................................................................................. 15) Which features would you add to the platform? .............................................................................................