Digital democracy and direct digital participation in policy making gain unprecedented momentum. This is particularly the case for preferential voting methods and decision-support systems designed to promote fairer, inclusive and legitimate collective decision-making processes for citizens' assemblies, participatory budgeting and elections. So far, a systematic human experimentation with diferent voting methods is cumbersome and costly. This paper introduces VoteLab, an open-source and well-documented platform for modular and adaptive design of voting experiments. It supports a visual and interactive building of reusable campaigns with diferent voting methods, while voters can easily respond to subscribed voting questions on a smartphone. A proof-of-concept with four voting methods and questions on COVID-19 have been used in an online lab experiment to study the consistency of voting outcomes. This demonstrates the Votelab capability to support rigorous experimentation of complex voting scenarios.
1. Introduction
Digital democracy initiatives with direct citizens’ participation in decision and policy-making
gain significant momentum, for instance, citizens’ assemblies and participatory budgeting [
To change this, this paper introduces VoteLab, an open-source platform for modular and
adaptive experimentation with diferent voting methods on smartphones. VoteLab allows
users to visually and interactively design a voting campaign without writing a single line
of code. Designers can even preview the users’ voting experience in diferent smartphones
before deployment. They can easily match voting questions to diferent voter groups, using
assignments of tags/topics via a publish-subscribe system. VoteLab can collect useful
metainformation to understand voting behavior such as voting time duration, time of choice, change
of choices and feedback on voting outcomes. This allows one to conduct studies with
betweenand within-subjects designs, including factorial designs with diferent voting questions, diferent
voting methods and diferent (treatment) groups. As a proof-of-concept, an online experiment
is conducted to study four voting methods [
The main contributions of this paper are (i) a modular and adaptive modeling architecture
for flexible experimentation with diferent voting methods; (ii) an open-source platform of
VoteLab that implements the modeling architecture with a Web dashboard and an Android app;
(iii) a proof-of-concept on COVID-19 to assess the practicality of VoteLab to support rigorous
experimentation of complex voting scenarios; (iv) a software artifact demonstrator running
on a virtual machine for reproducibility, assessment and engagement of the broader research
community; (v) a comprehensive documentation of VoteLab for end-users and developers as well
as a guide for the software artifact demonstrator [
Mobile Voting System (MVS) [
There are several limitations in current digital decision-making approaches. The design of these platforms is complex with limited modularity (i.e., modular architecture). They often require programming skills to obtain high-quality comprehensive data (i.e., simplicity, metadata collection), with diferent implementations ( i.e., native app). There is a lack of flexible platforms for voting experimentation, as existing tools are typically limited to specific voting scenarios (i.e., numbers of voting methods, verification method ). Meta-data about the voting choices such as recording the choice duration and evaluations of the voting results are often necessary to understand voting behavior (i.e. user feedback). Moreover, platforms provide a varying flexibility in customizing and reusing voting questions and settings (i.e. adaptation). Open-source digital voting platforms tend to be complex and inadequately documented (i.e., thoroughly documented). Table 1 provides a comparative summary of some prominent platforms. 3. System Architecture The system architecture of VoteLab is designed to create an adaptive system that facilitates the seamless integration of new voting mechanisms.
Comparison of Voting Methods. VoteLab provides an extensible testbed environment, enabling rigorous experiments to assess diferent voting mechanisms such as majority, approval, score or quadratic voting (currently 7 supported). By combining the collection of voting data and choice meta-data, VoteLab supports researchers and policy makers to study evidence-based decision making and design fairer, more expressive inclusive voting systems. Simplified Voting Experience. In VoteLab, voters can efortlessly create digital voting and data collection processes, using an intuitive visual interface. No coding is required, as the platform empowers users to visually design and implement complex workflows running on smartphones. For example, a community organization can use the platform to design a visually appealing ballot with clear instructions and options for voters.
Tag Assignment System. Voters can automatically access voting questions and campaigns via a tag assignment system, representing categories of interest. This system is implemented as a publish-subscribe mechanism, which efectively determines who can perform which actions. For instance, it allows precise control over which city district can access specific voting questions. In this way, voting designers can create tailored campaigns for specific groups and communities. Multiple Voting Campaigns for Field Tests. The platform empowers researchers to create multiple reusable voting campaigns that involve repeated measurements and group comparisons (between and within subjects experimental designs). Researchers and practitioners can easily design, setup and run voting processes managed via a user-friendly graphical user interface. Behavioral Analysis and Decision-Making Insights. VoteLab supports the recording of initial choices made, their timing, changed decisions as well as the duration of decision-making processes. By collecting such metadata throughout the voting process, new insights can be gained to better understand, efectively design, and improve voting procedures. Customizable and Seamless Feedback System. Votelab supports a built-in feedback system for gathering user opinions, ratings, or responses regarding voting outcomes and experiences. This feedback can assess voters’ satisfaction and the legitimacy of the voting processes. Ubiquitous Online Voting Via VoteLab, voters engage in voting processes using personal devices they are already familiar with, without specialized or dedicated voting hardware. This promotes flexibility and convenience, enabling voters to cast their votes online anytime and from anywhere with Internet access. 4. Key System Components The modular architecture of VoteLab prioritizes the separation of system components and leverages API calls for the communication interface, enabling seamless integration, replacement, or addition of code components. This design approach ensures flexibility, adaptability, and scalability, allowing for the inclusion of new code segments, such as an iOS application, or the integration of additional voting methods without disrupting the existing framework architecture.
VoteLab is implemented using Android Studio, utilizing Java, and Microsoft .NET for the Web application. Figure 1a depicts the architecture, which consists of three interactive parts: (i) an Android application enabling voters to actively participate in elections, (ii) a database server responsible for the storage and management of the collected data, (iii) a dedicated Web dashboard supporting the voting design with an intuitive and user-friendly interface. Database Server and its Components. The central system component is the database server, the hub for all communication and interactions. It plays a key role in handling changes, updates, and aggregations related to votes, i.e., the process of determining the overall voting outcome. The database server comprises three components: (i) a PostgreSQL database, (ii) API handlers for the phone application and Web server, and (iii) a vote processor. API handlers manage and translate external requests into PostgreSQL queries, ensuring efective communication with the database. Votes are processed by the vote processor, which generates the corresponding voting results. The processor regularly monitors the server for the closing date of voting, automatically calculates results, and stores them in the database upon completion of the voting period. If there is a need for calculating voting results other than the specified closing date, the API handlers request the calculation of results by establishing an open connection to the processor. Voting Management Dashboard. The Web server component encompasses the data and API calls within an intuitive Web interface, which can be used to design and deploy voting campaigns, as well as assign tags to voting questions, see Figure 2(a). Tag assignment involves associating specific tags or categories with voting experiments, allowing for easy categorization and organization of voting campaigns based on diferent criteria or themes. This enables eficient ifltering and analysis of voting results using the assigned tags.
(a) (b) (c) Voter Interface. The VoteLab Android application provides an intuitive interface for voters to actively participate in voting and experiments, see Figure 2, panels (b) and (c). Using the smartphone app, voters can access information about ongoing voting campaigns and experiments, view relevant details, and submit their responses. The platform uses the tag assignment system to match voters with voting questions. For example, if a voting campaign is related to a specific demographic or geographical region, voters assigned to the corresponding tags receive notifications and updates relevant to their specific group. This ensures that voters can receive tailored information and get opportunities to engage in voting processes that are more relevant to them. It also allows designers to easily create special group treatments to study voting behavior, presumably within the scope of proper ethics approvals. VoteLab Workflow . Figure 1b illustrates the lifecycle of a voting experiment, starting from the ballot design to the campaign deployment and the calculation of the voting results. (1) The process begins with the ballot designers creating an account on the Web dashboard and (2) subsequently logging in, to create and run voting campaigns. (3) Via an intuitive interface, designers can efortlessly create and customize voting experiments. (4) Voters, on the other hand, can easily access the created voting campaign, express their preferences, and (5) view the results, once the voting period concludes. Voters can provide valuable feedback on voting results. The dashboard allows users to assign tags and reuse voting processes, enabling the deployment of multiple voting campaigns at diferent time points with diferent or the same voting participants. This feature simplifies experimentation with several waves of a panel study.
Voters have the option to (i) download the Android app and (ii) log in, using valid credentials.
(iii) Once logged in, they can participate in voting by selecting the relevant voting tags they
wish to contribute. (iv) As such voters are granted access to the corresponding voting questions.
The tagging system allows voters to easily engage with projects that align with their interests.
The privacy of votes and voters’ anonymity is preserved in this process.
5. COVID-19 Online Experiments
To assess the modularity and adaptation capabilities of VoteLab, a proof-of-concept lab study
is illustrated, based on an online experiment with human subjects. The experimental design
involves a level complexity that is hard to manage with existing platforms: (i) four voting
methods, (ii) four voting campaigns, (iii) two experimental conditions, (iv) a within-subjects
design with repeated measurements, and (v) the collection of meta-data. The study is performed
in 2021 and is related to COVID-19. It aims to understand how diferent preference elicitation
methods influence voting outcomes in a polarized voting context. The in-depth analysis of the
collected data is not subject of this paper and performed in earlier work [
The questions to be voted upon were the following: (1) What are you most concerned about the COVID-19 vaccines? [vaccine] (1) How to be vaccinated as soon as possible. (2) Their long-term side-efects. ( 3) Their overall efectiveness. ( 4) Their misuse by governments & companies. (5) Discrimination, e.g. travels, access to facilities & services. (2) Among COVID-19 patients, which criterion should grant one access to an intensive care unit? [icu] (1) Being the youngest. (2) Being the oldest. (3) No denial of vaccination. (4) No violation of lockdown rules. (5) No health self-damage, e.g. smoking, drugs, alcohol. (3) Which is the most efective protection measure against a COVID-19 infection? [protection] (1) Wearing a mask. (2) Physical distancing. (3) Vaccination. (4) Regular hand washing. (5) Maintaining a healthy lifestyle. (4) Which is the most significant problem that the lockdown has caused? [lockdown] (1) Economic recession & unemployment. (2) Government control & suppression of freedom. (3) Social segregation & increased inequality. (4) Mental distress. (5) Reduced physical health condition.
Each participant answered each question with four diferent voting/input methods, which vary in terms of the degree of freedom and expressiveness: (i) majority voting (= {0, 1}), (ii) combined approval voting (= {0, 0.5, 1}), (iii) score voting (= {0, 0.2, 0.4, 0.6, 0.8, 1}), and (iv) modified Borda count (= {0, 0.2, 0.4, 0.6, 0.8, 1}, if all options are selected, otherwise adjusted accordingly). Each input method scores the options in a diferent way. The scores refer to the numerical values assigned to a choice and represent the degree of preference. Majority voting is the least flexible method. Via combined approval voting, participants express disapproval or support. Score voting allows even more fine-grained expression of preferences, as participants assign a score to each option. The modified Borda count encourages participants to make compromises: the more options selected, the higher the assigned score to each option.
Table 2 illustrates the aggregate scores of each option for each question and voting method. Figure 1c illustrates the consistency of voting outcomes for each of the 1st, 2nd, ..., 5th ranked option, derived from Table 2. For instance, consistency of 1.0 for the 1st ranked option means that all voting methods determine the same option as ranked 1st. A consistency of 0.5 for the 2nd ranked option means 2 out of 4 voting methods determine the same option as 2nd ranked. The results reveal the following: (i) Voting methods seem to show higher consistency with disagreements rather than agreements. (ii) For the 1st ranked option, the highest consistency is observed for the protection question. (iii) The vaccine question has the lowest mean consistency among all five ranked options. The highest mean consistency is found for the protection question.
Inconsistency may reflect potential voters’ disagreements with voting outcomes, i.e. results
likely to be disputed. For highly polarizing questions, voting methods are expected less
consistent. Using a more sophisticated voting method allows voters to express their preferences in a
more diferentiated way [
However, VoteLab also has some limitations and ofers opportunities for future work. The
development of an iOS app and the option to vote directly from a browser is currently ongoing
work. The same applies to experiments with diferent aggregation methods and applications to
participatory budgeting. Other interesting study scenarios include policy-making for Smart
Cities and sustainability applications, for instance, crowd-sensing of air/water quality [