What is Polis?

Ask for opinions on an online message board and the loudest voices tend to dominate, turning the thread into a shouting match. Replies breed more replies, and in the end nobody can tell which views are held by how many people. Polis was built to do the opposite. Instead of letting people argue, it shows how their views divide and where they overlap.

In short

Polis (pol.is) is a web tool in which many people write short statements on a single question, and click "agree", "disagree" or "pass" on each other's statements [1][2].

From the pattern of votes, it shows clusters of people who think alike as a kind of map [1]. It then highlights the statements that are agreed on by every cluster [1][2].

For example

Regulating Uber in Taiwan (2015). vTaiwan, a platform for public discussion in Taiwan, opened a Polis conversation on whether to allow the ride-hailing service Uber [3]. "Close to 2000 people participated and 100 people submitted nearly 200 comments" [3].

Participants split clearly into two groups: pro-Uber and anti-Uber (pro-taxi) [3]. Even so, one statement won the agreement of nearly everyone, including taxi drivers: "I feel like the government should be able to face the challenges posed by Uber while improving Taxi's evaluation and supervision, so that taxi drivers and passengers are able to enjoy the same quality of service as Uber" [3].

A constitution for AI, written by the public (2023). The nonprofit Collective Intelligence Project (CIP) and the AI company Anthropic used Polis to ask about 1,000 people, selected to match the makeup of the US population, how an AI chatbot should behave [4]. 1,002 people wrote 1,127 statements and cast 38,252 votes [4]. The statements supported across groups were compiled into a "constitution" (a set of principles for the AI's behavior) and used to train a language model [4]. The project is called Collective Constitutional AI.

An "opinion map" for Japan's 2024 general election. On November 18, 2024, the Japanese election information site JAPAN CHOICE published an "opinion map" built with Polis as one of its components [5]. In about two weeks, 4,403 people voted [5]. It placed icons for each political party on the map of citizens' opinions, showing each party's position on statements taken from the party manifestos [5]. Hirokazu Nishio, who was involved in building it, described it as the largest application of Polis in Japan [5].

How it works

What participants do

The organizer posts a question (for example, "What do you think about generative AI?") and a few starter statements as examples [1][3].

Participants are shown other people's short statements one at a time (under 140 characters in English) and click "agree", "disagree" or "pass" [1][3]. If what they want to say hasn't come up yet, they can write a new statement. New statements are handed out in turn to other participants [1][3].

There is deliberately no way to reply to someone else's statement. The Japanese-language Wikipedia explains that this design is meant to curb trolling and personal attacks [6].

On the results screen, you can see a chart in which people with similar views sit close together, along with those clusters (opinion groups) [3]. It also shows the statements that characterize each group, and the statements that most people agree with overall [3].

The calculation behind it

According to the foundational paper on Polis (Small and others, 2021), the steps are roughly as follows [3].

1. Build a table of participants and statements: agree is +1, disagree is −1, pass is 0.
2. Use a method called "principal component analysis" to compress this big table into a two-dimensional "opinion map".
3. Use a method called "k-means" to sort participants into 2 to 5 opinion groups.
4. For each group, pick the statements it is more likely to agree with than other groups do, as that group's representative statements.
5. Rank the statements that every group agrees with by multiplying together the probability that members of each group agree with a statement (estimated from the share of agree votes within the group). The paper calls this a "group-aware consensus" metric.

Step 5 is the key. Here is an illustrative example (the numbers are made up; the real calculation adds a small correction so that values don't become extreme when there are few votes). A statement agreed to by 90% of group A and 80% of group B scores 0.9 × 0.8 = 0.72. A statement agreed to by 100% of group A but only 10% of group B scores 1.0 × 0.1 = 0.1. Because the values are multiplied, being disliked by even one group pulls the score down. The paper says this helps "protect from tyranny of the majority and respect minority dissent" [3].

According to the official explanation, the calculation uses only the table of votes and does not read the content of the statements [7]. That is why Polis can be used in any language [7].

Origins and spread

According to the English-language Wikipedia, Polis was created by Colin Megill, Christopher Small and Michael Bjorkegren after the Occupy Wall Street and Arab Spring movements, and released in 2012 [8]. The software is open source and is now run by a nonprofit, The Computational Democracy Project [8].

The official collection of case studies lists vTaiwan (over 200,000 people), Austria's citizens' assembly on climate (2022, over 5,000 people), a national referendum in Uruguay (2020–21), and conversations with young people run by the United Nations Development Programme (UNDP, over 30,000 people), among others [9].

Combining it with AI

Polis builds a map of votes, but does not read the content of freely written text [7]. So tools appeared that use large language models (LLMs) to interpret Polis results. In a 2023 discussion in Taiwan, opinions gathered with Polis were visualized with a tool called "Talk to the City" [10]. The Japanese book Broad Listening stresses a difference: each dot on a Polis map represents one person, while each dot in Talk to the City represents one opinion [10].

What is debated

• It has clear limits. The developers say Polis can be used in place of, or alongside, traditional surveys, but they also list its limits [3]. It doesn't work well with small numbers of participants or statements ("a lower bound of dozens of people and tens of comments"). It is unsuited to yes/no questions or leading questions. It is weak at ranking options. And there is a bias from timing: statements from early arrivals get seen more, while late arrivals get to vote on more statements [3].
• It takes work to run. CIP reports that it had to rephrase or remove statements that were meaningless, discriminatory or off-topic, following a set policy [11]. In the Collective Constitutional AI project, the tool was modified so that people could not write statements until they had cast at least 30 votes, to reduce duplicate and meaningless posts [4]. The English-language Wikipedia also notes that, as of 2023, Polis "still required significant human labor and analysis" [8].
• Participants are not a cross-section of society. The official report on Taiwan's 2023 discussion stated clearly that most participants were self-selected volunteers or invited experts and did not represent society as a whole [10]. So in 2024 the process moved on to a meeting of citizens chosen by lottery [10].
• vTaiwan's decline. The book Plurality itself lists the problems: COVID-19 halted face-to-face meetings and participation dropped, the volunteer workload was heavy, and the government had no obligation to respond [2].

Where it fits in Plurality

In chapter 2-2, Plurality introduces Polis as the tool behind vTaiwan [2]. The book explains that "Polis functions similarly to conventional microblogging services like X (formerly Twitter)", except that "instead of displaying content that maximizes engagement, Polis shows the clusters of opinion that exist and highlights statements that bridge them" [2]. "This approach facilitates both consensus formation and a better understanding of the lines of division" [2].

Chapter 5-4, "Augmented Deliberation", treats Polis as a prime example of "bridging systems" — systems that look for opinions supported across divides [12]. Pairing Polis with Community Notes, it names "how systems like Polis and Community Notes could be extended with modern graph theory and GFMs" (generative foundation models, the technology behind today's AI chatbots) as "one of the most obvious directions" [12].

Related articles: - vTaiwan — the Taiwanese discussion platform where Polis was used - Bridging — the idea of valuing support that crosses divides - Community Notes — the same idea applied to notes on social media - Augmented Deliberation — the chapter of the book that discusses Polis - Talk to the City and Alignment Assemblies — developments that combine it with AI

Further reading

1. Plurality, chapter 2-2 "The Life of a Digital Democracy" (English, book; on vTaiwan and Polis) — https://www.plurality.net/v/chapters/2-2/eng/
2. "What is Polis?" (English, official explanation) — https://compdemocracy.org/polis/
3. Small and others, "Polis: Scaling Deliberation by Mapping High Dimensional Opinion Spaces" (2021, English, paper) — https://doi.org/10.6035/recerca.5516
4. "Collective Constitutional AI: Aligning a Language Model with Public Input" (English, paper) — https://arxiv.org/abs/2406.07814
5. Broad Listening, section 4.4 "Making a map of public opinion with Polis" (in Japanese, book) — https://github.com/digitaldemocracy2030/broad-listening-book/blob/f69c60c838a91b91668abf109d7093b240610139/en/04_04_mapping_public_opinion_with_polis.md
6. Pol.is, Japanese-language Wikipedia (in Japanese, encyclopedia) — https://ja.wikipedia.org/wiki/Pol.is

Sources

1. "What is Polis?" (The Computational Democracy Project) — https://compdemocracy.org/polis/
2. Plurality, 2-2 "The Life of a Digital Democracy" — https://www.plurality.net/v/chapters/2-2/eng/ ［CC0］
3. Small and others, "Polis: Scaling Deliberation by Mapping High Dimensional Opinion Spaces" (2021) — https://doi.org/10.6035/recerca.5516 ［CC BY-SA 4.0］
4. Huang, Siddarth and others, "Collective Constitutional AI: Aligning a Language Model with Public Input" (2024) — https://arxiv.org/abs/2406.07814 ［CC BY 4.0］
5. Broad Listening, 4.4 "Making a map of public opinion with Polis" (in Japanese) — https://github.com/digitaldemocracy2030/broad-listening-book/blob/f69c60c838a91b91668abf109d7093b240610139/en/04_04_mapping_public_opinion_with_polis.md ［CC BY-NC 4.0］
6. Pol.is, Japanese-language Wikipedia — https://ja.wikipedia.org/wiki/Pol.is ［CC BY-SA 4.0］
7. "Algorithms" (The Computational Democracy Project) — https://compdemocracy.org/algorithms/
8. Pol.is, English-language Wikipedia — https://en.wikipedia.org/wiki/Pol.is ［CC BY-SA 4.0］
9. "Case studies" (The Computational Democracy Project) — https://compdemocracy.org/Case-studies/
10. Broad Listening, 11-1 "Taiwan" (in Japanese) — https://github.com/digitaldemocracy2030/broad-listening-book/blob/f69c60c838a91b91668abf109d7093b240610139/en/11_01_taiwan.md ［CC BY-NC 4.0］
11. CIP, "Alignment Assemblies: Nine Months In" (2023-11-16) — https://www.cip.org/blog/alignment-assemblies-nine-months-in
12. Plurality, 5-4 "Augmented Deliberation" — https://www.plurality.net/v/chapters/5-4/eng/ ［CC0］