What is superintelligence?

An intelligence that greatly outperforms the smartest humans in almost every field. Many AI companies have made it their goal, and at the same time it is feared as something that could threaten humanity. People on the Plurality side reframe it: superintelligence, they say, is already here, in ourselves working together.

You may have heard on the news that AI will one day be smarter than humans. Such an AI is called a "superintelligence". It is spoken of as a dream technology, and it is also feared as something that might bring about the end of humanity. This article explains what the word means and the different ways of thinking about it.

In short

Superintelligence is an intelligence that far outperforms the smartest humans in almost every field, from scientific creativity to social skills [1].

It is a goal that AI companies such as OpenAI, Meta and Microsoft have declared they will build. At the same time, some people see it as a possible threat to humanity's survival and call for a ban on developing it [2].

Audrey Tang and E. Glen Weyl, the authors of the book Plurality, reframe the word. Superintelligence, they say, is not something built in a data center: it is already here, in ourselves working together [3][4].

For example

Declarations by AI companies (2023–2025). In May 2023, OpenAI's leadership discussed governing superintelligence as future AI even more capable than artificial general intelligence (AGI). Saying that within ten years AI could exceed expert skill in most fields, they proposed an international body like the International Atomic Energy Agency (IAEA) [5].

In June 2024, Ilya Sutskever, formerly one of OpenAI's leaders, and others founded a new company, Safe Superintelligence Inc., saying that superintelligence was within reach [6]. In July 2025, Meta's Mark Zuckerberg wrote of bringing personal superintelligence to everyone [7]. In November 2025, Mustafa Suleyman of Microsoft AI announced an aim of "humanist superintelligence", limited to particular domains rather than an entity that decides everything for itself [8].

A statement calling for a ban (October 2025). A statement organized by the Future of Life Institute noted that many AI companies say they will build, within the next decade, superintelligence that outperforms humans at essentially all cognitive tasks. It called for a ban on developing it until there is broad scientific consensus that it can be done safely and controllably, and strong public buy-in. As of September 24, 2026, it had 75,329 signatures [2].

"We the People are Truly the Superintelligence" (May 2026). Tang opened her keynote at MIT SOLVE 2026 like this: "Today I want to talk about superintelligence. Superintelligence, to me, is not something we build in a data centre somewhere. It is in this room. It is in all of us. It is us" [3]. "The question today", she went on, "is whether the technologies we are building amplify that superintelligence, or whether they crowd it out" [3].

In more detail

Where the word comes from

In a 1998 paper, the philosopher Nick Bostrom defined superintelligence as an intellect much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills, whether it is built with computers or by some other means [1]. His 2014 book Superintelligence uses a definition of an intellect that greatly exceeds human cognitive performance in virtually all domains (as quoted in a paper he co-wrote with Vincent C. Müller the same year) [9].

Earlier, in 1993, the science fiction writer Vernor Vinge wrote, in essence, that within thirty years we would have the means to create superhuman intelligence, and that shortly after, the human era would end. He called this turning point the "Singularity" [10].

The singleton

In 2005, Bostrom used the term "singleton" for a world order in which there is a single decision-making agency at the highest level, able to prevent threats to its own supremacy and in effective control of the world. A democratic world government, a worldwide dictatorship, or a friendly superintelligent machine could all be examples, he wrote [11]. In arguments on the Plurality side, this singleton comes up again and again as a destination to avoid [12].

Superintelligence as national strategy

A March 2025 paper by Hendrycks and co-authors named a deterrence regime in which states sabotage each other's superintelligence projects, for example through cyberattacks, "Mutual Assured AI Malfunction" (MAIM), likening it to nuclear deterrence [13]. The debates over forecasts and races like this are summarized in the article on the AI futures debate.

Plurality's reframing 1: "The Plurality is here"

Chapter 2-2 of Plurality opens with a description of her own job that Tang wrote in 2016, in the form of a poem. It ends: "When we hear 'the singularity is near' — let us remember: The Plurality is here" [14].

In a 2022 essay, Weyl criticized mainstream thinking on AI for pursuing a single general intelligence that would leave the variety of intelligences behind, swapping the untidy work of ⿻ (plural) cooperation for optimization toward ends said to match human desires [15].

Reframing 2: the real superintelligence is already around us

In a talk at Harvard in November 2025, Weyl said that real superintelligence already surrounds us, in the form of religions, companies, markets and democracies. Instead of building an AI that defeats us, he called for thinking about how, together with our environment and our machines, we could become a planetary-scale collective superintelligence [4]. A report on the talk sums up Weyl's position as saying that superintelligence has already been here for thousands of years [16].

Reframing 3: horizontal, not vertical, superintelligence

Tang's manifesto on AI ethics, the 6-Pack of Care (September 2025), says: "The vertical narrative of a technological 'singularity' needs a horizontal alternative." Instead of a singleton, a single AI that manages everything, it pictures a society in which many specialized AIs coexist [17]. The manifesto writes: "The superintelligence we need is already here. It's the untapped potential of human collaboration. It's 'We, the People'" [17].

The front page of civic.ai, where it is published, adds a caveat, however: "It is us — not because a mini-public equals machine superintelligence, but because, under the right institutional conditions, people with tools produce accountable collective intelligence that no central optimiser can legitimately replace" [18].

What is debated

Does the Plurality side deny the risks of superintelligence? The civic.ai FAQ answers that this is a misunderstanding. It acknowledges the problem of aligning the values of powerful general AI with humanity as a real problem still unsolved after more than a decade, and treats the work of individual researchers and AI companies on it as necessary. The 6-Pack of Care does not claim to solve it, and takes on a different question: to whom AI is accountable in the settings close to people's lives [19]. On what happens if an uncontrollable superintelligence is built anyway, it does not claim to solve the control problem: "It does not hope. It builds. But it builds the second line of defence, not the first" [19].
Isn't "we are the superintelligence" just a metaphor? As the caveat above shows, Tang's side does not claim that a gathering of people has the same abilities as a machine superintelligence. The claim is that collective intelligence with legitimacy and accountability cannot be replaced by a central device [18].
How does it differ from Suleyman's "humanist superintelligence"? Suleyman's call for domain-limited superintelligence looks similar to the Plurality side's idea of many AIs with limited powers [8]. However, no writing from the Plurality side that directly discusses Suleyman's vision has been found so far.

Where it fits in Plurality

Chapter 2-0 of the book, "Information Technology and Democracy: A Widening Gulf", gives examples of science fiction and futurism shaping the direction of technology, including "the tendency of technology to concentrate power through creating 'superintelligences'", as found in the fiction of Asimov and Banks, the futurism of Kurzweil and Bostrom, and films such as Terminator and Her [20]. The same chapter groups AI-centered visions of the future as "Technocracy", one of the three paths [20].

Tang's poem at the start of chapter 2-2 is also discussed in the article on 數位 and ⿻. In a July 2026 interview, Tang said that four of the five lines of the 2016 poem had largely come true, and that she would add as a fifth: 我們全民才是真正的超級智慧 (we, all the people, are the true superintelligence) [21].

Vitalik Buterin offers a different path: for now, build AI as tools rather than as agents that act on their own, and eventually move toward a superintelligence in which humans and machines are tightly connected [22]. Buterin's thinking is introduced in the article on d/acc, Weyl's development of these ideas in the article on reverse alignment, and Tang's in the article on Civic AI.

Sources

Nick Bostrom, "How Long Before Superintelligence?" (1998) — https://nickbostrom.com/superintelligence
Future of Life Institute, "Statement on Superintelligence" (2025-10-22) — https://superintelligence-statement.org/
Audrey Tang, "We the People are Truly the Superintelligence" (MIT SOLVE 2026, 2026-05-14) — https://civic.ai/we-the-people-are-truly-the-superintelligence/ ［CC0］
Berkman Klein Center, "How to Be the Superintelligence You've Been Waiting For" (talk abstract, 2025-11) — https://cyber.harvard.edu/events/how-be-superintelligence-youve-been-waiting
OpenAI, "Governance of superintelligence" (2023-05-22) — https://openai.com/index/governance-of-superintelligence/
Announcement of the founding of Safe Superintelligence Inc. (2024-06-19) — https://ssi.inc/
Mark Zuckerberg, "Personal Superintelligence" (2025-07-30) — https://www.meta.com/superintelligence/
Mustafa Suleyman, "Towards Humanist Superintelligence" (2025-11-06) — https://microsoft.ai/news/towards-humanist-superintelligence/
Vincent C. Müller and Nick Bostrom, "Future Progress in Artificial Intelligence: A Survey of Expert Opinion" (2014) — https://nickbostrom.com/papers/survey.pdf
Vernor Vinge, "The Coming Technological Singularity" (1993) — https://edoras.sdsu.edu/~vinge/misc/singularity.html
Nick Bostrom, "What is a Singleton?" (2005) — https://nickbostrom.com/fut/singleton
civic.ai Glossary, "Singleton" — https://civic.ai/glossary/ ［CC0］
Hendrycks, Schmidt and Wang, "Superintelligence Strategy" (2025-03-05) — https://arxiv.org/abs/2503.05628 ［CC0］
Plurality, 2-2 "The Life of a Digital Democracy" — https://www.plurality.net/v/chapters/2-2/eng/ ［CC0］
Glen Weyl, "Why I Am A Pluralist" (2022-02-10) — https://www.radicalxchange.org/updates/blog/why-i-am-a-pluralist/
Berkman Klein Center, "Rethinking and reframing superintelligence" (2025-12) — https://cyber.harvard.edu/story/2025-12/rethinking-and-reframing-superintelligence
6-Pack of Care: A Manifesto (2025-09-01) — https://civic.ai/manifesto/ ［CC0］
civic.ai front page — https://civic.ai/ ［CC0］
civic.ai FAQ (Q15 and Q17) — https://civic.ai/faq/ ［CC0］
Plurality, 2-0 "Information Technology and Democracy: A Widening Gulf" — https://www.plurality.net/v/chapters/2-0/eng/ ［CC0］
唐鳳 Open Commons 專訪 (Open Commons interview with Audrey Tang, 2026-07-16, in Chinese) — https://sayit.archive.tw/2026-07-16-open-commons-%E5%B0%88%E8%A8%AA ［CC0］
Vitalik Buterin, "d/acc: one year later" (2025-01-05) — https://vitalik.eth.limo/general/2025/01/05/dacc2.html

We welcome error reports and questions ("What is…?"). Submissions become public issues.

Report an error in this article ・ Ask a question