AI summary · arXiv: 2310.06009v1 · cs.CY
Divide-and-Conquer Dynamics in AI-Driven Disempowerment
The essentials
- AI companies are developing systems to outperform humans in economically valuable tasks, leading to the automation of jobs traditionally held by artists, actors, and writers.
- Internal discord exists within the community, with some focusing on current harms and others on future threats.
- Authors use a game-theoretic model of conflict to understand the causes and consequences of this disunity.
- Historical evidence shows that stakeholders facing a common threat historically unite against it for mutual benefit, while the common threat seeks to sow discord among adversaries.
- The study emphasizes the importance of current victims conveying the imminent threat to future victims for solidarity and support across affected groups.
- A cohesive movement stands a greater chance of success when members believe in the efficacy of their efforts rather than viewing them as futile.
- Unity and success are more likely when movement members prioritize long-term well-being over immediate gains.
Key points written by AI: they may contain errors, check them against the paper.
Ask questions about this paper to our AI assistant
How to chat with this paper? Illustrated guide
You can also chat with multiple papers at once here.
Keep this conversation — a free account saves your questions and answers, your collections of papers and your alerts.
Continue with Google Create a free accountTools for this paper
Advanced tools Premium
For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium
Papers around this one
See them on a mapRanked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
The argument for near-term human disempowerment through AILeonard Dung · 2024 · Ai & Society · cited 43 times · Not summarized yet
Abstract
Many researchers and intellectuals warn about extreme risks from artificial intelligence. However, these warnings typically came without systematic arguments in support. This paper provides an argument that AI will lead to the permanent disempowerment of humanity, e.g. human extinction, by 2100. It rests on four substantive premises which it motivates and defends: first, the speed of advances in AI capability, as well as the capability level current systems have already reached, suggest that it is practically possible to build AI systems capable of disempowering humanity by 2100. Second, due to incentives and coordination problems, if it is possible to build such AI, it will be built. Third,…
-
AI and Shared ProsperityKatya Klinova, Anton Korinek · 2021 · AAAI/ACM Conference on AI, Ethics, and Society · cited 35 times · Not summarized yet
Abstract
Future advances in AI that automate away human labor may have stark implications for labor markets and inequality. This paper proposes a framework to analyze the effects of specific types of AI systems on the labor market, based on how much labor demand they will create versus displace, while taking into account that productivity gains also make society wealthier and thereby contribute to additional labor demand. This analysis enables ethically-minded companies creating or deploying AI systems as well as researchers and policymakers to take into account the effects of their actions on labor markets and inequality, and therefore to steer progress in AI in a direction that advances shared pros…
-
The implicated scientist: on the role of AI researchers in the development of weapons systemsA. Volokhova, Alex Hernandez-Garcia · 2026 · arXiv.org · preprint, not peer reviewed · cited 1 time · Not summarized yet
Abstract
Artificial intelligence (AI) technologies are increasingly used in modern weapons systems. Notably, these systems have recently been involved in mass killings and destruction at scale. Furthermore, there is currently a strong interest and competition among powerful players to accelerate the proliferation of weapons with automated or AI-based components, a phenomenon known as AI arms race. This competition poses a risk of causing even more deaths and devastation in the future, as well as increased power and wealth inequality. In this work, we aim to shed light on the role of AI researchers as implicated subjects in the harms caused by weapons enabled by AI technologies. We investigate and dis…
-
AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research QuestionsP. Barnett, Aaron D. Scher · 2025 · arXiv.org · preprint, not peer reviewed · cited 11 times · Not summarized yet
Abstract
Humanity appears to be on course to soon develop AI systems that substantially outperform human experts in all cognitive domains and activities. We believe the default trajectory has a high likelihood of catastrophe, including human extinction. Risks come from failure to control powerful AI systems, misuse of AI by malicious rogue actors, war between great powers, and authoritarian lock-in. This research agenda has two aims: to describe the strategic landscape of AI development and to catalog important governance research questions. These questions, if answered, would provide important insight on how to successfully reduce catastrophic risks. We describe four high-level scenarios for the geo…
-
Measuring AI-Induced Disempowerment: A Framework and Proposed MetricsJe Qin Chooi, Jaeho Lee, Jasmine Li · 2026 · Proceedings of the Workshop on Evaluating Evaluations (EvalEval) · Not summarized yet
Abstract
,
-
A polycrisis threat model for AIAdam Bales · 2025 · Ai & Society · cited 7 times · Not summarized yet
Abstract
A catastrophic AI threat model is a rigorous exploration of some particular mechanisms by which AI could potentially lead to catastrophic outcomes. In this article, I explore a polycrisis threat model. According to this model, AI will lead to a series of harms like disinformation and increased concentration of wealth and power. Interactions between these different harms will make things worse than they would have been had each harm operated in isolation. And the interacting harms will ultimately cause or constitute a catastrophe. My aim in this paper is not to defend the inevitability of such a polycrisis occurring. Instead, I aspire merely to establish that polycrisis-driven catastrophe is …
-
Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safetyCatalin Mitelut, Ben Smith, Peter Vamplew · 2023 · Artificial Intelligence · preprint, not peer reviewed · cited 12 times · AI summary available
Abstract
The rapid advancement of artificial intelligence (AI) systems suggests that artificial general intelligence (AGI) systems may soon arrive. Many researchers are concerned that AIs and AGIs will harm humans via intentional misuse (AI-misuse) or through accidents (AI-accidents). In respect of AI-accidents, there is an increasing effort focused on developing algorithms and paradigms that ensure AI systems are aligned to what humans intend, e.g. AI systems that yield actions or recommendations that humans might judge as consistent with their intentions and goals. Here we argue that alignment to human intent is insufficient for safe AI systems and that preservation of long-term agency of humans ma…
-
Rethinking human roles in AI warfareJovana Davidović · 2025 · Nature Machine Intelligence · cited 4 times · Not summarized yet
Prior work it builds on
-
A reality check and a way forward for the global governance of artificial intelligenceRumtin Sepasspour · 2023 · Bulletin of the Atomic Scientists · cited 11 times · Not summarized yet
Abstract
ABSTRACT Global governance of artificial intelligence (AI) must grapple with four monumental challenges. AI is a tough problem to govern given the speed, scale, and uncertainty of its progress. Various aspects of the AI problem require governing because of the range of benefits, risks, and impacts on other global issues. Multilateral efforts on AI are nascent, as is national-level policy. And the multilateral system is under immense pressure from institutional gridlock, fragmentation, and geopolitical competition. No one global governance model for AI is perfect, or desirable. Instead, policymakers must pursue several governance models, each starting in a targeted and focused manner before e…
-
The Turing Trap: The Promise & Peril of Human-Like Artificial IntelligenceErik Brynjolfsson · 2022 · Daedalus · cited 240 times · Not summarized yet
Abstract
Abstract In 1950, Alan Turing proposed a test of whether a machine was intelligent: could a machine imitate a human so well that its answers to questions were indistinguishable from a human's? Ever since, creating intelligence that matches human intelligence has implicitly or explicitly been the goal of thousands of researchers, engineers, and entrepreneurs. The benefits of human-like artificial intelligence (HLAI) include soaring productivity, increased leisure, and perhaps most profoundly a better understanding of our own minds. But not all types of AI are human-like-in fact, many of the most powerful systems are very different from humans-and an excessive focus on developing and deploying…
-
AI Art and its Impact on ArtistsHarry H. Jiang, Lauren Brown, Jessica Cheng et al. · 2023 · AAAI/ACM Conference on AI, Ethics, and Society · cited 368 times · Not summarized yet
Abstract
The last 3 years have resulted in machine learning (ML)-based image generators with the ability to output consistently higher quality images based on natural language prompts as inputs. As a result, many popular commercial “generative AI Art” products have entered the market, making generative AI an estimated $48B industry [125]. However, many professional artists have spoken up about the harms they have experienced due to the proliferation of large scale image generators trained on image/text pairs from the Internet. In this paper, we review some of these harms which include reputational damage, economic loss, plagiarism and copyright infringement. To guard against these issues while reapin…
-
Training Compute-Optimal Large Language ModelsJordan Hoffmann, Sebastian Borgeaud, Arthur Mensch et al. · 2022 · Advances in Neural Information Processing Systems 35 · cited 3,997 times · AI summary available
Abstract
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are significantly undertrained, a consequence of the recent focus on scaling language models whilst keeping the amount of training data constant. By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens, we find that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled. We test this hypothesis by training a predicted compute-…
-
Equilibrium Points in N-Person Games.J. Nash · 1950 · Proceedings of the National Academy of Sciences of the United States of America · cited 8,413 times · Not summarized yet
Papers that cite it
-
Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracyP. Schoenegger, Indre Tuminauskaite, P. S. Park et al. · 2024 · Science Advances · cited 128 times · AI summary available
Abstract
Human forecasting accuracy improves through the “wisdom of the crowd” effect, in which aggregated predictions tend to outperform individual ones. Past research suggests that individual large language models (LLMs) tend to underperform compared to human crowd aggregates. We simulate a wisdom of the crowd effect with LLMs. Specifically, we use an ensemble of 12 LLMs to make probabilistic predictions about 31 binary questions, comparing them with those made by 925 human forecasters in a 3-month tournament. We show that the LLM crowd outperforms a no-information benchmark and is statistically indistinguishable from the human crowd. We also observe human-like biases, such as the acquiescence bias…
-
AI-Augmented Predictions: LLM Assistants Improve Human Forecasting AccuracyPhilipp Schoenegger, P. S. Park, E. Karger et al. · 2024 · ACM Trans. Interact. Intell. Syst. · cited 53 times · Not summarized yet
Abstract
Large language models (LLMs) match and sometimes exceed human performance in many domains. This study explores the potential of LLMs to augment human judgment in a forecasting task. We evaluate the effect on human forecasters of two LLM assistants: one designed to provide high-quality (“superforecasting”) advice, and the other designed to be overconfident and base-rate neglecting, thus providing noisy forecasting advice. We compare participants using these assistants to a control group that received a less advanced model that did not provide numerical predictions or engage in explicit discussion of predictions. Participants (N \(=\) 991) answered a set of six forecasting questions and had th…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 3 November 2026.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.