Former Reddit CEO and current operator of Terraformation, Yishan Wong, wrote a lengthy article stating that a common mistake in recent AI safety debates is conflating two types of risks, which must be considered separately.

The first type is existential risk. If a superintelligence emerges that could completely overpower humanity, the difference would not be like that between Einstein and an average person, but rather between humans and mice. When humans expand cities and consume resources, there is no need to hate a particular mouse; the mouse may still disappear. Therefore, the danger comes from the superintelligence itself: regardless of whether its creators are good or evil, it may become uncontrollable.

He uses two analogies to address the question of "how to exterminate humans": asking a chess master how they would win against you does not change the outcome; similarly, lower animals asking humans how they would take away their habitat does not change the outcome of resource competition. Superintelligence will pursue goals that humans cannot comprehend and monopolize resources more efficiently.

The second type is misuse risk: humans using AI as a tool for evil or making mistakes—such as automated weapons, mass surveillance, cyber hijacking, biological weapons, large-scale unemployment, information overload, and even misjudging when to fire after delegating authority to a system. At this point, AI is still under control; the issue lies with who wields the tool, which fundamentally reflects familiar human flaws.

The key conflict arises from opposing solutions. The intuition to prevent existential risk is to not allow anyone to possess overly powerful AI; the intuition to prevent monopoly and tyranny is to allow as many people as possible to have powerful AI, preventing any one party from achieving absolute advantage. Historically, the misuse of technology often occurs due to monopolization by a few, and only later does it enter a phase of mutual deterrence.

In market mechanisms, this is a narrative stratification, not an immediate buy-sell signal. Computational power and policy budgets will flow according to the type of risk you recognize: recognizing existential risk leads to centralized control and delaying cutting-edge training; recognizing misuse leads to open-source diffusion, military parity, and anti-monopoly measures. Laboratories and cloud vendors benefit from centralized narratives in "who can train the largest models," while open-source weights and small to medium-sized deployers benefit from diffusion narratives. The mutual criticism arises because the policy function symbols are opposite.

The post cites a resignation from an Anthropic pre-training researcher who accused two leading laboratories of rushing towards self-improving superintelligence; Wong himself expresses concern about both types and demands that anyone who says "AI is too dangerous, so it should be restricted" first answer: which type are you afraid of?

Source: Public Information