Nick Bostrom

Nick Bostrom is a Swedish philosopher whose work established existential risk as a research programme and whose 2014 book Superintelligence moved the risk from advanced artificial intelligence into mainstream policy discussion. He founded and directed the Future of Humanity Institute at Oxford from 2005 until its closure in 2024, and he is one of the small number of academic philosophers who has taken transhumanist claims seriously enough to argue for them in peer-reviewed venues.

§Career

Bostrom studied physics, computational neuroscience and philosophy across four institutions before taking a doctorate at the London School of Economics in 2000 on anthropic reasoning — the problem of what an observer may infer from the fact of their own existence. That thesis became Anthropic Bias (2002), and the observation-selection machinery in it recurs throughout his later work.

In 1998 he co-founded the World Transhumanist Association with the philosopher David Pearce; the organization later became Humanity+. In 2005 he founded the Future of Humanity Institute within Oxford's Faculty of Philosophy, assembling an unusual group of philosophers, mathematicians and computer scientists, among them Anders Sandberg, Toby Ord and, later, Eric Drexler. FHI produced work on Whole brain emulation, global catastrophic risk, Human enhancement and AI alignment for nearly two decades. It closed in April 2024 after prolonged administrative conflict with the faculty, including restrictions on hiring and fundraising. Bostrom left Oxford and now runs the Macrostrategy Research Initiative.

§Key ideas

§Existential risk

Bostrom's 2002 paper defined an existential risk as one that would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential, and classified such risks as bangs, crunches, shrieks and whimpers.¹ The definition does real work: it separates catastrophes that a civilization recovers from within centuries from those that foreclose the future, and it makes the size of the loss depend on how large that foreclosed future could have been — an argument he developed in "Astronomical Waste".² Engineered pandemics, discussed here under Dual-use research of concern, are usually treated as the leading biological case. The framework is now standard vocabulary; see Existential risk.

§The simulation argument

The simulation argument is a trilemma rather than a claim that reality is simulated. Bostrom argues that at least one of the following holds: almost no civilization reaches a stage capable of running ancestor simulations; almost no such civilization chooses to run them; or almost all observers with experiences like ours are simulated.³ The argument depends on Substrate independence — that a sufficiently detailed computational process could instantiate conscious experience — which he flags explicitly as a premise rather than a conclusion.

§The vulnerable world hypothesis

Bostrom asks what follows if some technology, not yet invented, is easy to build and reliably destructive: an "easy nukes" scenario in which civilization draws a black ball from the urn of possible inventions.⁴ His conclusion is uncomfortable and he says so — that surviving such a draw would require either extremely effective preventive policing or strong global governance — and the paper functions partly as an argument for Differential technological development, the sequencing of technologies so that defensive capability arrives before offensive.

§Enhancement and transhumanism

Bostrom has defended human enhancement in analytic terms. "In Defense of Posthuman Dignity" (2005) answers Leon Kass, Francis Fukuyama and Jürgen Habermas by arguing that dignity does not attach to a particular biological configuration.⁵ With Sandberg he formulated the "wisdom of nature" heuristic, which asks why evolution has not already made a proposed improvement and treats a good answer as a precondition for attempting it — an unusual concession to the Precautionary principle from a pro-enhancement writer. "The Fable of the Dragon-Tyrant" (2005) recasts aging as a monster society has rationalized into acceptance, and remains the most widely circulated argument for treating defeating aging as urgent.

What the fable does and does not argue

The Dragon-Tyrant is an argument against the "wisdom of acceptance" position on mortality, not a forecast. Bostrom does not claim aging is close to solved; the fable's point is that the moral case for trying does not depend on how close it is.

§Superintelligence

Superintelligence: Paths, Dangers, Strategies (2014) argues that a machine intelligence exceeding human capability across the board would be difficult to control by default, because capability and goal content vary independently, and because a wide range of final goals generate the same instrumental subgoals of self-preservation, resource acquisition and resistance to modification.⁶ The book gave AI safety a shared vocabulary — orthogonality, instrumental convergence, the treacherous turn, decisive strategic advantage — and moved the subject from mailing lists into governments. Its central scenario, a single system undergoing fast recursive self-improvement in isolation, has aged less well than its vocabulary: the actual trajectory of Artificial general intelligence since 2022 has involved many large models improving gradually and in public.

§Reception

Bostrom is cited across philosophy, AI policy and popular writing, and Superintelligence influenced public statements by several technology executives and national AI strategies. Criticism comes from three directions. Philosophers dispute the probability assignments underlying the simulation argument and the anthropic reasoning it uses. AI researchers, including prominent figures who consider present systems far from general intelligence, have argued that his scenarios abstract away from how machine learning actually works. Critics of longtermism argue that weighting vast hypothetical future populations distorts present priorities.

In 2023 a message Bostrom sent to an extropian mailing list in 1996, containing a racial slur, resurfaced. He apologized; the apology drew further criticism, and Oxford said it was looking into the matter. The episode preceded FHI's closure, though the institute's difficulties with the faculty were longstanding and predated it.

§Legacy

Two of Bostrom's constructions have outlived the debates that produced them: the definition of existential risk, which now organizes an entire research and funding ecosystem, and the vocabulary of AI alignment. Deep Utopia (2024) turns to the harder question left over — what people would do with themselves in a world where technology has solved the instrumental problems, and whether a "solved world" leaves any content to a human life. That question, not the risk arguments, is the one the field has least equipment to answer.

§See also

• Existential risk
• Technological singularity
• Whole brain emulation
• Differential technological development
• Transhumanism
• Anders Sandberg
• Posthuman
• Leon Kass

§References