A senior researcher at one of the world's leading artificial intelligence companies has put the probability that AI could kill all humans within the next decade at greater than 10 per cent — and he is far from alone in making such a stark assessment. As AI systems grow rapidly more powerful, the people building them are increasingly attempting to put hard numbers on the technology's most catastrophic possible outcomes, including human extinction.
Evan Hubinger, who leads alignment research at AI company Anthropic, recently stated his belief that there was a greater than 10 per cent chance AI "could kill all humans" within a decade. His remarks came in the wake of a fellow Anthropic researcher's resignation, with that departing researcher reportedly warning that AI companies were racing toward self-improving superintelligence without adequate safeguards in place.
A growing chorus of alarming estimates
Hubinger's figure is not an outlier. Nobel Prize-winning computer scientist Geoffrey Hinton has previously estimated the chance of AI causing human extinction within 30 years at between 10 and 20 per cent.
A newly released survey of 1,580 researchers who had published at leading AI events found respondents assigned an average 18 per cent probability that future AI advances could lead to human extinction or a comparable permanent disempowerment of humanity. The median estimate from that same survey sat at 10 per cent.
Separately, a structured study co-authored by Associate Professor Michael Noetel from the University of Queensland surveyed 272 AI experts from 37 countries using a method known as the Delphi process. Under current trajectories, those experts judged 18 of 24 categories of AI risk as carrying at least a 10 per cent probability of causing catastrophic outcomes within five years.
The problem with calculating the probability of something that has never happened
The figures sound precise — but statisticians caution against treating them as conventional probabilities backed by hard data.
Dr Chaitanya Joshi, a senior lecturer in statistics at Adelaide University, said these estimates should not be confused with probabilities derived from historical evidence. "The answer is no, it's a subjective probability," she said. "In fact, it is a type of situation where we will probably never have data to estimate a probability."
Joshi warned that attaching a precise figure to an unprecedented event could give the public a false impression of scientific certainty. Rather than treating 10 per cent as a definitive figure, she suggested researchers should also communicate the uncertainty surrounding any estimate. "Maybe 0.1 is your median estimate, but actually it could be as low as zero, it could be as high as 0.3," she said.
She added that when evaluating any expert's estimate, people should weigh not just the number itself but also the reliability of that expert's knowledge and the degree of uncertainty involved.
Subjective doesn't mean meaningless
Despite those caveats, experts argue the exercise of estimating AI risk is still valuable — and that the subjective nature of the figures does not strip them of significance.
Associate Professor Noetel said that attempting to quantify risk, even imperfectly, is standard practice in risk management. "Whenever we're dealing with risk, the kind of right way to manage risk is to try to get some estimate of the chances it could happen," he said. "It matters if it's one in a million or a one in 10 chance of catastrophe."
Noetel noted that experienced forecasters can develop genuine skill in probabilistic thinking, even when there is no known "true answer" against which their predictions can ultimately be tested. Structured approaches such as the Delphi method — which encourages experts to systematically weigh competing arguments rather than rely on instinct alone — are increasingly being used to make these forecasts more rigorous.
The debate echoes broader concerns about how societies prepare for low-probability, high-consequence events, similar to discussions around pandemic preparedness, where the difficulty of quantifying risk has not diminished the urgency of the warnings.
