People are asking to slow the pace of AI development

Concerns about the advent of new technology and the long-term prospects for humanity have been voiced for quite a long time. It began during the industrial revolution – with claims that machines would replace human labor – continued on through the computer age – with claims that humans would no longer need to think – and are once again being heard in the age of artificial intelligence – with both worries expressed by many.

The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question.
~ Samuel Butler, Darwin Among the Machines, June 13 1863.

Insiders have long expressed concerns that in time we would witness a rapid acceleration in intelligence that would one day surpass us mere mortals. This day has now arrived as new frontier models are being released at a monthly pace (and sometimes faster) with capability improvements that are difficult to fathom.

Let an ultraintelligent machine be defined as a machine that can far
surpass all the intellectual activities of any man, however clever. Since
the design of machines is one of these intellectual activities, an ultra-
intelligent machine could design even better machines; there would
then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind.
~ Irving John Good, Speculations Concerning the First Ultraintelligent Machine, Advances in Computers, Volume 6, 1965

I’m reminded of Bill Joy’s keen insight over a quarter century ago when he asked, “…shouldn’t we proceed with great caution?” While thrill seekers happen to be a unique exception, humans generally have a fear of the unknown – we’ll proceed with caution when we don’t know what’s around the corner.

Given the incredible power of these new technologies, shouldn’t we be asking how we can best coexist with them? And if our own extinction is a likely, or even possible, outcome of our technological development, shouldn’t we proceed with great caution?
~ Bill Joy, Why the Future Doesn’t Need Us, Wired Magazine, April 2000

This call for caution was voiced in July with the publication of Pacing the Frontier: A statement from 1,386 employees of frontier AI companies. Their warning that, “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems” mirrors those previous concerns that we are heading towards a cliff (my words) and that it might be time to change course. But the message fell on deaf ears in Washington, D.C.

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.

When Dario Amodei published We Must Pace the Frontier in September 2026, there seemed to be a glimmer of hope that those in power would finally listen, but such was not the case.

But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute.

While a number of states have passed AI-related legislation, the U.S. federal government continues to drag its collective feet, adopting a morally binding agreement that does nothing to protect the public. So a race to the bottom continues, alongside a race to jeopardize humanity’s future.