Roko Mijic suggests rapid growth in AI capabilities could outpace human governance

Roko Mijic suggests rapid growth in AI capabilities could outpace human governance

On September 12, in a post on X, Roko Mijic described a risk: legal and political institutions may not keep pace with the sharp rise in AI capabilities to channel them into medicine, pensions, goods, and security. He treats governance as the condition required to convert new capabilities into tangible benefit for people.

According to Mijic, new ways of working and finding effective solutions benefit people through rules and institutions that distribute their results. He describes these as the fragile link:

"The central problem humanity faces with AI is that the legal and political systems we rely on to convert labor and optimization into benefits for ourselves, such as medical care, pensions, goods, and safe streets, are quite fragile."

These rules, Mijic writes, developed gradually alongside a familiar way of life. This is why he focuses on the speed of change. In his hypothetical example, a roughly 50% annual increase in AI's optimization ability could still be absorbed by institutions as unusually fast economic growth. At 1,000 to 10,000% per year, the author suggests, AI gains enough optimization power to bypass human governance.

In an April conversation, David Wood proposed a negotiated pause in the race toward general-purpose AI. Mijic also speaks of slowing down, but ties it to an additional task: rebuilding governance so that it can handle abrupt change.

In his examples, instead of paying taxes AI overthrows the state, and instead of curing aging it destroys people. This is how Mijic describes the scenario in which humans lose control of AI before they can direct it toward medical care.

"To survive AI, we will need to combine slowing AI down with rebuilding governance that can handle far more disruptive change."