Anthropic claims to be near something along the lines of recursive self-improvement, or RSI, according to the latest revision of its S-1/A filing, published Monday. The new warning in the “Risk Factors” section of the document is a subtle but notable update spotted by TechCrunch.

RSI, if real, would be a point at which a computer system can improve itself autonomously, and then the improved version would be able to improve itself, and on, and on. A version of the concept was central to my own humble work on “the singularity” back in 2016, but jargon has changed with the times. RSI as it relates to contemporary AI is a somewhat hazy concept, although some specifics have been hashed out over the past few years in rationalist and effective altruist-affiliated spaces online like Lesswrong. It seems to have evolved from an earlier term, “seed AI,” coined by the godfather of AI doomerism, Eliezer Yudkowsky.

So Anthropic is disclosing to risk-averse investors—or perhaps announcing to a world eager to greet its mechanical god—that RSI may be arriving “imminently.” The October 26 version of Anthropic’s filing contains a section on the rate of “AI-driven acceleration,” but the company added a chunk claiming as follows:

“[…] our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI.’”

And according to my reading of the Responsible Scaling Policy, Anthropic is claiming that if dramatically accelerated progress attributable to AI were to arise, the company would hold itself to a pretty astonishing standard for preventing AI-related catastrophes.

Source link