Anthropic warns it is nearing milestone for self-improving AI

Artificial intelligence company Anthropic has warned it is close to building a machine that could speed up its own enhancement, raising fears about the safety risks of AI becoming more independent, TechCrunch reported.

The warning was issued in its updated registration statement with the US Securities and Exchange Commission on Monday. Anthropic said its own calculations have shown that it may hit its definition of “significantly accelerated progress caused by AI” in the near future, which is linked to the concept of recursive self-improvement (RSI).

RSI is the point when AI systems can augment the power of their own systems in order to build newer, larger models, which could accelerate technological progress.

“Employing AI to develop better AI might also present significant safety concerns, especially if it significantly speeds up the rate of progress,” Anthropic said in the filing.

The firm agreed that research and development carried out with AI assistance might proceed more rapidly than its capacity to introduce appropriate safeguards.

But the company said its models have not yet crossed the line and that it does not intend to give Claude the task of self-designing and self-building newer models of increasingly capable AI that can do so without any human oversight.

The company said it was exploring revisions to its Responsible Scaling Policy to ensure humans keep control of frontier AI.