Skip to content
What's happening. What it means
ViralBrief

What's happening. What it means

Latest Former Anthropic Researcher Leads Protests Against AI ‘Recursive Self-Improvement’ on IPO Day
Viral Trends

Former Anthropic Researcher Leads Protests Against AI ‘Recursive Self-Improvement’ on IPO Day

Olamide

As Anthropic celebrated a $1.7 trillion market debut, safety advocates gathered at the company's San Francisco headquarters to demand a ban on recursive self-improvement technology.

A Day of Contrasts at Anthropic HQ

While Anthropic celebrated a massive milestone on Thursday with a blockbuster initial public offering that valued the company at $1.7 trillion, the atmosphere outside its San Francisco headquarters was marked by tension. Former Anthropic researcher Jacob Coxon, whose departure in September sparked widespread public discourse regarding AI safety, led a demonstration aimed at refocusing attention on the risks associated with advanced artificial intelligence development.

The protest, dubbed “Ban RSI,” centered on the concept of recursive self-improvement (RSI)—a process where advanced AI systems are utilized to train and develop subsequent generations of technology. Coxon, who was joined by other prominent figures in the AI safety space, argued that the financial success of the company should not overshadow the potential dangers of the technology being developed within its walls.

The Call for a Global Ban

During the demonstration, Coxon addressed the crowd, emphasizing that the financial valuation of the company is secondary to the risks posed by RSI. “It literally doesn’t matter whether they invest,” Coxon stated. “We have to stop RSI.” He argued that because only a limited number of facilities globally possess the capability to perform recursive self-improvement as of 2027, the responsibility lies with these specific organizations to halt the practice. Coxon called for Anthropic to actively lobby the government and coordinate with industry peers to establish a comprehensive ban on the technology.

See also  Sir Patrick Stewart, Iconic Star of Star Trek and X-Men, Dies at 86

The protest featured speeches from several notable figures, including former OpenAI researcher Daniel Kokotajlo and YouTuber Rob Miles. Kokotajlo highlighted the disconnect between public perception and the private concerns held by many in the research community, noting that the rapid advancement of AI often feels like the work of a “comic book supervillain.”

Industry Concerns and Regulatory Scrutiny

The protest comes at a time when the risks of AI have gained traction in legislative circles, with members of Congress beginning to explore potential restrictions on RSI. Within the industry, there is a growing debate regarding the balance between competitive advantage and safety protocols. Oliver Habryka, who operates the LessWrong forum, expressed a cautious optimism that Anthropic’s new status as a public company will mandate greater transparency. However, he also voiced concerns that the pressure to maintain a market lead could incentivize the company to accelerate the use of RSI, particularly in sensitive fields like biology.

“AI has grown rapidly more capable over the last few years,” Habryka noted. “With RSI, things can go much faster than that. And that is terrifying.”

See also  Israeli President Tasks Gadi Eisenkot with Government Formation Following Stalled Election

Internal Policy and Public Perception

Anthropic has acknowledged the complexity of these issues in its IPO prospectus, stating that it does not believe allowing AI to design its next generation without human supervision would be safe. The company indicated it is considering policy updates to ensure humans remain in control of frontier model development. Furthermore, an independent report from the AI safety organization METR recently suggested that AI is already playing a role in accelerating Anthropic’s internal research and development.

Despite these acknowledgments, critics remain skeptical. David Krueger, head of the nonprofit Evitable, suggested that public awareness of RSI is still too low, noting that if the general public understood the implications, they would likely oppose the practice. Krueger also expressed doubt regarding the immediate impact of the protests on the company’s internal culture, suggesting that employees may be deeply committed to the company’s current trajectory. Anthropic did not provide a comment regarding the demonstration or the specific demands made by the protesters.

Source: businessinsider.com

ViralBrief NEWSLETTER

Enjoyed this story?

Get more important stories from ViralBrief delivered directly to your inbox.

Choose topics (optional)
By subscribing, you agree to receive email updates from ViralBrief. You can unsubscribe at any time.
What do you think of this story?Your feedback helps readers discover useful reporting.
COMMUNITY

Join the conversation

0 comments

No comments yet. Be the first to share a thoughtful response.

Leave a comment

Keep it respectful and relevant. Comments may be moderated before appearing.

Leave a Reply

Your email address will not be published. Required fields are marked *

error: Content is protected !!