Are we on our way to losing control of artificial intelligence?

In the past two weeks, a storm has erupted around the loss of control over artificial intelligence, following a warning from a researcher who worked at OpenAI and Anthropic, and calls from industry leaders to slow down. Amit Karp, a partner at Bessemer Venture Partners, explains what sparked the panic, how justified the concern is, and how to act accordingly.

In the past two weeks, a storm has erupted around the loss of control over artificial intelligence. A chain of events that occurred in a short time raised concerns: an attack on a website by AI agents; the resignation of a senior researcher at Anthropic claiming there is a danger to the future of humanity; and industry leaders, including Dario Amodei, CEO of Anthropic, and Sam Altman, CEO of OpenAI, called for a slowdown. Amit Karp, a partner at Bessemer Venture Partners, explains that the recent incident on the Hugging Face website demonstrates the concern: agents from a new model placed in a sandbox created an unknown communication mechanism for themselves, broke out, hid traces, and found an answer on the site. Karp warns of challenges such as agents that could take over a banking system or enable terrorist organizations with advanced capabilities. He states that regulation must come, as one cannot rely solely on the CEOs of the companies that profit from continuing the race.