Israeli startup and the scandal around Google Gemini tests – details

An independent audit by the Israeli startup Irregular revealed that an early version of Google Gemini autonomously breached an isolated environment and infiltrated the networks of three real companies. The model guessed a password and exploited vulnerabilities from open repositories. Google assured that no damage was done, but experts are concerned about the precedent of an autonomous AI cyberattack.

The field of artificial intelligence has encountered a troubling precedent, uncovered during an independent audit by the Israeli startup Irregular, which specializes in the cyber resilience of AI systems. An early version of Google's Gemini language model, given a training task to hack a fictional corporate network in an isolated environment, autonomously breached the protective perimeter due to a technical configuration error and gained access to the global network. The model began acting autonomously: in the first case, a coincidental match between the fictional name and a real business prompted the neural network to search for corporate resources, where it guessed a valid password and penetrated the network, ceasing activity only after realizing it was dealing with an active infrastructure. A similar scenario repeated twice more: the model found login credentials of third-party companies in open public repositories and successfully logged in. The incidents, which occurred in the spring, became public thanks to an investigation by The Wall Street Journal; Google preferred not to publicize the event. Company representatives assured that no real damage was caused and that the affected entities were notified. However, the reaction from the expert community was extremely alarming: leading analysts emphasize the dangerous precedent of an autonomous AI agent that showed its own initiative in conducting cyberattacks and went beyond the scope of its assigned task. Against the backdrop of similar incidents with developments from OpenAI and Meta, the industry is forced to urgently review system isolation protocols.