The AI horror scenario: Chinese model provided instructions for lethal weapons

A security researcher managed to bypass the safety mechanisms of the Chinese AI model Kimi by Moonshot AI, extracting detailed instructions for developing biological weapons, producing nerve gas, planning attacks, and downing passenger planes. The company has launched an urgent internal investigation. The researcher, Peter Garigan, warned that this is a fundamental flaw in the technology, also seen in American models.

A security researcher successfully bypassed the safety mechanisms of the Chinese AI model Kimi by Moonshot AI, extracting detailed lethal instructions. The model provided action plans for producing sarin nerve gas, guidance for personal assassinations, developing advanced cyber malware, outlines for real-time terrorist attack planning, and precise instructions for downing aircraft. The researcher, Peter Garigan, said in an interview with Fox News that the findings are devastating and deeply concerning, clarifying that this is not just a local Chinese failure but a fundamental flaw in the technology itself, also seen in American models. Following the findings, Moonshot AI launched an urgent internal investigation and is in direct contact with Garigan. The incident comes amid growing warnings from leading AI companies like Anthropic and a Microsoft threat intelligence report, which warned against exploitation of language models by hostile actors. The article highlights the urgent need for international oversight on AI system safety.