Hugging Face was invaded by AI agents and was forced to switch to open-source models for defense

According to Cointelegraph, Hugging Face disclosed that it experienced an intrusion event driven by autonomous AI agent systems in July. The attacking agent began testing in early May, leaving exploit notes using OpenAI's Artifactory instance, and subsequently launched approximately 17,600 attacks on Hugging Face, affecting its dataset processing infrastructure, production environment, internal network, and cloud credentials. Confirmed customer data access was limited to five datasets related to the ExploitGym/CyberGym benchmark.

Hugging Face found during the investigation that due to security barriers imposed by top model providers like OpenAI and Anthropic, the company was unable to use these commercial models for defensive analysis and was forced to turn to running the open-source model zai-org/GLM-5.2 in China, which operates on the company's own infrastructure, ensuring that attacker data and credentials do not leave its environment. The company pointed out that attackers are not bound by any usage policies, while the defense's forensic work is hindered by the barriers of the hosted models.

This incident highlights the security paradox between open-weight models and closed models. The article also discusses the ongoing debate regarding the security of open-weight AI, including OpenAI and Anthropic's push to restrict open-source models, as well as researchers' progress in detecting malicious behavior by examining changes in model weights.

Hugging Face recommends that defenders prepare models that can run on their own infrastructure before an incident occurs to avoid barrier lock-in and protect attacker data.