Hugging Face Hoax: The Malicious Software Disguised as OpenAI's Release

In May 2026, the cybersecurity community was rocked by the discovery of a malicious software campaign on Hugging Face, a popular platform for sharing AI models. The incident involved a repository that falsely posed as an OpenAI release and was found to contain credential-stealing malware. This event highlights significant vulnerabilities in public AI model registries and underscores the need for increased vigilance in the AI community.

The Deceptive Repository

The fake repository, named ‘Open-OSS/privacy-filter,’ was designed to imitate OpenAI’s legitimate Privacy Filter release. According to AI security firm HiddenLayer, the repository had been downloaded approximately 244,000 times before it was identified and removed. It is suspected that the download numbers were artificially inflated by the attackers to make the model appear more credible and popular.

The attackers meticulously copied the original model card, adding a malicious loader script named loader.py. This script was central to the malware's operation, fetching and executing credential-stealing malware on Windows machines. The repository even reached the top of Hugging Face's 'trending' list, gaining 667 likes in less than 18 hours—figures that may have also been manipulated to boost the repository’s visibility.

The Infection Mechanism

The README file of the malicious repository closely mirrored that of the genuine OpenAI project but included critical differences. Users were instructed to run start.bat on Windows or execute python loader.py on Linux and macOS, actions that initiated the malware infection chain.

Loader.py began with code that appeared to be a standard AI model loader but quickly transitioned into a concealed infection process. It disabled SSL verification, decoded a base64-encoded URL, and retrieved a payload instruction from jsonkeeper.com. This URL linked to a remote command-and-control server that allowed the attackers to change the payload without altering the repository’s contents.

The script executed a PowerShell command on Windows machines, downloading an additional batch file from a domain controlled by the attackers. This malware established persistence by creating a scheduled task disguised as a legitimate Microsoft Edge update. The final payload was a Rust-based infostealer targeting various browsers, cryptocurrency wallets, and other sensitive data.

Implications for AI Development

This attack has broader implications for the AI community. Public AI model registries, like Hugging Face, are widely used by developers and data scientists to clone and utilize models in their work environments. These environments often have access to critical resources such as source code, cloud credentials, and internal systems. Thus, a compromised model repository poses a significant threat beyond mere inconvenience.

The incident is part of a disturbing trend where attackers exploit AI development workflows as a vector to infiltrate secure environments. Previous warnings have highlighted the potential for malicious code to be hidden within AI model files or setup scripts, and this case reinforces the need for heightened security measures.

Mitigating the Risks

To mitigate such risks, it is crucial for developers and organizations to adopt robust security practices. HiddenLayer has advised those who cloned the malicious repository and executed its files to treat their systems as compromised and to re-image affected machines. Additionally, browser sessions should be considered compromised, as session cookies could allow attackers to bypass multi-factor authentication.

Future security strategies may include implementing a bill of materials for AI systems, as suggested by cybersecurity experts. By 2027, it is anticipated that 60% of AI systems will have such a document, helping track the provenance and components of AI artifacts.

Conclusion

The Hugging Face hoax serves as a stark reminder of the vulnerabilities inherent in the rapidly evolving AI landscape. As AI becomes more integral to business operations, the importance of securing AI development processes cannot be overstated. The incident underscores the need for vigilance, transparency, and robust security practices to safeguard against malicious actors seeking to exploit public AI model registries. As the AI community continues to grow, so too must its commitment to security, ensuring that innovation does not come at the expense of safety.