Malicious ML Models on Hugging Face: How Cybercriminals Exploit Pickle Files 2025

With the rapid growth of artificial intelligence (AI) and machine learning (ML), cybersecurity threats targeting these technologies are on the rise. Recently, cybersecurity researchers uncovered malicious ML models on Hugging Face that exploit broken Pickle files to bypass security scans and deploy malicious payloads.

This technique, called nullifAI, represents a sophisticated attempt to compromise ML supply chains and evade detection by traditional security tools. In this article, we will break down how these malicious ML models on Hugging Face operate, why Pickle files are a security risk, and what steps developers can take to protect themselves.

How Malicious ML Models on Hugging Face Were Discovered

Uncovering the Threat

Researchers at ReversingLabs identified two malicious ML models on Hugging Face that leveraged a new evasion technique. The suspicious repositories were:

• glockr1/ballr7
• who-r-u0000/0000000000000000000000000000000000000

The malicious ML models on Hugging Face contained hidden Python-based reverse shell payloads embedded in Pickle files within PyTorch archives. These payloads enabled remote execution by connecting to hardcoded IP addresses.

Why Pickle Files Pose a Security Risk

The Pickle serialization format is commonly used in ML for saving and loading models. However, it has a major security flaw: it allows arbitrary code execution during deserialization. Cybercriminals exploit this feature to inject malicious scripts, making malicious ML models on Hugging Face a growing concern.

How Malicious ML Models on Hugging Face Evade Detection

1. Inserting Malicious Payloads at the Start of Pickle Files

One key aspect of the attack is that the malicious ML models on Hugging Face insert their payload at the beginning of the Pickle file. This ensures that:

• The payload executes before the Pickle decompilation fails.
• The security scanner registers an error but fails to stop the execution.

2. Using 7z Compression to Bypass Picklescan

Hugging Face relies on Picklescan, a tool designed to detect malicious Pickle files. However, the malicious ML models on Hugging Face used 7z compression instead of ZIP, preventing Picklescan from recognizing them as PyTorch archives.

3. Exploiting a Deserialization Mismatch

The malicious ML models on Hugging Face also take advantage of a discrepancy between Picklescan’s detection process and Python’s deserialization behavior. This allows the attack to execute even when Picklescan flags the file as potentially dangerous.

The Implications of Malicious ML Models on Hugging Face

1. Increased Risks for Open-Source ML Projects

The discovery of malicious ML models on Hugging Face highlights a critical weakness in open-source AI platforms. Developers often download and use pre-trained models without thorough security checks, making them vulnerable to:

• Supply chain attacks that inject backdoors into AI applications.
• Data exfiltration via remote access payloads.
• Widespread vulnerabilities in AI-driven industries.

2. The Need for Stricter ML Security Standards

These malicious ML models on Hugging Face show that existing security tools are not enough. The AI community must adopt:

• More robust model verification methods.
• Enhanced sandboxing for running untrusted models.
• Stronger security policies for open-source contributions.

How to Protect Against Malicious ML Models on Hugging Face

1. Avoid Using Pickle for Model Serialization

Instead of Pickle, use safer alternatives like:

• ONNX (Open Neural Network Exchange) – A standardized, secure format.
• TensorFlow SavedModel – Offers integrity verification mechanisms.

2. Verify ML Models Before Use

Always check malicious ML models on Hugging Face for:
–Unusual compression formats like 7z instead of ZIP.
– Suspicious contributors or repositories with limited activity.
–Unexpected network connections initiated by the model.

3. Run ML Models in a Sandbox

Before deploying models, execute them in isolated environments such as:

• Docker containers
• Virtual machines (VMs)

4. Implement Advanced Security Scans

Use additional security tools beyond Picklescan to detect malicious ML models on Hugging Face. Tools like:

• Static analysis scanners for detecting hidden payloads.
• Behavioral analysis tools to monitor execution patterns.

Conclusion: Strengthening ML Security Against Emerging Threats

The discovery of malicious ML models on Hugging Face leveraging broken Pickle files highlights the growing cybersecurity risks in AI. Attackers are continuously developing new evasion techniques, such as nullifAI, to bypass existing security tools.

To stay ahead of these threats, ML developers and organizations must adopt stronger security practices, including alternative serialization methods, sandboxing, and model verification. As AI continues to advance, ensuring a secure and trustworthy ML ecosystem should be a top priority.

Stay informed & protect your AI models! Share your thoughts or learn more about securing ML pipelines.