Unpickling PyTorch

Picklescan, one of the community’s leading tools for scanning Python pickle files, relies on Python’s built-in ZipFile module to inspect serialized model contents. However, ZipFile can be brittle and refuse to open zip archives that deviate from strict formatting standards. PyTorch, by contrast, does not depend on ZipFile to load model archives. This difference allows attackers to craft models that intentionally fail ZipFile checks but still load successfully in PyTorch.

For example, attackers can introduce inconsistencies between the central directory and zip entry file names in the archive, causing ZipFile to crash while PyTorch proceeds unhindered. Another technique involves tampering with the zip file’s metadata flags. By manually setting bits that denote encryption or compression status, such as flagging an unencrypted file as encrypted, ZipFile misinterprets the file and fails to open it. PyTorch, on the other hand, ignores these malformed indicators and executes the file as normal. These inconsistencies create blind spots for scanning tools and highlight how attackers exploit mismatches between security analysis tooling and actual runtime behavior.

technillogue/waifu-diffusion

Downloaded approximately 300 times before being flagged, this malicious AI model demonstrated a creative and stealthy approach to remote code execution (RCE). The model was embedded with an obfuscated Python script using a ROT13 cipher (as seen in blue in Figure 1.1) — an old but effective method to mask plaintext. Hidden within the pickle file was a tarball payload.

During the unpickling process, the script was executed and proceeded to decipher a secondary Python script. This secondary script attempted to establish a connection to a remote server (a classic “call home” behavior), likely to receive further commands or payloads. It then extracted and executed a tarball embedded directly in the pickle file.

To distract the user or obscure the payload’s true purpose, the tarball launched a playable version of the classic video game “Doom” (pictured in pink in Figure 1.2) — a technique that blended novelty with deception. While the game ran visibly in the foreground, malicious code executed quietly in the background.

This particular model was able to evade static analysis due to the ROT13 encoding technique, which obscured its malicious logic from basic scanners. While ProtectAI has since flagged the model as dangerous, both picklescan and JFrog’s scanning tools continued to mark it as safe (pictured in Figure 1.3) at the time of discovery — highlighting the need for multi-layered analysis and continuous updates to detection heuristics.

This particular model was able to evade static analysis due to the ROT13 encoding technique, which obscured its malicious logic from basic scanners. While ProtectAI has since flagged the model as dangerous, both picklescan and JFrog’s scanning tools continued to mark it as safe (pictured in Figure 1.3) at the time of discovery — highlighting the need for multi-layered analysis and continuous updates to detection heuristics.

MustEr/gpt2-elite

With over 2,450 downloads, the MustEr/gpt2-elite PyTorch model represents a case of reconnaissance-style malware embedded within an AI artifact.

Rather than delivering an overtly destructive payload, the package attempted to quietly test system behavior — an increasingly common tactic in multi-stage or proof-of-concept attacks. What makes this example particularly notable is how it evaded detection.

The model’s payload used the Python runpy module to execute:

The use of runpy allowed the malware to fly under the radar of conventional scanners, as the module itself was not flagged as suspicious. In this instance, the malware triggered the system calculator (Calc.exe), suggesting either a benign proof-of-concept or a test to evaluate execution success without raising immediate alarm.

Further investigation revealed that this model was committed by a JFrog engineer as a demonstration of how easily malicious behavior can slip through Hugging Face’s pickle file vetting process. While not weaponized in a traditional sense, this case underscores the growing use of serialized models as reconnaissance tools and highlights the blurred line between benign research and exploitable risk.

The example also draws attention to the limitations of static analysis when facing obfuscated or seemingly innocuous code execution patterns — further reinforcing the need for runtime behavioral analysis and layered defenses.

wn3/gpt2

Downloaded 188 times, the wn3/gpt2 model provides a subtle yet effective example of DNS beaconing — a reconnaissance technique used by attackers to monitor when and where a compromised artifact is executed. Rather than executing an immediate payload, the model silently signals its activation to a remote domain.

Once loaded, the model triggers a DNS query to 2ca7281e.log.dnslog.sbs. This action occurs automatically during deserialization, using the following code snippet:

This code enables the attacker to receive a notification, including IP and timestamp, whenever the model is run in a new environment. This creates an opportunity for attackers to later deliver a tailored payload to the identified host, effectively turning the model into a passive first stage in a larger attack chain.

Unlike more aggressive payloads that draw attention, this form of beaconing is lightweight and stealthy, often evading detection unless DNS logs are closely monitored. The model’s low profile and apparent benignity allow it to persist undetected in environments lacking strict behavioral analysis.

As with previous examples, the wn3/gpt2 case underscores the importance of treating all serialized models as executable code. Even something as seemingly innocuous as a DNS query can act as a stepping stone in broader reconnaissance and exploitation efforts.

During Sonatype’s research into PyTorch pickle file security, four vulnerabilities were discovered in picklescan and responsibly disclosed to its maintainers. Each of these CVEs has been addressed in picklescan version 0.0.23:

These CVEs reinforce the importance of layered defenses and deep model inspection. Attackers continue to evolve techniques that exploit the differences between static analysis tools and the runtime behavior of ML frameworks like PyTorch. Tools must not only be continuously updated, but also capable of understanding adversarial misuse of serialization formats.

Beyond Sonatype’s findings, the broader security community has documented similar attacks, including the use of model files as droppers, miners, or backdoors — some discovered in community repositories such as Hugging Face and GitHub. These threats often exploit the implicit trust placed in open source artifacts and the lack of provenance controls.

Organizations should treat AI models as critical software components. Security teams must apply the same diligence to AI model ingestion as they do to traditional codebases, including routine vulnerability scanning, SBOM tracking, and policy enforcement across the lifecycle of model development and deployment.

The vulnerabilities uncovered in PyTorch’s use of pickle serialization are a wake-up call for enterprise AI practitioners. As AI adoption accelerates, so too does the sophistication of attackers seeking to exploit gaps in model provenance, serialization hygiene, and dependency oversight.

The key takeaway? Treat AI models as critical software artifacts. Unsafe formats like pickle should be avoided whenever possible, and when they are used, it must be within tightly controlled, sandboxed environments. AI model ingestion is not just a data science issue — it’s a software supply chain security issue.

By implementing safer serialization formats, enforcing strict model validation and provenance policies, and layering static and dynamic scanning tools throughout the AI pipeline, organizations can dramatically reduce their exposure to malicious code and ensure the integrity of their machine learning systems.