Last week, Sonatype discovered a dubious package 'vapi-client-bindings' published to the PyPI open source repository. The discovery was made by Sonatype's automated malware detection bots.
The package contained proof-of-concept (PoC) dependency confusion code that we have now repeatedly seen in tens of thousands of open source packages.
Further analysis uncovered that the package is named exactly identically to a private (internal) dependency that is used by VMware's VSphere automation SDK, prompting Sonatype to notify the PyPI registry to take down the PyPI package.
VMware has confirmed no impact to its products or users of the VSphere SDK dependency thus far.
A real VMware VSphere SDK dependency
On March 26th, Sonatype's automated malware detection bots flagged a suspicious Python package called: 'vapi-client-bindings' — the same day it was published to PyPI.
This package is recorded under the sonatype-2022-1754 identifier in our security research data.
Although at first glance the name 'vapi-client-bindings' may not raise alarms or reveal any obvious insight, it turns out it is a legitimate (internal) dependency used by VMware VSphere Automation SDK which exists on VMware's GitHub (but not PyPI).
And, time and time again, developers who've had trouble finding or working with this dependency, have raised issues on VMware's GitHub [1, 2]:
Naturally, if multiple issues have been raised on GitHub about an internal (private) dependency, it doesn't take long for a pentester, bug bounty hunter, or malicious actor to squat the dependency on a public open source registry as well.
Note: Shortly after the publication of this blog post and after PyPI took down the PoC attack version 3.7.0 of the package, VMware appears to have published a safe placeholder (empty) version 7.0.0 to prevent recurring attacks of this sort in the future. Customers interested in using the "vapi-client-bindings" dependency should obtain it from official VMware channels, such as their GitHub repository (if applicable).
Ethical hacker targets VMware
In this particular case, Ukraine-based ethical hacker and bug bounty hunter, Vladyslav Kotko aka 'kotko' published the 'vapi-client-bindings' on PyPI last week.
The inside of 'vapi-client-bindings' comprises simple code that uploads your IP address, hostname, working directory path, and basic system fingerprinting information to the researcher's domain, "kotko[.]me."
As we explained last year, "the dependency confusion problem is an inherent design flaw in the native installation tools and DevOps workflows that pull dependencies into your software supply chain."
In this context, dependency confusion refers to the inability of your development environment to distinguish between a private, internally-created present package in your software build, and a package by the same name available in a public software repository.
This means, should there exist a public package on an open source registry like npm, PyPI, or RubyGems that has the same name as one of your private dependencies, the public package may instead get pulled into your software build, if your development tools aren't configured correctly.
While it may be tempting to dismiss this attack as unrealistic, researcher Alex Birsan alone earned over $130,000 in bug bounties capitalizing on dependency confusion attack vector through ethical research, and thousands of copycat bug bounty hunters followed.
Ever since Birsan's grand success with the technique, several bug bounty enthusiasts have followed the researcher's lead and published thousands of dependency confusion packages to date, and continue to even today.
VMware confirms no impact to its products or users
After reporting the suspicious dependency to PyPI, Sonatype reached out to VMware and the ethical hacker (kotko) to better understand the impact of this dependency confusion attempt.
"Based on the results of our investigation, VMware products are not impacted by this dependency confusion package on pypi.org (i.e., vapi-client-bindings), and no users have reported impact," a VMware spokesperson told me in an email.
"VMware has successfully reclaimed the package by working with pypi.org."
The spokesperson further reassured that the package was removed from PyPI after being flagged for 'offensive content.' Note, the removal occurred following Sonatype's report of the package to the PyPI admins.
"VMware inspected that content and has determined that no attempt was made to exploit the environment of the user downloading the package. VMware was informed about the issue by the researcher who claimed the vapi-client-bindings package on pypi.org, and who we determined had no malicious intent."
Kotko also acknowledged his "proof of concept" ethical research to us after we reached out to him. The researcher said he had reported his PoC research to VMware via the company's HackerOne vulnerability disclosure program, but declined to share specific details related to this attempt.
"Unfortunately, a lot of large companies are subject to this. The projects are very large, which means there are a lot of dependencies. It's hard to keep track of everything," Kotko tells Sonatype.
Our automated malware detection bots flagged the suspicious PyPI package 'vapi-client-bindings' on Saturday, March 26th, keeping Sonatype Repository Firewall and Sonatype Lifecycle customers protected. However, our security research team, battling an ongoing influx of hundreds of suspicious packages, got to analyze 'vapi-client-bindings' on Tueday, March 29th. We reported the package on March 29th to the PyPI security team who took down the package within a few hours of our report.
Sonatype Repository Firewall and Sonatype Lifecycle users remain protected
At the beginning of last month, we reported on a sharp uptick in open source attacks after discovering a 'fix-crash' info-stealer and 500+ malicious npm packages. That was on top of the 400+ packages targeting Azure, Airbnb, and Uber developers discovered recently.
And as predicted, the attacks on open source registries are continuing to surge as the cybersecurity community from across the world is focused on battling the ongoing international crisis.
Users of Sonatype Repository Firewall can rest easy knowing that such malicious packages would automatically be blocked from reaching their development builds.
Sonatype Repository Firewall instances will automatically quarantine any suspicious components detected by our automated malware detection bots while a manual review by a researcher is in the works, thereby keeping your software supply chain protected from the start.
Sonatype's world-class security research data, combined with our automated malware detection technology safeguards your developers, customers, and software supply chain from infections.
Written by Ax Sharma
Ax is a Staff Security Researcher & Malware Analyst at Sonatype with a penchant for open source software. His works and expert analyses have frequently been featured by leading media outlets including the BBC. Ax's expertise lies in security vulnerability research, reverse engineering, and cybercrime investigations. He has a passion for educating a wide range of audiences through writing and vlogs.
Explore All Posts by Ax Sharma