Over the past few years, I've interacted with hundreds of people about build tools and repository management. It continues to surprise me how many people don't realize where these artifacts come from. When you run a build and these JARs show up alongside all their dependencies, it's like magic to most people. If you know how it works, it's obvious to you that running a repository manager is the right thing to do. This post is a reminder to everyone using build tools that rely on Maven Central: take time to proxy Central with a repository manager.
"Wait, That's How Central Works?"
There’s something so automatic about dependency management in Maven that it often takes people a few months to understand exactly where those JAR files are coming from.
In an 8-hour Maven class, I get to dependencies in the third hour, and after describing Central, what it was before, how metadata is stored in a repository alongside binaries, transitive dependencies. It all falls into place, and people realize that this simple thing they've grown accustomed to is only easy because of a ten year effort to refine the model, the creation of a support structure for source forges at places like Oracle and Google, and constant investment in infrastructure.
On the one hand, it's a great success that Central is, for the most part, an invisible utility that supports developers. On the other hand, it's the kind of thing that people can start to take for granted easily.
For example, a few months ago, I spoke to someone who worked in an environment disconnected from the internet for security reasons. This individual talked about how limiting it was to manually download JARs from open source projects and assemble them in a project. His words were: "It's like programming Java again in 2001."
How Can You Help?
Imagine millions of developers spread all over the world: different time zones, different applications, but they all hit the same service: Central. Some regions have more developers than others, so we certainly see peaks in usage throughout the day. But in general, Central's serving thousands of files throughout the world at any time during the day.
Maybe someone just installed Maven for the first time, or maybe they blew away a local repository. With numbers like these, we see a world with a constant appetite for artifacts. It's not a problem for Central, and I’m not writing this because Central is falling down on the job. Central can handle it, but it certainly isn't the most efficient way to support millions of developers. It isn't a good use of network bandwidth, and it isn't a good use of energy to constantly cart around the same static JARs over and over again when the solution is so easy.
If everyone who used a build tool that interacted with Central adopted a repository manager such as Nexus, we'd have a faster, more responsive system. Central's maintainers would focus less on addressing the occasional runaway build and could spend more time and resources on increasing availability and functionality of this essential service.
Broken Builds
The other factor is that Maven builds only download releases once. It's not like these build tools are repeatedly returning to Central to download release artifacts over and over again.
Well, actually, that isn't true. We've seen some Hudson installations configured to delete a local repository before every build, placing a high load on Central. Imagine a build that downloads 50 MB of dependencies once every 5 minutes. That's one build consuming ~14 GB a day, never mind the time wasted downloading static artifacts.
While these broken builds are the exception, they still show up from time to time. Central can handle the load, but imagine 1000 of these broken builds running continuously, and you can see the challenge.
A Simple Reminder: Please Proxy Central
We're constantly watching the performance of the system and making sure it stays up and running for an entire world of developers. If you use a build tool that hits Central, whether it is Buildr or Maven or Gradle or Ivy, you can help us by running a Nexus instance.
Even if all your builds work perfectly, running a local Nexus instance helps preserve Central as a public, free resource, and leads to faster, more responsive builds.
Tim is a Software Architect with experience in all aspects of software development from project inception to developing scaleable production architectures for large-scale systems during critical, high-risk events such as Black Friday. He has helped many organizations ranging from small startups to ...
Explore All Posts by Tim OBrienTags
Try Nexus Repository Free Today
Sonatype Nexus Repository is the world’s most trusted artifact repository manager. Experience the difference and download Community Edition for free.