Sonatype Blog
Posts by Brian Fox

Sonatype makes it easy to add your projects to the Central Repository with a free, public hosting service called OSSRH. We first blogged about this back in 2009, but given the growth in the community, we thought some of you may not have seen that post, so we decided to update it. (more…)

We’ve made several improvements to the Central Repository (Maven Central) to support the incredible growth in both the number of components and the number of developers using it. If you use specific IPs to allow access to Central, you’ll need to update your firewall as described below.

Since 2007, Central has been hosted at Contegix in a shared rack with 100mbps data connections to the Internet. We’ve worked with Contegix to acquire a new dedicated switch that will have a 1gb connection directly to their core routers. The routing to the switches is done at the Layer 3 (IP) level and this means we are moving to a new dedicated ip subnet:

• 207.223.241.64/27 (207.223.241.65 – 207.223.241.95)

In addition to the network upgrade, we’ve added an entirely new tool to our belts: Dyn (formerly DynDNS.com) is partnering with us to provide active monitoring, failover and global load balancing along with enterprise DNS services for maven.org via their DynECT Managed DNS solution. DNS resolution time should be noticeably faster as Dyn has DNS servers all around the world.

(more…)

A few months ago we announced that the US Maven Central server had been moved over to a virtual system.

In the natural course of machine rotations, I had some out of warranty machines de-racked, packaged up and sent from the Contegix datacenter to our headquarters in Silver Spring, MD.

When I was unboxing them it dawned on me for the first time that I was laying eyes on a machine so many people have relied upon for years and yet had been so far unseen. Well here it is:

A few facts about Central during the time it was hosted on the machine you see here (3/2007 – 3/2011):

• Original configuration:
  • Dell PowerEdge 2950
  • 2 x E5310 Xeon 1.6ghz processors
  • 4gb 533Mhz RAM
  • 3 x 73gb SAS 15k Hard drives
• Artifacts requested over 12 billion times by 14.3 million unique IP addresses
• Repository size as of Jan, 2009: 60gb (this is the earliest confirmed size I can track down)
• Repository size as of today: 286gb
• Projected size next month with the addition of Java.net: 350gb
• This machine never had a hardware failure. In fact, even the original drives and RAM are still functioning perfectly.
• It was only rebooted / powercycled twice, once to add more RAM and once to add some bigger disks

It boggles my mind to think about how many applications both commercial and open source contain bits fetched from this singular machine. Now that we have 2 machines in the UK and 2 VMs floating across 6 hosts in the US, there can never be a single machine in the future we can gaze upon and say “that was Central.”

Sonatype is going through the archives and digging up articles that we think would be useful to developers using our tools. If you use Maven, keep reading the post below from Sonatype Vice President of Engineering Brian Fox on Maven best practices and how-tos.

We have a handful of Maven best practice and how-tos documented in the blogs. Over time they get buried by newer posts, but the content is still just as relevant. Learn more about Maven by checking out the following blogs:

(1) Why Putting Repositories in your pom is a bad idea

How to properly manage repository definitions and why you shouldn’t declare them in your poms.

(2) Best Practices for Releasing with 3rd Party Snapshot Dependencies

How to most effectively create your own internal release of a snapshot dependency.

(3) Maven Continuous Integration Best Practices

Multiple techniques for running Maven builds from a CI system.

(more…)

Spoiler Alert! This post contains information about a change to Maven Central’s IP addresses. If your network has firewall rules in place that need specific IPs, be sure to read this post.

We’re working hard and investing continued effort into making sure that Central is as available as possible. As Maven Central supports a world of developers, even a few minutes of downtime is completely unacceptable to us. In line with our previous efforts to make Maven Central as bullet-proof and available as possible, we are planning to make the US repository even more fault tolerant using a tool called Pacemaker. Once we’ve had time to evaluate the impact of the changes described in this post we will deploy similar measure for our European Union and Asia/Pacific mirrors.

As a follow up to our previous enhancements to central, we are planning to make the Maven Central US repository even more fault tolerant.

The US repository runs on two virtual machines (VMs) in a VMWare cluster with 4 physical nodes configured to use the High Availability support in VMWare. Despite having multiple levels of fault-tolerance, recovery from a misconfiguration or other catastrophic failure still requires a DNS update to a standby IP to restore Maven Central. This DNS change requires time: time to make the change and then an often unpredictable time for DNS changes to propagate over the entire Internet.

This is unacceptable to us. Millions of developers depend on Maven Central, we’ve invested in redundant virtual machines running on redundant physical hardware. If there is an unforeseen event, the problem should be addressed in a few seconds.

To achieve immediate failover in the event of failure we will be using a tool called Pacemaker to manage Maven Central’s floating IP cluster. Pacemaker monitors the repository IP address, Nginx process status, and sample content from Maven Central. If Pacemaker identifies a failure in any one of these components it will immediately failover to the backup machine. In my testing, this takes about 3-5 seconds to occur.

In a previous post I discussed the systems we have in place and how the IPs are configured:

We are aware that some users have firewall rules that are locked to the external service IP. Because of this, we strive to maintain a consistent IP for each system, however the primary mechanism for accessing the repository is by DNS for most users. At times, our failover escalation or maintenance procedures may require us to redirect the DNS for one system to another. For this reason, if you have firewall rules in place that need specific IPs, please allow this list so that you won’t be affected by any temporary transitions:

• 207.223.240.88 : US primary
• 207.223.240.92 : US staging / standby
• 89.167.251.252: UK Primary
• 89.167.251.253: UK standby

(more…)