Archive

Author Archive for john

The Hudson Build Farm Experience, Volume IV

In Progress: The Learning Curve We’re Still Climbing

Now that we’ve covered the high points of our Hudson build farm setup here at Sonatype, I want to discuss some of the current issues we’re facing at the moment. It’s important to realize that providing high-quality continuous integration is a long, involved process…not a quick, one-off event. Sure, you can get Hudson up and running fairly rapidly in a non-distributed environment. However, the path to distributed, multi-OS builds that capture a full range of testing can be very, very complex. In the end, if you can get by simply compensating for the problems I talked about in this series of posts, then you’re probably pretty lucky. Here at Sonatype, we’re certainly very conscious of the fact that our continuous integration setup could run more perfectly, and we continue to chip away at the list of things we’d like Hudson to verify automatically on our behalf. So, in the interests of full disclosure, I’m including a short wish list of items we’re currently working on. Read more…

 

How-To, Maven , , , ,

The Hudson Build Farm Experience, Volume III

I’ve been working on a Hudson-based build farm for Sonatype and Maven open source builds since sometime in September of 2008. I’ve learned a lot as a result, so I thought I’d share some experiences from the trenches. In this third – and probably, final – installment I’ll discuss some issues we tackled with our VMWare environment itself, and look ahead to some issues with which we still grapple on a day-to-day basis.

VMWare, Efficiency, and the Space-Time Continuum

Compared to what we went through trying to get Windows builds running reliably out on the build farm, this discussion is going to seem somewhat…nitpicky. However, there are some important things to understand when you’re running a build farm on VMWare ESXi, so let’s dive in and take a look.

Read more…

 

How-To, Maven , ,

The Hudson Build Farm Experience, Volume II

I’ve been working on a Hudson-based build farm for Sonatype and Maven open source builds since sometime in September of 2008. I’ve learned a lot as a result, so I thought I’d share some experiences from the trenches. In this second installment I’ll discuss a few more details related to remote maintenance, along with the hurdles we encountered integrating Windows into our Hudson farm (and the solutions we found).

Eyes and Ears: Getting Access

Having access to the build farm is critical for maintenance, but it can also be very important to developers who are debugging a failing build. In our build farm, we’re using various mechanisms to provide this access, largely based on what is best suited for a particular VM operating system. The basic requirement here is to provide “natural” browsing capabilities for the filesystem on each VM, along with the ability to upload files if necessary (this came in very handy for installing and testing FlashPlayer, for instance).

Read more…

 

How-To, Maven , ,

The Hudson Build Farm Experience, Volume I

I’ve been working on a Hudson-based build farm for Sonatype and Maven open source builds since sometime in September of 2008. I’ve learned a lot as a result, so I thought I’d share some experiences from the trenches. In this first installment, I’m only going to cover our goals and outline the basic setup of our farm; I’ll save discussion of specific hurdles and advantages offered by our environment for the next post.

Read more…

 

How-To, Maven , ,

Abstraction-Addicted Ant

June 29th, 2007 By john
Comments Off

I’ve been staring at quite a few Ant scripts lately, trying to find patterns that I can convert to Maven plugin configurations. The one thing that stands out apart from the general gunkiness of Ant scripts is that Ant – at least in its older versions – forces you to the extremes of abstraction for the sake of reuse.

Reuse in [legacy] Ant seems to be all about one of two approaches: using XML entities to “import” script snippets directly into your build, and calling Ant scripts from other Ant scripts. The former looks a lot like the JSP @include directive, where the scripts importing these fragments have to have a similar property structure and such if you want the resulting script to be sane. The latter forces you to enumerate exactly which properties you will push down to the delegate script, and which ones you’ll extract once it’s done its thing. All of this makes for a true nightmare, if you need to trace and learn a build script. Properties magically appear in some places, and are defined with no apparent purpose in others. Oh, and the names of these properties? They get boiled down the the least-common-denominator of meaning, so their names no longer give any clear indication of what they’re for or where they’re defined/used.

I’d honestly forgotten about this aspect of Ant. I know some will argue that the newer versions allow macros and such that avoid the need for magic/useless (depending on which way you look at them) properties. But the hard fact is there are millions of lines of legacy Ant script out there, and much of it doesn’t take full advantage of such elegant features. And, since I am doing a conversion here, why not convert to a system where the “macros” are binary artifacts maintained by a community, and where the entire configuration for the build is distilled into a single, easy-to-read file that’s (at least, usually) unpolluted with build commands? Why go on using ad-hoc measures to stitch together builds that are in themselves patchwork quilts consisting of numerous factored-out build-script snippets with a system that would seem to have no consistent, standard mechanism for accessing them in a distributed environment?

The consequences of such an environment are plain, and have been realized by many people. I’ve heard it referred to as the Inner Platform Effect, where you now have an entire software platform for running builds, and it needs the same care and feeding – maintenance, testing, architecture – that your business software needs. This is where Maven shines, because it provides a standard access mechanism to plenty of binary, third-party plugins that are (at least to some extent) tested. This means that, instead of importing or delegating to another script or macro in your build script, you supply only high-level configuration for prefabricated, standard, binary build steps to consume. No code, just configuration.

Looking at this stuff again, I’m beginning to realize how cushy my life in Maven world has been.

Technorati Tags: , ,

 

Uncategorized