The goal of the Hudson book is to be the authoritative source of documentation for Hudson users. We expect it to be available soon. Stay tuned to this blog for updates on the availability of the Hudson book.

For the perspective of one of the key contributors to the project, see Manfred Moser’s recent blog post, “Hudson – Book, Changes, and Community.“

Since the [Hudson] project lives on github it is pretty easy to follow the usual clone, branch, hack, commit, pull request sort of flow and thanks to Maven the build is as easy as .. surprise .. mvn clean install.

Read more of Manfred’s blog on Simpligility.

Earlier this year Sonatype and Oracle sponsored a Hudson user survey to gather input from the community about the future of Hudson.  We want to thank everyone who participated (over 1200 people from a variety of industries) as this type of survey is extremely valuable to our product team.  The survey was a huge success in that regard as it clearly points out both the popularity of Hudson as well as some key areas for improvement.

One of the most striking findings is that while our earlier Software Development Infrastructure Survey showed that Hudson is the clear Java CI market leader (over 70% of those using CI), the vast majority consider “stability” and “performance” to be the most critical “features” that need to be added (see figure 1).  This feedback rings true to us as Hudson runs at the core of our development efforts, supporting our work on Apache Maven, Nexus, Guice, and other core open source and proprietary work.  The product works well, but we have identified and solved a number of bugs that can cause performance and stability problems in Hudson. This experience seems to match closely that of Hudson users, both as indicated by the survey and through direct discussions and feedback. That is why our development team, in cooperation with Oracle and the larger Hudson developer community,  has spent a lot of time working on performance and stability improvements and has donated those improvements back to the open source Hudson project (soon to be at the Eclipse Foundation) for all to benefit.

Some of the other findings from the survey were also quite interesting, including:

• Open source development tools are widely used by over 75% of the respondents. This illustrates the success of such tools as Hudson, Maven, and Nexus.
• As shown in figures 2 and 3, the number of Master and Slave servers used with Hudson varies widely, with the number roughly correlating to organization size.  Surprisingly though, quite a few large organizations are only utilizing a few servers and missing the performance gains that are possible with a large-scale master/slave architecture.
• Only 8 plugins out of 29  were considered to be important by a majority, led by Subversion and Maven.  This was a surprising result considering the huge importance of plugins to the Hudson community.  See figure 4 for the full list.

Finally, when asked what additional features either the respondent or their boss would like to see added to Hudson, many wrote that they’d like to have commercial support available for the product.  Sonatype, responding to this need, now offers Sonatype Pro for Hudson, a fully supporting version of Hudson backed by the same engineering team that’s been making improvements to the open source project. Find out more about this product at http://www.sonatype.com/Hudson.

You can view the full survey results here.

Figure 2. Master server usage by organization size

Figure 3. Slave server usage by organization size

Figure 4. % rating plug-ins as critical or important

Looking at the interested parties in the Hudson proposal it’s apparent that more resources than ever will be poured into the Hudson project. Oracle and Sonatype have been working diligently to add fundamental architectural improvements to Hudson — which has paved the way for a new stream of innovation. VMWare and Tasktop have also indicated that they will be providing additional development resources, and we’re keen to start collaborating with them.

Sonatype also hopes to attract more enterprise-class contributors by taking the lead and contributing our core Hudson innovations to Eclipse. This includes all of the Maven 3.x integration that we have created to date. We were originally only going to provide a portion of our Maven 3.x integration to the OSS community, but we are so excited about Hudson moving to Eclipse we want to stimulate community adoption and wider participation by providing the best Maven integration possible.

The Hudson proposal still needs to go through the 30-day review period within the Eclipse community, but we really think Hudson has found its new home.  The Eclipse Foundation is a highly respected organization, has proven to be a vendor neutral, and has fostered many successful projects.   Eclipse would be a great place for Hudson and Jenkins to reunite and now would be an ideal time.   It can only be a good thing for users and I sincerely hope that the Jenkins team will seriously consider this option.

Hudson plays a key role in Sonatype’s commercial product portfolio so we’re committed to making the project succeed at Eclipse. We will offer commercial support and value added functionality in our ‘Professional’ version of Hudson. We are planning to contribute all commercial work we’ve invested in thus far to the Hudson project but we have more commercial features in the pipeline. Our customers tell us that along with Apache Maven, Nexus, and m2eclipse, Hudson is a critical part of their software development infrastructure. Hudson will be successful at the Eclipse Foundation and Sonatype plans to take an active part in that success.

Sonatype has developed state-of-the-art support for Maven 3 focusing on creating seamless integration between the internals of Hudson CI and Maven.   Year of work on Maven 3 internals to support more efficient embedding along with our multi-year investment in bringing both JSR-330 support and GWT UI integration to Hudson CI  has produced a CI system that support Maven 3 like no other on the market.

If you’ve used Hudson and Maven together in the past, you might be familiar with the Maven 2 project type that provides users with the ability to define a Maven build that is aware of a project’s POM. While we understood the motivation for a Maven-specific project type, we took our Maven support in an entirely difference direction. We created a Maven build that can be used a part of a larger freestyle build, and the reason we did this was to get away for the idea that a Maven build can only include one step, one call to a series of goals.

Sonatype found this to be more of a limitation than an advantage.

Supporting Real-world Maven Builds

Why?  Consider an enterprise build – one that needs to prepare and perform a release.  In the scenario it is essential that the build process contain two separate calls to release:prepare and release:perform. It also might be essential to call out to supporting scripts or other parts of the build that might fall outside of the scope of the Maven build. In a complex build environment, Maven is often not the only build tool involved in a process. Instead of binding our users to a single Maven-specific project type, we implemented Maven support as a builder to make sure that Hudson Professional users had as much flexibility as they needed to create quality enterprise builds.

Creating a New Maven 3 Job If you are new to Hudson Professional and you are wondering how to create a new Maven 3 build, you should follow these simple steps:

1. Create a New Job
2. Select “Build a free-style Software Project”
3. Add a Build Step and select “Invoke” Maven.

Creating a New Maven 3 Project from a Git Repository

The following video demonstrates the process for creating a new Maven 3 project from a Git repository:

Creating a New Maven 3 Project from a Subversion Repository

The following video demonstrates the process for creating a new Maven 3 project from a Subversion repository:

Hudson Continuous Integration with Sonatype Professional webinar

Sonatype is hosting a free webinar on continuous integration. Register for this webinar today before spots fill up! Sonatype Professional is the only integrated suite created for Maven-centric builds. The suite includes Sonatype Certified & Integrated Distributions of: Maven, Hudson, Nexus Professional, Eclipse Integration, Developer Onboarding as well as support from the experts. The Sonatype-enhanced distribution of Hudson included in the suite is fully tested, supported and designed to meet the demands of mission-critical software development. Register to learn how Sonatype Professional empowers development teams to realize the promise of agile through continuous integration, while reducing project risk.

• Date: Tuesday, March 29, 2011
• Time: 9:00 am Eastern Daylight Time
• Duration: 30 minutes
• Presenter: Blaine Mincey, Sonatype Senior Systems Engineer

Register for this webinar today!

Register today before spots fill up

Hudson is a continuous integration (CI) server that helps you check-out source code, build and test your project, publish the results and communicate the results to team members. We’ve been working to make it easier to extend Hudson, to reduce the effort required to write a Hudson plugin, and to put in a new foundation for the next level of Hudson interoperability and performance. We’re excited about the work that has been done and the work yet to come. The benefits of these changes include better leveraging of industry standards, increased performance and stability, and tight integration with Maven 3 that provides greater visibility into running builds. Come to learn more about Hudson CI and future plans for the project.

Webinar details:

• Date: Tuesday, March 15, 2011
• Time: 10:30 am Eastern Daylight Time (New York, GMT-04:00)
• Presenter: Jason van Zyl, CTO & Founder of Sonatype
• Register here

These changes are now in the core of Hudson. Even though JSR330 can now be used by plugin authors these changes should, in no way, affect plugin authors using the existing API. Since this question came up on the mailing list, I’ll give a short description of how it works here. The JSR330 integration allows you to take advantage of JSR330, if you wish, by using an alternative plugin strategy. Our new plugin strategy interoperates with the existing, classic plugin strategy. Sonatype’s Hudson Professional distribution actually ships with a mixture of JSR330 plugins and classic plugins and we find this works quite well. We tried to make it easier to use new strategies for wiring up plugin, and Stuart McCulloch has offered this strategy on the Jenkins development list and it appears to have been absorbed as part of JENKINS-8897. Now that the absorbtion of our first proposal is complete we will move on to our next proposals. These proposals are a mixture of project infrastructure proposals and core Hudson proposals.

New infrastructure for plugin developers. Winston Prakash and I have been working on setting up a new infrastructure for Hudson plugin Developers where it’s very easy for them to develop, stage to Nexus OSS, and synchronize their plugins to Maven Central. As part of this work Sonatype is also working on making Hudson plugin development dead-simple with m2eclipse. Winston and I have started testing this new infrastructure with a few plugin developers over the last couple of days and it’s working out quite well.

Plugins update site generation from Maven Central. Once Hudson plugin developers have synchronized their plugins to Maven Central we want them to be made immediately available to Hudson users. I have been working on a plugin for Nexus that will listen for incoming changes to Hudson plugins and dynamically modify the JSON metadata required by the Hudson update manager. The information about available Hudson plugins will be made available from Maven Central as a set of REST services. The integration possibilities here are very interesting.

JAXRS-based REST API. Jeanfrancois Arcand has been working on adding support for the dynamic addition of resources in Jersey and adding support for a JSR330-based component provider. We want to be able to use JSR330 and we want Hudson plugins using JSR330 to be able to dynamically register their own REST resources. Once this work is done on Jersey we will be making the proposal for its inclusion to the Hudson core.

Modularizing the Hudson core using JSR330. Stuart McCulloch has started analyzing the Hudson core and finding ways to help us reduce the complexity by breaking it apart into distinct JSR330 components. This will be an ongoing process, but one we believe will help us with all other work we intend to do with Hudson.

Hudson Core Testing. There is simply no way that we can aggressively refactor the core and know the changes are not harmful without drastically increasing the amount of testing done. This will be another ongoing process and I believe the most important contribution we will be making.

We are moving more carefully and probably slower then we might like, but we feel that, in order to aggressively add features in the future, the testing infrastructure, development infrastructure, and core features need to be in place. All this work I’m talking about will likely take a release or two to get in place but once that is done we will be moving at a radically different pace.

Take a few minutes to fill out the Hudson community survey. To thank you for your time, survey respondents will be entered in a draw to win a 16GB 3G/WiFi Apple iPad.

Official Rules for the Sonatype Hudson Survey iPad drawing can be found here.

Working with the community, Oracle and Sonatype are each putting a number of full-time engineering resources on Hudson. The Hudson lead, Winston Prakash from Oracle, is highly skilled, very thoughtful, and he cares about the community. He is also the first person to create detailed, comprehensive architectural documentation.

This kind of documentation (which has never been available in the past) is required to understand how Hudson can be improved. The lack of architectural documentation, along with how decisions were made, left the Hudson community mostly dependent on a single individual for core changes.   Let’s be honest about where this led:

• The Hudson WAR distribution incorporates over 100 dependencies, a troubling number of which are forked.   As result, there is no easy way to incorporate improvements from those forked projects in Hudson. It’s also not very easy to know why they were forked in the first place.

• The core technology for the user interface in Hudson is Jelly. We used Jelly in Maven 1.0, but took it out more than five years ago for Maven 2.x because it is not maintained and very difficult to work with. Hudson is still dependent on Jelly, but there are many more standard, and flexible, options for the UI technology: choices like GWT, Vaadin, or JSF.

• Hudson has had minimal project infrastructure. There are extremely limited unit tests, and integration tests that leave a lot to be desired (e.g. they don’t run on recent versions of the Mac OS X). Weekly builds of Hudson without a sophisticated automated test infrastructure is just not wise. Winston and I will propose a plan to clean up some of the testing infrastructure as a first priority. We’re not in a rush to push out releases where the community has to act as QA. It’s great to get feedback from users but we think a better job can be done on the testing front to alleviate this burden on users. It’s not glamorous, it takes a lot of time, but it’s necessary if you value stability and quality.

• Little to no effort has been made to track the provenance of the code, and there are numerous licenses that are used throughout the codebase. The licensing issues can likely be resolved using different libraries, and doing some modularization, but the provenance and IP issues are of utmost importance when you care about downstream consumers. These issues are of little or no concern for SaaS providers, but critical for companies that need to deliver software used on-premise.

So, I’ve told our developers to stay focused on improving Hudson’s infrastructure and fundamentals. Here’s our current work log:

• Testing infrastructure, and bolster the unit and integration tests
• JSR330 support for plugins
• JAXRS support for web services
• Maven 3.x support (94% of Hudson installs also use the Maven plugin)
• Eclipse integration
• Netbeans integration
• Assessment of the current Hudson architecture
• JSR330 architecture work to modularize the Hudson core
• Using a standard view technology like GWT, Vaadin, or JSF
• Inspection of non-standard/forked dependencies and trying to re-align with current mainstream releases
• Isolating code with problematic licenses and trying to reduce the license footprint
• Using standard Maven plugins for creating ancillary Hudson distributions

I’ll describe all of this in greater detail in subsequent blog posts.

Very recently, Sonatype completed significant development in the evolution of Hudson’s core architecture. The benefits of these changes include better leveraging of industry standards, increased performance and stability, and tight integration with Maven 3 that provides greater visibility into running builds. We are continuing to add engineers to our Hudson team and are working hard with the Hudson community to move much of the work we’ve done here to Java.net.

Our work on Hudson is consistent with Sonatype’s long history of investment and community support. Our work with Maven at the Apache Software Foundation, with m2eclipse at the Eclipse Foundation, and with Nexus (our open source repository manager) all underscore our commitment to innovation for open source and commercial users alike.

Are we a commercial enterprise? Absolutely. However, our track record of open source innovation and community contribution speaks for itself. At Sonatype, we’ve always focused on the code, and that’s where we’re focused today with Hudson.