I’ve seen my fair share of projects migrating from Ant to Maven, and, for a complex project, this migration path can take some time. You have to worry about dependency management, project structure, and retraining an existing team to use Maven and understand the core concepts behind the tool. When you make the shift, you are often affecting development infrastructure for an existing project, and you need to take into account development environments as well as developer’s ideas about how code should be organized and stored in source control. In this post, I’m going to discuss a common pattern I’ve seen in Ant to Maven migrations: how to migrate the monolithic project.

A Common Pattern: The Monolithic Project

Ant projects which have evolved over many years often lack modular structure. While it is certainly possible to create the equivalent of a multi-module Maven project in Ant, the usual progression in an Ant project is to store all of your source in a single tree and use extra targets to selectively compile different packages. This approach is shown in the following figure.

In the most extreme cases, there is a single project which contains an array of modules. Maybe you have an entire enterprise system all stored in a single project alongside a complex Ant build.xml file which contains a collection of tasks for each component. A monolithic Ant project usually produces a series of artifacts: a JAR containing some API for clients, a server-side web application, a utility library, etc. Developers have usually grown so accustomed to this approach that the idea of splitting up a complex system into a series of related submodules can see daunting.

Moving to Maven: Do we need all these projects?

One of the first questions I get in a Maven migration from Ant is whether it is really necessary to modularize a project. “Do we really need to create all these projects for Maven?” (Short answer: Yes, there is no avoiding this.) This is usually accompanied by a concern that Maven limits each project to producing a single artifact. These are two core assumptions of Maven: your projects will be modularized and each project should produce a single build artifact.

Now, while these are core assumptions, there are ways around them, and I’ve even seen people go to extreme lengths in an attempt to preserve this single, monolithic project. You can create an elaborate set of profiles to modify the build depending on the context in which it is run, and you can attach extra build artifacts to a project using assemblies. Even though Maven makes assumptions about code layout and project structure, it can do just about anything, and in this case, it can be used to approximate the monolithic, combined Ant project.

If you do this, if you try to trick Maven the first thing you’ll notice is that your monolithic project’s POM is going to be somewhat unwieldy: it is going to be massive. Instead of a simple, declarative picture of a project, you are going to have a POM that contains multiple assembly definitions. Each assembly definition is going to have to explicitly define the structure of your build artifacts and you are going to find yourself venturing into includes and excludes patterns for things like the Maven Compiler plugin. In other words, you are going to have to expend a huge amount of effort to bend Maven to your assumptions.

If you do this, you really won’t be using “Maven”. You’ll be using your own interpretation of Maven, and once you go down this road, you are going to start having problems using standard tools designed for Maven. In short, don’t do this. Don’t try to bypass Maven’s approach to modularity with assemblies and profiles. If you do, you are going to find yourself “swimming upstream”, and your first hint is going to be the size and complexity of your POM files. I’ve certainly created some complex POMs in my time, but I only need to do this for projects that really require customization because they are doing something unique (see the POMs for the Maven book builds, they are large and extremely customized).

If you are creating web applications, EARs, and simple Java applications you shouldn’t have large POMs. If you do, it is probably a sign that you have ventured too far “off the map”. While you might be building your project with “Maven”, there is likely something wrong with your approach. If you find yourself constantly challenging an assumption as basic as project modularity, then you need to either rethink using Maven entirely or rethink your project structure.

Adopting Maven Means Adopting Modular Structure

If you are going to adopt Maven, you must adopt a modular project structure. If you don’t you will be fighting an uphill battle with Maven and the tools that have been designed to work with it. You will be fighting not only with Maven, but you’ll be doing constant battle with your IDE and your repository manager.

If you are moving from Ant to Maven, do you yourself a favor. Adopt a modular project structure. Don’t approach Maven as a “toolbox” like Ant with tasks and targets. Approach Maven as a framework with expectations and assumptions. If you do this, you’ll have a much easier time adopting the tool.

In the previous post in this series I discussed three compelling ways in which a repository manager can benefit the development cycle. It proxies artifacts locally, it is optimized to store binary artifacts, and it facilitates a new level of collaboration and agility that isn’t possible when your SCM is only way for workgroups to collaborate. In this post, I’m going to talk about how a repository manager works in concert with a continuous integration server like Hudson or Bamboo.

First, the how, what, and when of a continuous integration server. Continuous integration (CI) servers are an established fact of of modern development infrastructure. It is a server which, for the most part, waits and watches. It keeps a vigilant eye on your source control system and jumps into action every time it sees a code change. When code changes, your CI system is usually configured to run the entire build, execute all of your unit and integration tests, and send out an email to every developer if it identifies a defect or a failed test.

It does this so that you will have an easier time identifying where a particular problem was introduced to the source code. If John checks in some bad code, the CI system runs the build immediately, and about 30 minutes later, everyone in the group receives an email with the subject header “John just broke the build”. It is a great way to identify errors, and it is also a great way to motivate developers to test locally before committing to a source control system as no one likes to be the reason for a build failure email.

Running a CI server is more than “just a good idea”. Once your system reaches a certain level of complexity you can’t scale a system without commiting to continuous integration and testing. If you don’t have continuous integration, you end up having to put all development on hold each time you want to perform a release. If you don’t build, test, and deploy your system on a regular basis – if it isn’t something that is well rehearsed, integration becomes a time consuming nightmare of manual testing and builds that often leads to inconsistent builds. This is especially true if your development effort spans multiple systems and multiple development workgroups. You run a CI system because building, testing, and deploying your system should be automatic: it should be as trivial as pressing a button.

The concept of a CI server is only slightly more established than a repository manager, and very often you will see that an organization has identified the need for a CI server before they’ve identified the need for a repository manager. If you are coding a complex system, there is a very good chance that you are already running a CI server. The most popular servers out there are Hudson, Bamboo, and CruiseControl. While the connection between CI servers and repository managers might not be immediately obvious, when used together they can introduce some new possibilities for the way you develop your systems.

Continuous Publishing

When you have a system to continuously build your code, you also have a system that can continuously publish SNAPSHOT artifacts to a repository manager to enable a more granular approach to development. What do I mean by “a more granular approach to development”? To answer that question, let’s take a look at a complex multi-module project using the example of the eCommerce group from the previous post in this series.

Assume you have a new programmer starting tomorrow. Instead of throwing him at the entire forty-thousand lines of code, you would like to be able to give that developer a small, easy to digest task. You want this developer to add support for PayPal’s Adaptive Payments API in your eCommerce system. That’s it. You don’t want them to be distracted by the overwhelming scope of the project, and you certainly can’t afford for them to take a three month voyage through your project’s code before they start contributing to the effort. Deadlines are tight, and you don’t have enough people on your team. It is important that new hires start programming as soon as they walk in the door.

Without a repository manager hooked up to a continuous integration server, if you try to checkout just the ecom-paypal project, the build is going to fail because it will try to download dependencies from a repository manager. In the case of the ecom-paypal project, assume that the dependency graph looks like this.

When you have a repository manager and a continuous integration server, you can configure your continuous integration server to publish SNAPSHOT artifacts (in-progress SNAPSHOT binaries) to your repository manager. This will allow you to just check out a single, isolated portion of a much larger multi-module project.

Without a repository manager, trying to build version 1.3-SNAPSHOT of ecom-paypal in isolation is going to generate errors because you are forced to checkout the entire codebase to build and install all of the dependencies in your local repository. With a repository manager, SNAPSHOT artifacts are being continuously published because Hudson is checking you SCM every few minutes and building the latest code. When you run the ecom-paypal module’s build in isolation, Maven is going to download the most recent SNAPSHOT.

Without a repository manager, your new developer is going to have to download the entire codebase and run a large time-consuming build. With a repository manager you can work on specific components of a larger multi-module project. This ability to divide and conquer your codebase comes in very handy when you need a consultant to take a look at a specific problem, or when you need to look at a coding problem in isolation.

When you continuously publish build artifacts to a repository manager, you move away from the single monolithic project build and toward a project layout and architecture that lends itself to modularization.

In tomorrow’s post: How a Repository Manager decouples deployments from source code, and what that means for developer operations.

Sonatype has spent the last few years investing in the Maven ecosystem. We’ve created solid documentation, we’ve invested heavily in making sure that m2eclipse provides solid IDE integration, and we’re about to unleash even more tools that will make it even easier for developers to develop and share software. Here’s an excerpt from Dave Johnson’s entry on converting the Roller project’s build to Maven:

I was a Maven hater and resisted it for a long time but over the years Maven has gotten much better, it’s well supported in IDEs and as far as I can tell, Maven has replaced Ant as the de facto build system for Java projects. If you want new developers be able to easily build, debug and run your code via command or their favorite IDE then Maven is the way to go, and that’s especially true for open source projects like Roller.  That’s why I spent a couple of weekends learning Maven and converting Roller’s build process from Ant to Maven (ROL-1849). The process of conversion wasn’t too difficult.

You can read more of Dave Johnson’s entry, or you can watch this short video showing the ease with which you can now obtain and build Roller.

Les Hazlewood has an objective summary of why he eventually came around to deciding that Maven is a better overall solution then Ant + Ivy. This is an evolution in thought process that we, Sonatype, often see in enterprises and Les has two blog entries that illustrate this evolution perfectly.

Maven 2 vs Ant+Ivy: Our selection process: This entry gives Les’ original perspective on Maven and the process by which he originally decided to choose Ant + Ivy.

Maven 2 vs Ant+Ivy: Revisited: This entry gives Les’ new perspective and why his enterprise team, and the Apache Shiro project, have chosen Maven.

I now firmly believe that Maven 2 is a better build and project management tool than Ant+Ivy. I was wrong.

Yep, I said it. I’m man enough to admit when I’ve made mistakes and that I’ve learned from my experiences. And this is coming from the guy that wrote a (still popular) OnJava article for Ant in the enterprise.

Overall, life with Maven is good. I’m glad that I was able to swallow my pride, really give it a chance, and in turn reap the benefits. I haven’t used Ant in over a year since switching, nor have I ever felt the need to go back.

We hope this perspective helps potential enterprises save time when looking for a build and release infrastructure.

ODFDOM, the OpenDocument Java API, has been released. You can read more about the project release here. From the site: “ODFDOM is an Apache 2 licensed Java library to easily create, access and manipulate ODF documents.” What’s interesting about this particular software release is the switch from Ant to Maven. Here’s the quote that caught my attention.

Last but not least, with the support by Benson Margulies, we were able to switch our build environment from ANT to Maven. By using Maven we made our dependencies declarative. We no longer add dependent JARs to our sources (e.g. the parser XercesImpl.jar), but are able to download them via Maven [Central] on demand. Following this idea of modularization, we were able to simplify the source structure of ODFDOM by moving our code-generation to a [new] project called relaxng2template.

Inspiration

In Mercury we made an attempt to bring some contemporary ideas into Maven, make Maven not a monolithic build system, but rather a lego-like construction set that allows users to create systems they need. One set of building blocks in particular – implementation of the idea that dependencies is a universal commodity, that exists outside of Maven world. And operation of resolving conflicts on those dependencies can apply to a broader range of dependencies, not just <dependencies/> tag in the POM file.

For example – it should be relatively easy to plug in an implementation to read OSGi bundle dependencies. Or read a .properties file with dependencies.

Or – my favorite – keep dependencies in a Drools-backed DSL file. I will try to create this implementation after we integrate Maven POM dependency reader into Mercury.

How is it done:

• Dependency trees and manipulated by DependencyTreeBuilder object
• DependencyTreeBuilder instance is created out of a collection of Repositories (to find Artifacts) and and instance of DependencyProcessor.
• DependencyProcessor abstraction is that externalization component for reading/processing depedencies, making the builder independent of actual dependency information storage

Example code

1
2
3
4
5
6

    DependencyTreeBuilder dtb
        = new DependencyTreeBuilder( null, null, null, reps, processor );
    MetadataTreeNode root
        = dtb.buildTree( new ArtifactBasicMetadata( "org.apache.maven:maven-core:2.0.9" ) );
    List cp
        = dtb.resolveConflicts( ArtifactScopeEnum.compile );

This example demonstrates full dependency resolution cycle:

• full tree is created in root variable
• compile classpath is stored in variable cp.

The two initialization steps required:

• create a collection of repositories in reps
• get an instance of DependencyProcessor into processor

Also interesting – resulting classpath is a list of artifact metadata objects, not artifact binaries. So in order to actually bring the binaries to local machine, we’ll need one more call – to VirtualRepositoryReader, which I will describe it in a separate posting.

What’s important here is the fact that operation of fetching binaries from remote repositories is separated from building dependency tree as well as from resolving conflicts. Which means that you can call all of those operations independently to construct a workflow you need for a particular application. All the metadata that was fetched in the process of creating dependency tree is cached by a RepositoryMetadataCache component, that complies with repository update policy.