Luke Kanies, the creator of Puppet, commented in his last entry about Open Source business models, specifically the idea of an Open Core and what that means. As an Open Source company do you have an open version of your product that’s crippled? Or do you an open version of your product that is truly useful? This was the crux of the questions I asked all the Sonatype CEO candidates, and this turned out to be the reason it took me almost 8 months interviewing 17 candidates to ferret out the right person. It was a grueling process finding Mark de Visser but I was adamant and our VCs, Hummer Winblad & Morgenthaler, were very patient and let me take my time to find the exact right match. I got pretty ornery at one point &emdash; I thought I would never find the right person in Silly Valley.

• The Open Source product you provide to users must be great: the Open Core should stand on its own as something truly useful without any additional commercial add-ons. The software must perform well in a production environment.
• The Open Source product you provide should go through an ungodly amount of testing and QA. Testing and QA on the Open Core are the cornerstone of quality and should not be reserved for commercial versions of your product.
• The Open Source product you provide should be architected such that all commercial features are plug-ins to the Open Core.
• The Open Source product you sell should have completely open pricing. If someone cannot clearly see what your pricing is and what the difference is between your open and commercial versions, you likely have a predatory and opportunistic pricing model.

At Sonatype with our first product, Nexus, I can say that I feel internally consistent about our process and our products. I’m satisfied that we have achieved the right balance between our Open Core and the commercial plugins. I feel internally consistent about the way we have participated as individuals in the community. While I’ve spent a decade contributing to open source software, I’m also aware that I occasionally need to eat.

Nexus’ Open Core

The Open Source version of Nexus is good and stands on its own. People can use it in production environments. We have an enormous amount of integration tests with coverage reporting. We have dedicated QA staff, and we’ll be taking the next step with help from Patrick Lightbody to setup completely automated, x-browser, Selenium testing in mid-March. We have a book on Nexus that is free. Being open and not hiding the online documentation behind registration has been a good thing for the community

We have a modular platform where the commercial features are a clear superset of the Nexus core. We have no special branches for the Nexus core for the commercial version. All of our QA and testing for the core happen in the open. Our commercial SCM contains nothing but plug-ins and our build simply drops those plug-ins into the core structure where they detected on startup and activated.

In a Nexus plugin core functionality can be added, UI features, REST services, and security capabilities. When a plugin is detected all of these capabilities contribute to well defined extension points in the Nexus core and are automatically wired in. We have no additional code for the core in the commercial version of Nexus. We don’t need to. We are still working through our APIs but users in the community have already contributed plug-ins (the first was a plugin to integrate Nexus with Atlassian’s Crowd product) and everyone will be able to extend Nexus in the same way Sonatype does. That does mean we have to make sure that we provide a lot of value in the commercial version and that’s fine with us.

Open Pricing Model

Our pricing model is also completely open. I think without question that Atlassian has this right. Atlassian is more like an Open Source company then most Open Source companies. If you show everyone the same thing you don’t have to remember the variations that are just going to get you in trouble. If you don’t have a clear pricing model driven by channels and inside sales you’re just dead as a company. The days of enterprise elephant hunting is over. Potential customers who start out as your Open Core users need to see exactly what they get and how much it costs. If they can make all the decisions by easily trying your commercial product and comparing features then you have a viable company. It’s all predicated on being truly open.

I want to share with you what Sonatype is planning to do with Hudson – I hope you will be interested.  We are planning a lot of work on the OSS side and will contribute that all back (provided the license of Hudson does not change to the CDDL). We are also planning to work on a commercially supported version of Hudson and we will create some additional commercial plugins. I think people here will be most interested in the OSS work so I’ll start there.

It all starts with the work we’ve done with Tom Huybrechts over the last few months to embed Plexus inside Hudson. This has several implications, especially for those who are interested in Maven integration. Tom made the PluginManager itself pluggable and the Plexus version of the PluginManager that was created finds Plexus components in its standard way. As a result plugins now work the same way in Hudson, Maven and Nexus.

From a Sonatype perspective we want to use the same system inside Maven, Nexus, Hudson, and Eclipse. Maven 2.x has used Plexus from the beginning, Nexus has used Plexus since its inception, the work Tom has done for Sonatype now allows us to write any Hudson extension using Plexus, and we are currently working on a Plexus/OSGi bridge as part of our work in m2eclipse so Plexus components will operate in OSGi runtimes as bundles.. We want to take a Plexus component and reuse it across all of these systems.

With the Plexus integration we have for Hudson we can now leverage any of the components that were created for Continuum (Continuum was probably the first Plexus application) and we want to create tight integration with Maven SCM (also a set of Plexus components) and the quality reporting system in Maven (also a set of Plexus components). We fully realize that changes will need to be made in Maven SCM and in the reporting/quality system in Maven to facilitate the integration. We already see changes that we must make in Maven 3.x and the underlying plugin and reporting APIs so that we can run efficiently in an OSGi-based environment like Eclipse. As an example of what we’re doing lets take Emma as an example.

We have started work to create Plexus components for Emma, we now have a reworked set of Maven Emma plugins, and we’ve attempted to bridge our work into Eclemma (which is currently the best Emma plugin for Eclipse). We now want to take the Plexus components and make it work for use inside Hudson. We want to be able to contribute to data collection and trending.  We have done all sorts of prototype work to create a new Maven API for quality metrics by moving away from a document production model to a data production model. I think it’s unfortunate that a very large number of Hudson plugins essentially duplicate many of the Maven plugins that exist because it wasn’t a huge change we made to collect data as opposed to producing a document.

In the same vein we will make any modifications necessary to the Maven SCM API to integrate better with Hudson and Eclipse team providers. When our Plexus/OSGi bridge work is done any component written in Plexus will be available inside an OSGi environment as a bundle. I don’t know how popular OSGi will be as runtime for applications as it has its great parts, but has some serious detractions as well. What is certain is that OSGi rules on the developer desktop with Eclipse so we have taken that into account and need to make sure the tools we produce function equally well in Maven and Eclipse. We also feel that Hudson rules in the arena of build automation and so we’re focusing on what we feel is an ideal set of tools consisting of Maven, Nexus, Hudson, and Eclipse and we are working on production quality integration between these four systems.

So the first thing we would like to contribute is the Plexus integration that Tom has worked on. This integration is will not work with the current form of Hudson because an old version of the Maven Embedder is used in the core of Hudson and it depends on a much older version of Plexus. It’s up to you guys but if you want the Plexus integration in your SVN I am happy to work on a branch helping to decouple the current form of Maven integration from the core so that both versions of Plexus can exist. This would allow anyone to write a Hudson plugin in much the same way they would write a Maven plugin.

We are also working on a new Maven job type and some new Hudson features which depend on Maven 3.x. So this work I wouldn’t recommend for public consumption yet, but as Maven 3.x reaches GA so will the Hudson integration we are working on. So we are also happy to contribute this if you are interested.  We would also like to work on integrating JSecurity and our REST framework (currently based on Restlet but moving toward Jersey). For anyone who is familiar with Nexus we want to integrate the security systems and we want to be able to create a UI for Hudson based on ExtJS. These parts we would also be happy to contribute if you are interested. This is something that we want at Sonatype and we fully realize this may not mesh with what you want. We’re not going to try and shove anything down anyone’s throats. We just want to be open about what we’re working on and the direction we’re going and if you guys want the code we’re happy to contribute.

Beyond that we will be working on some commercial extensions. We want to integrate a workflow system, create bullet proof release management that is integrated with Nexus, tools for the automatic provisioning of build nodes and custom data collections tools the creation of what we think is a pretty cool dashboard idea.

Happy to answer any questions, and if people want the work that we’ve slated for OSS we are more then happy to contribute it. We use Hudson on a daily basis and couldn’t live without it at this point so we feel it only fair to give something back.

Thanks,

Jason

Bechtolsheim helped turn Sun, Granite, Google, VMWare and now Arista into some of the most successful startups ever. You could pick worse role models than Andreas von Bechtolsheim, when it comes to starting a company.

The Sonatype I joined is a loose confederation of top open source developers, tackling really hard problems, validated by some of the largest enterprise software development teams in the world.  Jason Van Zyl and Brian Fox have managed to get this team focused, and they are delivering iterations on Maven, Nexus and M2Eclipse with better predictability than most more traditional teams I have worked with. And the users I talk to are excited about and keen to use these products.

So what should I do as the new CEO of this company? Many would choose to impose structure and bring in an executive team than will predict revenues, define corporate messaging, create multi-year product schedules, international expansion and more.

See what Bechtolsheim has to say in this New York Times article: “One mistake a lot of start-ups make with the encouragement of venture capitalists is to hire the whole management team upfront, You have a lot of people twiddling their thumbs and spending money.”

It resonates with me. Customers are already expressing interest, so rather than invest in aggressive marketing and sales, let me make it easy for these customers to transact with us. Rather than having product managers offering R&D their own biased opinions, let me further facilitate the conversation between users and our engineers. And rather than drive to an ‘industry standard’ percentage for R&D costs, G&A, Marketing and Sales, let me minimize all the none-core staffing.

A venture backed company like Sonatype needs to impress investors, and high-profile executives coming from top enterprise software companies are a standard way to do so. I suggest that the VC’s should be impressed instead by our focus on open source to drive product excellence and corporate communications.

I joined Sonatype on October 1st. I have found an enormous pool of talent, technology assets and – importantly – customer interest. Exciting stuff – nothing of what Sonatype delivers is a luxury item, it should play out well regardless of the economic climate in the months and years to come.

What is needed now is a public product roadmap and the ability of expressing and interpreting feedback. We’ll focus on that in the coming week.

In Maven and Eclipse strive for Visual Studio ‘power’, Gavin Clarke of The Register covers Sonatype’s bold moves to redefine excellence in the Java IDE market.

“A Maven plug-in for Eclipse called M2Eclipse is due in the next eight weeks. This will integrate the two environments, providing automatic mapping of assets from build to release, eliminating the potential for bugs to creep in to the handover between teams. Integration is currently tricky and done by hand. M2Eclipse will map repositories, project metadata, dependencies and configuration information of software built in Eclipse to the Maven project object model. Those building inside Eclipse will also be able to search projects and find plug-ins held in the Maven Central Repository. Maven [Central Repository] creator Jason van Zyl told El Reg the duo are striving to create a development environment as powerful as Visual Studio – only for Java.”

Read more of this article on The Register

• For more information about m2eclipse, see the m2eclipse site

I’ve had plenty of opportunities to manage releases of products in my career. As I mentioned in an earlier post, most of them were large mission-critical enterprise applications. Open source projects are different beasts entirely, but they don’t have to be. A little bit of Enterprise rigor combined with the vast resources of the community can reap great rewards.

Lately the Maven project has been taking a lot of heat from various sources about stability and over-all quality. For the most part, they were right. The Maven team is very strong and certainly no one intends for these problems to happen, but ultimately they were.

In a well run commercial project, you have a team of QA engineers dedicated to validating fixes and checking for regressions. Unfortunately no matter how hard you test and identify regressions, business pressures often force a release to occur before it’s truly ready.

In an OSS project, the team is typically made up of developers and not professional testers (or docs writers). As such, you tend to end up with lots of code and less than stellar testing coverage. However, if you expand the concept of the team a bit to include the user community at large (aka The Community), you can easily smoke the commercial equivalent for a couple of reasons:

• The user community is comprised of a nearly limitless number of expert users who are highly motivated to check for regressions
• Each user in the community brings a slightly different set of use cases to the testing effort
• The absence of business pressure for a release can be a blessing if harnessed correctly.

During a review of the open issues while planning 2.0.10, I became aware that a significant number of open issues start with “this used to work until 2.0.[x]“. I became frustrated and a little embarrassed to realize how bad of a regression problem we had going on.

At that point I decided that my personal priority for 2.0.9 had to be No more regressions.

The normal release process for Maven is to stage a release, email the dev list and wait for votes or show stopper issues to occur. The norm for most releases is 72 hours, but with Maven core releases it was common to let it bake for a week or more. Based on history, I was positive that the first few attempts wouldn’t make it through, so we started with a “pre vote” instead of a vote email.

It seemed that each “pre vote” staged release we posted for dev list testing showed yet another how come no one noticed that? regression. It became apparent that we needed more than ever to harness the power of the full community to squash these regressions. Since tossing out multiple versions all called “2.0.9″ to such a wide audience was clearly a bad idea, we started appending -RC[x] to distinguish them. Additionally, we needed to have a set of operating parameters to guide this broad level of testing, lest we have chaos in the flood of bug fix requests.

The gist of the operating parameters was:

• We won’t fix more issues unless it’s a demonstrated regression from the previous version. (2.0.8)
• We will fix all regressions identified unless fixing it is riskier than leaving it.
• All changes will be accompanied with a core Integration Test.
• Community participation will drive the quality of this release.
• We will continue this progress as long as it takes.

That last item relates specifically to the lack of business pressures to force a premature release. Being OSS allows us to harness the immense power of the community for as long as it takes to get the release right.

So far we’ve gone through 8 release candidates, and, as a team, we were able to correct every regression identified, something I wasn’t sure would be possible. The official release is now staged awaiting a last check and formal vote. If all goes well, the release will be out later this week.

The feedback received on the new process has been overwhelmingly positive and the level of testing by users was surprisingly high for the first try. A special thanks to all of you that tested and provided feedback, even if it was just confirming that everything looked good. Several users went through the effort to validate all eight of the RCs on corporate CI systems and that’s no small feat.

Going forward, I’m hoping to be able to put out more regular core releases and by holding the No regression mantra from the beginning, future releases should be easier. As promised the lessons learned from this release will be codified into a new formal release procedure. I’m currently toying with introducing the idea of milestone releases for regression testing should any future release go beyond a month or so since the last release.

Only time will tell if 2.0.9 stands out from all the predecessors in terms of quality, but it feels like we’re on the cusp of a new era. WDYT?