Jenkins Defends Split from Oracle's Hudson
Published: March 02, 2011 10:18
After my discussion with Oracle Senior Vice President and Chief Architect Ted Farrell about Oracle’s perceptions of the Hudson/Jenkins split was posted last week, it became apparent that not everyone was entirely willing to let the matter lie.
This was made evident when Andrew Bayer of the Jenkins project contacted me to clarify Oracle’s comments from the Jenkins point of view. Bayer wasn’t upset in any way, but after hearing Oracle and Sonatype executives accuse the Jenkins team of pretty much setting out to fork their project away from the mainline Hudson project no matter what Oracle said or did, the Java developer requested to discuss the Jenkins position.
For those of you who haven’t followed the story thus far:
The Jenkins fork from Hudson, a continuous integration server for Java development, started back in the Fall of 2010 when Hudson developers, frustrated with the performance of hosting their project on the Java.net infrastructure, decided to migrate the project to GitHub. The move came after a miscommunication about a planned internal migration from older Java.net resources to Java.net’s Kenai system left Hudson developers unexpectedly locked out of Java.net and their code.
When they discovered that their access to the Hudson source code was suddenly blocked for no apparent reason, the Hudson development team was upset. Eventually, miscommunication was discovered, but not before Hudson founder Kohsuke Kawaguchi put forth the proposal that since the mailing lists were already being migrated, and with yet another problem with Java.net, why not just finish the move and get the source code off Java.net and onto GitHub?
Hearing no major objections from the rest of the Hudson community to Kawaguchi’s proposal, the Hudson team made plans to switch their code repositories over to GitHub on November 30.
But the Hudson code initially remained on the Java.net servers, because Farrell requested that Hudson needed to stay on Java.net for the sake of the larger Hudson user community, which hadn’t been heard from yet about a move to GitHub. Farrell also stated that Hudson should stay on Java.net, and that any move to host it elsewhere would be considered a fork.
When Hudson itself actually moved to GitHub recently, it seemed hugely ironic, since most people considered the Jenkins move to GitHub the incident that started the split in the first place. Last week, Farrell had clarified that Hudson moving to GitHub was never Oracle problem.
"That was a misrepresentation of statements I made which caused a lot of confusion. I had asked to hold off on the github move until we could coordinate with more of the community. I clarified multiple times in later posts that Oracle was ’in favor of moving to a git-based repository, including possibly github, and we just wanted some time to evaluate what that means and the best way to achieve it.’," Farrell said.
So, I put the question straight to Bayer: why did the now-Jenkins team move to GitHub and Google Groups in November 2010 without waiting for Oracle to make their case against the move, which, according to Farrell, was all Oracle wanted to do?
"When the Java.net outage/migration started, the Hudson community had no warning. As it turned out, this was basically due to bad luck--the mail sent to Kohsuke to notify him of the move bounced (I think they were going to a defunct email address, but I don’t remember exactly) and no one else was sent any notification. So we, the developers, had no idea what was going on, and were told it’d be days before source control and mailing lists at java.net came back online (which in fact did turn out to be the case)," Bayer wrote. "From our perspective, we’d suddenly lost our communications and source control, so we moved quickly to make sure we had a way for the community to communicate with each other by setting up the Google Groups. We also needed to get a release out the door that week, so we opted to use the existing GitHub mirror of the Subversion source tree for Hudson core, knowing we could then sync back to SVN if/when the Java.net repositories came back online."
Bayer acknowledges that the tension between the future Jenkins team and Oracle was not based on accurate communication.
"The conflict that started over those moves was due to miscommunication and misunderstandings. Ted’s initial response to our moves to keep the project afloat in a confusing-at-best situation came across as abrasive to many of us, and from there, things just got worse for a while. Once we (Ted, me, Kohsuke, and others) actually spoke directly, the GitHub and Google Groups matters were put to bed--Ted was open to the community deciding where to have the mailing lists and source control, and we polled the community accordingly, resulting in the definitive moves to GitHub and Google Groups," Bayer stated in an e-mail to me last week.
Bayer himself supported Farrell’s assertion that the GitHub migration was never an Oracle concern.
"It’s not fair to Ted and Oracle to claim that they were against the move to GitHub--I chalk those problems up to the communication problems for both sides around the time of the Java.net migration," Bayer wrote.
The issue that both sides cite as irreconcilable was over the Hudson trademark. The Hudson community developers wanted Oracle to relinquish control, something which Oracle was unwilling to do. Why did the Jenkins team feel so strongly about it?
"The trademark was always a concern--it’s difficult for an open source project to be truly independent if a corporation owns its name. From the time of Kohsuke’s departure from Oracle until the Java.net migration, we, the Hudson community, didn’t hear much from Oracle. We knew that Winston had been moved to work on Hudson full time, but Ted’s assertions of Oracle’s authority over the project in posts during the Java.net migration drama were the first we’d heard of any intention of Oracle’s to exert any sort of control," Bayer told me. "Once tempers had cooled down and negotiations were underway between Kohsuke, myself and Sacha Labourey (CEO of CloudBees, brought into these talks in large part because Kohsuke and I felt we needed someone with more experience in this sort of situation than either of us had) and Oracle (Ted most notably), I felt it was important to get a guarantee that the Hudson project and community had rights to its own name going forward, so that we wouldn’t have to worry that a future architecture or infrastructure decision would aggravate Oracle and lead to them revoking the rights to the name."
Farrell and Sonatype’s Jason van Zyl informed me that Oracle did indeed offer the Hudson trademark, with the stipulation that anything called Hudson would have to come from the maintained Hudson core binaries. Bayer indicated that wasn’t enough.
"Oracle’s offer of use of the trademark in the context of the ’core binaries’ didn’t resolve this--who would determine what the core binaries contained? Shouldn’t that be the developers of the project?," he wrote. "I asked Ted and Oracle for a guarantee that the Hudson project would always have the right to call itself Hudson, even if it went in a direction Oracle did not approve of at some point in the future. Ted declined to provide this. Oracle wanted or needed to retain the right to decide what Hudson was, and the overwhelming majority of those community members who expressed an opinion on the matter agreed with me that this wasn’t sufficient."
That "overwhelming majority" is a characterization that both Farrell and van Zyl have sharply disputed. Given that only 214 (of 228) members of the original Hudson community voted to move Jenkins away, when about 1,300 members of the Hudson mailing list were actually eligible to vote on the move, both the Oracle and Sonatype executives don’t feel a real majority was represented. In that context, the 214 votes for creating Jenkins represented about 17 percent of the total Hudson community, still a small minority. Representing it as something larger, van Zyl said a few weeks ago, "was a bit disingenuous."
Bayer, strongly disputes this assertion.
"Yes, only 228 out of over a thousand eligible voters cast a vote, but it’s absurd to lump all the non-voters in with those in favor of the project going under Oracle’s control. If only 17 percent of the electorate voted to move on, well, then only one percent voted to go with Oracle," he wrote to me.
"This wasn’t a grand conspiracy to ditch Oracle--I negotiated in good faith, and very much wanted to reach an agreement that would guarantee the Hudson project its freedom and keep Oracle involved. This didn’t happen, and I think that’s a shame, but that’s what we’ve got to work with. Oracle and Sonatype are now taking their version of Hudson in a direction they think is best for their customers, and I wish them the best of luck. Jenkins will continue to be a community-driven project, with hundreds of plugins and contributors from around the world. I believe that’s the best future for the project, and so far, it seems that plugin developers and users agree," Bayer concluded.
Having watched this split unfold from start to finish, it seems a shame that neither side could reach a compromise with the other, because hearing each perspective of the discussion it doesn’t seem like the Hudson or Jenkins teams were being completely unreasonable. Could anything have prevented this fork? That’s something to wonder about, so hopefully such events can be mitigated in the future.