I was doing a bit of data analysis of the data that drives our Nexus Professional popularity results and I came across some statistics that show demand for Google Guava has been picking up over the last year. Our Top 10 list for general utilities contains the usual suspects. Libraries like Commons Lang and Commons Beanutils are predictably near the top of the list as are both log4j and slf4j. Not only are these the utilities you’d expect to see in almost every Java project, many of the dependencies you depend on also reference these libraries. This list is a list of utilities and projects you’d better be familiar with if you are programming in Java because you will undoubtedly encounter them.

Here is a list of the Top 10 Utilities from April 2012. Note how Google Guava jumped three places from #15 to #12 with a 2.5% increase in demand from March. While I don’t expect Google Guava to surpass the popularity of Apache Commons components any time soon, it will be interesting to see if Guava becomes a standard that challenges Commons Lang. Guava, like Apache Commons, is a collection of utilities and classes that supplement Java, while they have overlapping purposes, I tend to continue to have both on my classpath whenever I’m coding.

Caveat: I’m comparing utility libraries with the exception of JUnit. JUnit is downloaded automatically by a number of tools (tools that don’t appear to cache artifacts between instantiation). Because of this JUnit downloads are off the chart. If you average out the data, JUnit is being downloaded approximately once a second (across the entire month).

One of our customers asked me for a presentation deck making a simple case for bringing Nexus into a development environment: what are the broad stroke benefits of the repository from the perspective of the Enterprise? This video is that presentation, it doesn’t spend too much time enumerating a list of pro features. It focuses on the two core benefits: consuming OSS and internal sharing.

If you have five minutes and you are looking for something that might convince others in your organization, this video will be of use. Here’s the video “OSS and the Enterprise: How Nexus can Help” followed by a very brief summary:

A summary in four sentences:

• The Enterprise has shifted dramatically over the last decade and OSS has been a major force driving the evolution of Enterprise software development.
• As organizations have adopted OSS, developers have a new interest in consuming OSS and bring some of the practices of OSS in-house.
• Nexus was designed to make is easier for you to support your developer’s interest in OSS consumption while giving you the necessary controls.
• Better yet, Nexus allows you to adopt the same mechanism for collaboration that is used by OSS projects.

In other words, it isn’t just about software.

There are over 400,000 components in the Central repository including everything from servlet containers like Apache Tomcat to critical application infrastructure like Spring and Hibernate.    When you are designing an application or trying to update an application’s dependencies, how do you choose which component to use?

Here’s an example of a decision you may have to make in the next few months.    Assume you have the chance to use a newer version of Spring, evaluate Hibernate vs. iBatis, and adopt a new REST-friendly web framework.   For each of these new and updated components you are going to have to ask yourself three questions:

• Which version of the library has the largest “install base”?  It often doesn’t make sense to use the latest version of a component, especially if it is a major release.   If you are looking to reduce risk, don’t code on the “bleeding edge” of technology.  Use the most popular version of a component.
• Which version of the library is free of security vulnerabilities?  The only thing worse than getting hacked is realizing that you got hacked because you weren’t paying attention to known vulnerabilities.   If you are upgrading to a new version of a library, make sure it is secure.
• Which version of the library is compatible with your OSS license policy?

Nexus Professional 2.0.4 brings the answer to all three of these questions to the search results.  Here are the search results showing the results for tomcat-catalina.    We’ve combined popularity data from Central with security and licensing information.

Without the popularity data you might have just selected the latest version of the library, version 7.0.27 which has been available for 37d.   If I were selecting components for an application, I would likely stick with Tomcat 7.0.25 based on the relative popularity of the artifact alone.   7.0.25 is, far and away, the most popular artifact of this group.

Sonatype’s Nexus Professional is the only product that incorporates popularity data directly from Central.   If you are interested in using Nexus Professional to evaluate your dependencies, download a copy and start your trial today.

With nearly 19,000 active instances of Nexus around the world, we thought we would reach out and ask the community to share their Nexus stories and experiences with us. Over the last six weeks we held our Nexus Stories Contest and we were blown away by the tremendous response we received from around the globe.

Thank you to all of you who took the time to participate. We hope you are all enjoying your wicked awesome Nexus mugs! As promised, we will be sharing your Nexus stories on our blog over the coming weeks.

After the overwhelming response we received, we decided drawing one grand prize winner just wasn’t good enough (plus we love to take advantage of Jason every chance we get). So we decided to draw three grand prize winners!  Our three lucky winners will each be receiving one day of onsite consulting with Jason van Zyl. Our grand prize winners are:

• Alexis Morelle, Colombes, France
• Rueben Jimenez, San Francisco, CA
• Henry Hughes, Los Angeles, CA

Congratulations to our winners and thanks again to everyone who participated!

Stay tuned next week, when we will be sharing some of your stories on what life was like, before Nexus.

Sonatype is pleased to announce the release of Nexus OSS 2.0.4. Nexus 2.0.4 OSS is available and ready for download immediately. If you are new to Nexus, or if you are an existing user, go to http://www.sonatype.org/nexus/go, click on the download button and get started.

Nexus OSS 2.0.4: A Focus on Usability

One of the common complaints we hear from new users is that it isn’t immediately obvious how to interact with the tool. How do you connect Maven or other build tools to it? What are the basic benefits to gain from integrating Nexus into your development infrastructure?

If you are new to repository management, a single, Google-style search field isn’t enough. You are looking for answers. In this OSS release, we’ve added some contextual help to the Welcome page as well as some links to new and existing resources for Nexus users.

For years we’ve always tried to go out of our way to document the product: we have a free Nexus book and we maintain a knowledge base covering common tasks. We wanted to connect these resources with the initial Nexus experience and to do that we’ve listed three popular getting pages from our Knowledge Base:

• Configuring Maven to use Nexus
• Adding a New repository
• Proxying a remote repo in Nexus

If you find our Knowledge Base useful, we encourage you to leave comments on existing articles or sign-up and add a new article. We built our new support and knowledge base tools atop Zendesk and they complement our active community-oriented presence on GetSatisfaction.

Staying Connected to Nexus Updates

If you use Nexus OSS, we encourage you to sign-up for the Nexus Newsletter. This is a low-volume mailing list for product and security announcements and a monthly newsletter highlighting new resources for Nexus users. If you depend on Nexus, you should be on this list to be to the first to find out about relevant updates and security patches. To sign-up for this list, click on the first link listed under Stay Connected, or click here and click on “Sign-up for Nexus Newsletter”.

Other resources on the Welcome page of Nexus are Twitter and the Sonatype Blog. Sonatype’s Twitter feed (@sonatypecm) is a steady stream of relevant blog posts, links, and retweets from the community, and our blog is focused repository management.  Our goal with this release is to make sure that every user understands that even Nexus OSS comes with a rich set of community support resources.

Please let us know if you have any feedback, concerns, or requests about our new effort to “forward-deploy” our documentation and support for Nexus.

Nexus OSS 2.0.4 Release Notes

Bugs

• [NEXUS-5024] – Problem reporting doesn’t work through http proxy
• [NEXUS-5032] – XSS vulnerability in /artifact/maven/resolve REST endpoint

Security and License

• [NEXUS-5028] – update embedded eula
• [NEXUS-5031] – Upgrade to latest Jetty 7.x to solve known denial of service security vulnerabilities

Get a cool mug and a chance for a day with Jason

*All entries must be received by April 29, 2012

Tell us how Nexus has helped your development organization and you’ll be entered into a drawing for one day of onsite build and repository consulting with Jason van Zyl, Sonatype CTO and founder of the Apache Maven project. And everyone who enters will also get one of these wicked awesome Sonatype Nexus travel mugs.

It’s easy. Just copy and paste the questions below into an email and send your answers to sonatypestories@sonatype.com. We’ll publish your stories on our blog.

We’ll need your name, role, company and shipping address along with answers to the following questions:

1. Are you using Nexus OSS or Pro?
2. What was life like before Nexus?
3. How is your team using Nexus?
4. What value did you see after using Nexus?
5. What would you tell somebody considering using Nexus?
6. Is there anything else you would like to tell us?

Contest rules can be found here.

Here’s a message to nexus-user from Eric Kolotyluk an active Nexus user who just upgraded his instance of Nexus and sent this message to the Nexus Users mailing list:

OK, I had to move my Nexus to a different server.

1. I installed the latest version of Nexus Pro on the new server.
2. I copied over sonatype-work from the old server to the new server
3. I installed the license
4. It just works.

Great work Sonatype for continuing to improve the ease of administration.

Cheers, Eric

Sonatype invests an huge amount of effort in unit testing, integration test, and regression testing. Engineers are never pressured to sacrifice quality to meet a deadline, and our Engineering team goes out of their way to make sure that upgrades and installations are as straightforward as possible. It’s great to get feedback like this, especially when it contains one of the more famous Apple, Inc slogans: “It just works”.

While this user had a seamless upgrade experience, we still recommend that everyone installing and upgrading Nexus read both the Nexus 2.0 Release Notes and the Nexus Upgrade Instructions.

We’re often asked by customers to prove that Nexus can scale to meet the demands of thousands, and sometimes tens of thousands, of developers. Fortunately, we don’t have to stand up an expensive set of machines for a proof-of-concept as we have the world’s largest collection of active open source projects hosted on a single instance of Nexus Professional running at http://oss.sonatype.org. This instance isn’t just proof that Nexus Professional can scale, it serves as a public instance that you can model your own instance after.

If you are looking for an estimate of the hardware required to support your instance of Nexus, this post will detail the configuration and specifications of the Nexus OSS repository instance. This instance is the largest known deployment of a repository manager in active use.

Performance of Nexus OSSRH

Nexus OSSRH serves requests on the order of 1,400-2,500 requests per minute. What drives this level of activity? First, the instance serves as a snapshot repository for many open source projects. If you look at the list of projects hosted on OSSRH, it is a large list. As we examine the logs for oss.sonatype.org we regularly see thousands of unique IP address every day, and oss.sonatype.org is involved in a number of OSS project’s CI builds. This means that at any given time, OSSRH is supporting any number of simultaneous CI builds and over the course of a given day we’re serving artifacts to thousands of developers.

OSSRH approximates the performance characteristics required for the largest development efforts in the world: with multiple geographic locations, 24/7 uptime requirements, and very high performance standards. This service has to stay up. If OSSRH were to become unavailable, you would hear an immediate outcry from every affected OSS developer. Just choose a day and search for projects announcing that they’ve pushed artifacts to oss.sonatype.org on Twitter and you’ll see that every day has several critical releases.

When a customer asks us to prove that Nexus Professional scales, we don’t have to stop and setup a contrived performance test. We support this level of activity every single day. All we need to do is point them at OSSRH.

Nexus OSSRH Specifications

We’ve established that OSSRH is at the center of a large amount of active OSS development. It serves between 1400 and 2500 requests per minute, and it is a mission critical resource. It would be reasonable to expect that this service runs on a cluster of machines distributed throughout the world to minimize latency. Think again, this is a single VM with modest specifications running at Contegix and constantly monitored by New Relic.

Our standard setup for all managed forges is:

• 2 CPUs
• 3GB RAM
• 400GB disk (this is completely dependent on your repository contents)
• RHEL 5.6 x64 (Contegix, our managed hosting service, recommends using this OS)
• Java 1.6 x64 with 1GB Heap* (see correction below)
• The virtual disk is located on a SAN connected with iSCSI over 1GBE

If you are supporting a global-scale network of thousands of developers, the hardware cost for this Nexus instance is a “drop in the bucket”. The specifications for one instance of Nexus Professional running on a service like Amazon EC2 would easily fit on an m1.large instance with space to grow or a very modest VM. (The only thing you might spend on is the disk requirement. For OSSRH, we have a six-disk RAID 50 approach described below.)

Scaling Nexus: I/O Requirements, Network, and Disk

Under heavy load, increasing the number of CPUs and amount of RAM may help, but often the gating factor is either disk I/O or network. We do not recommend using NFS to mount a virtual disk for the working folder as many customers have had trouble with locking and corrupted indexes. iSCSI is working very well for us on oss.sonatype.org and it also works for many of our flagship customers.

Over the course of a day, the system typically needs to scale up in terms of network and IO. And, Nexus “sings” under heavy load because we have made numerous code-level optimizations to ensure that we’re making effective use of caching to reduce roundtrips to disk. For I/O performance, we recommend a redundant solution that maximizes disk spindles, while maintaining fault tolerance. We use RAID 50 in our SAN. A RAID 50 combines the straight block-level striping of RAID 0 with the distributed parity of RAID 5. It is a RAID 0 array striped across RAID 5 elements. This approach emphasizes both performance and extreme reliability, it requires at least 6 drives.

If you need scale, Try Nexus Pro

Sonatype designed Nexus to meet the demands of the OSS community from the beginning. We’ve been supporting global-scale OSS communities for years, and we’ve integrated the lessons learned from supporting active OSS development into Nexus Professional. If you need to scale, try Nexus Professional Today.

Correction from Mike Hansen: With 2.0 we upped that to 2GB, at least on OSSRH.  But that pretty much just provides some extra headroom…  Actually, IIRC, the reason we went to 2GB was because we were battling memory consumption with some repository indexes that had not been optimized (i.e. the index optimization task had not been run for a very long time).

Whenever I’m at a client I tend to ask, “Who decides what open source packages are acceptable?” Nine times out of 10, people will say something about an “Architecture” group. Maybe there’s a single architecture group that sets standards across the entire department, or, more often, there are several groups that offer a set of services that may overlap. The more moving parts in an IT department the less clear people are about who will be responsible for running Nexus.

Take one example: I encountered a company that had a central architecture group alongside another team that managed deployments. There was some confusion: who would manage Nexus. Eventually they decided that Nexus fell under the auspices of the Architecture group because this group was setting license policy and that’s why they were bringing it in in the first place. I had two takeaways from this:

• This is an entirely new problem that most organizations haven’t adapted to yet – having a comprehensive view of everything in your Development Infrastructure stack.
• While everyone was cordial, there was some tension. Some unknown set of overlapping responsibilities that wasn’t entirely development nor ops. I do think that while DevOps preaches harmony it may also encourage lack of clarity for who is responsible for running something like a repository manager.

…and this wasn’t a one-off situation. Take OSS governance as an example: it could be a function of Legal. At other times, it is performed by a group that works for the CTO. But, it’s still all over the map. In some cases you might have individual technical development groups that take the initiative. Even though no one is telling them to, they understand what’s coming.

When it comes to leadership, I do think there’s something of a leadership vacuum for development infrastructure…

…Enter the ALM Architect

I was recently checking out Tasktop Sync, if you’ve never heard of Tasktop Sync or Tasktop Dev you should take a look. If you’ve downloaded Eclipse you’ve probably been exposed to Mylyn. Tasktop Dev is a commercial version of the open source tool. Mik Kersten, TaskTop’s CEO has been at this for many years, his efforts have often overlapped with Sonatype’s as Tasktop and Sonatype are both very involved in the Eclipse Foundation. Kersten has started to use a new term “ALM Architect” as he did in the Eclipsecon 2012 keynote.

So, is the ALM Architect the missing leader in the IT department? Would the ALM Architect be the person ultimately responsible for directing decisions around continuous deployment, repository management? Maybe…

ALM? What do you mean by that?

First, let’s talk about ALM. You don’t hear us using the term ALM too often, but our tools and products fit very squarely into this area. From Wikipedia: “Application Lifecycle Management (ALM) is a continuous process of managing the life of an application through governance, development and maintenance.” So, yes, if there is to be an ALM architect, I think they make decisions about builds, CI servers, repository managers, etc.

I think it is the norm right now for most organizations to have as many approaches to ALM as there are development teams. Some corporations do consolidate all teams on a single technical standard, but, in most organizations, reality forces teams to adopt different approaches for different projects. Many corporations could benefit from centralizing the decision-making process for ALM “design” into a single group, and things would be more efficient if you could sit back and say, “Right, repository managers, that’s the responsibility of the ALM Architect”.

ALM is getting more and more complex over time…

While I think this job description has promise, I have yet to meet anyone that would call themselves an “ALM Architect”. What’s relevant about this term is that it captures the evolving scope of the job. Here’s what I mean…

A few years ago, you could have a build manager. They simply managed the build and set up a simple CI server. Maybe they also ran the SCM system. That all worked fine until the emergence of cloud-based approaches to development, continuous deployment, and an array of development infrastructure technologies that is increasingly both heterogeneous and highly-integrated.

Get that? Five years ago you used a fairly bland collection of technologies and everyone used the same tools. Today, there’s a new tool, language, SCM system, or development approach introduced in the stack every month. From Cloud-based frameworks to different build tools, you need to support a heterogeneous development infrastructure or people are going to be less productive. If that business unit needs to deploy an app to Heroku to meet a deadline even though it is against policy, well they might have independent decision authority to just go off the roadmap and make it happen. Likewise with GitHub, how many times did GitHub show up by surprise because one of the engineers just put it on a credit card?

Conclusion: I think ALM Architect is something you’ll hear more often because Development Infrastructure isn’t getting any simpler.

That’s what I think of when I think about an ALM Architect. Someone who is responsible for setting high-level strategy, recommending which technologies are used, and managing the ongoing task of adapting your development infrastructure to constant change. Do you have an ALM Architect on staff?

ReadWriteWeb’s Joe Brockmeier has an interesting piece analyzing OpenStack Essex, while this isn’t an exact overlap with the kind of analysis we’re working on for Insight and Nexus, it’s a view into the social and open source dynamics of a project.

Brockmeier’s article is a summary of some analysis that OpenStack contributor Mark McLoughlin assembled from commits and Gerrit code reviews. It’s a breakdown of activity by organization, as with many open source projects that have corporate involvement, there’s always one or two companies that tend to dominate the commit breakdown.

Where the article is a little off-base is in the assessment of community health, you can’t judge the “health” of an open source project by the mix of companies represented in a commit breakdown alone. It’s an interesting statistic, but there’s so much more to open source than code commits including documentation efforts, marketing spend by companies invested in a project, and financial support for essential efforts not directly related to code (legal, infrastructure, etc.). Open source isn’t about code alone, and while it is an ideal for open source projects with corporate involvement to have balance, this balance can shift over time.

As Sonatype comes to market with more tools focused on helping you make better decisions about the components you use, we’re going to focus first on “actionable” metrics like popularity and quality. While these sorts of metrics are interesting (and you can get them from Eclipse BTW, if you go to this page), I don’t think it makes sense to create arbitrary, commit-based assessments about the health of a community.

I don’t think it would ever make sense to have a “Warning Project dominated by one corporation” flag, because I predict you’d end up seeing this flag on just about every open source project out there. It’s a subjective measure, and while these numbers may suggest that Rackspace is dominating the codebase of OpenStack, they may also just suggest that Rackspace has created a solid framework for a maturing community.

Cloud as the New OSS Battleground: Permissive vs. Copyleft

These open source cloud platforms (OpenStack and Eucalyptus among others) are the current battleground for computing. There’s a massive amount of investment and attention being paid to cloud computing, and as a result, there’s also a large amount of investment being poured into sponsored open source development. With CloudStack recently moving to the Apache Software Foundation and Eucalyptus recently signing agreements with Amazon to ensure AWS compatibility, there are weekly strategic moves from companies like Oracle, RedHat, Citrix, Amazon, VMWare, IBM, Intel, Appfrog, EngineYard….. (I’m going to stop because I could fill up several pages).

One of the big stories over the last week was Eucalytpus’ embrace of GPL versus CloudStack’s embrace of the ASL. Whatever your views on open source and licensing, it’s important to note that OSS licensing issues are front and center with commentators writing reams of analysis about how Cloud will eventually turn into a battle between permissive licensing models and GPL licensing models.

One thing is certain, people are paying attention to licensing issues like they never have before. If you haven’t yet done so, you should start too. Downloading a copy of Nexus are start getting a sense of your OSS license usage.