Mavenizing the AppEngine SDK

Today I attempted to Mavenize the AppEngine SDK and was more or less successful, but there are some problems which I'm not sure how to deal with if I this to work for everyone. I have JARs with names that are the same as JARs in Maven Central but have different contents (the checksums don't match), JARs which just don't exist in central, JARs which have Maven Repositories but have bad POMs where variables are not interpolated correctly, and some released projects that still have SNAPSHOT versions in them.

None of these things are terribly hard to fix, but I would like to do it once and publish something that's correct for Maven users so that every single person wanting to use Maven doesn't have to install all these specific JARs into their local repositories. There are already a couple of examples of people doing this and I can already see with the hodgepodge of libraries that it's just going cause a lot of problems. So I've pinged Greg Kick at Google and hopefully they will let me clean this up at the source and make it work for all Maven users.

I have some Java code that I use for analyzing libraries (and builds in general), but today I thought I would try the Nexus REST API and so I made something Ruby that takes the SHA1 of a file and sends it off to Nexus. Nothing fancy.

#!/usr/bin/ruby

require 'net/http'
require 'rexml/document'
require 'digest/sha1'
include REXML

checksum = Digest::SHA1.hexdigest(File.read(ARGV[0])).to_s
url = 'http://repository.sonatype.org/service/local/data_index?from=0&count=50&sha1='+checksum
begin
  resp = Net::HTTP.get_response( URI.parse( url ) )
  doc = REXML::Document.new( resp.body )
  groupId=XPath.first( doc, "//groupId" ).text
  artifactId=XPath.first( doc, "//artifactId" ).text
  version=XPath.first( doc, "//version" ).text
  result = "GOOD " + groupId + ":" + artifactId + ":" + version + "[" + ARGV[0] + "]"
rescue Exception => e
  result = "FAIL [" + ARGV[0] + "]"
end

puts result

I ran the script against each library shipped with the AppEngine SDK and these are the results that I found:

FAIL [./appengine-tools-api.jar]
FAIL [./impl/appengine-api-stubs.jar]
FAIL [./impl/appengine-api.jar]
FAIL [./impl/appengine-local-runtime.jar]
FAIL [./shared/appengine-local-runtime-shared.jar]
GOOD org.apache.geronimo.specs:geronimo-el_1.0_spec:1.0.1[./shared/geronimo-el_1.0_spec-1.0.1.jar]
GOOD org.apache.geronimo.specs:geronimo-jsp_2.1_spec:1.0.1[./shared/geronimo-jsp_2.1_spec-1.0.1.jar]
GOOD org.apache.geronimo.specs:geronimo-servlet_2.5_spec:1.2[./shared/geronimo-servlet_2.5_spec-1.2.jar]
FAIL [./shared/jsp/ant-1.6.5.jar]
FAIL [./shared/jsp/ant-launcher-1.6.5.jar]
FAIL [./shared/jsp/commons-el-1.0.jar]
FAIL [./shared/jsp/commons-logging-1.1.1.jar]
FAIL [./shared/jsp/jasper-compiler-5.0.28.jar]
FAIL [./shared/jsp/jasper-runtime-5.0.28.jar]
FAIL [./tools/jsp/jakarta-jstl-1.1.2.jar]
FAIL [./tools/jsp/jakarta-standard-1.1.2.jar]
FAIL [./tools/orm/asm-3.1.jar]
FAIL [./tools/orm/datanucleus-core-1.1.0.jar]
FAIL [./tools/orm/datanucleus-enhancer-1.1.0.jar]
FAIL [./tools/orm/datanucleus-jpa-1.1.0.jar]
GOOD org.apache.geronimo.specs:geronimo-jpa_3.0_spec:1.1.1[./tools/orm/geronimo-jpa_3.0_spec-1.1.1.jar]
GOOD org.apache.geronimo.specs:geronimo-jta_1.1_spec:1.1.1[./tools/orm/geronimo-jta_1.1_spec-1.1.1.jar]
FAIL [./tools/orm/jdo2-api-2.3-SNAPSHOT.jar]
FAIL [./user/appengine-api-1.0-sdk-1.2.0.jar]
FAIL [./user/orm/datanucleus-appengine-1.0.0.final.jar]
FAIL [./user/orm/datanucleus-core-1.1.0.jar]
FAIL [./user/orm/datanucleus-jpa-1.1.0.jar]
GOOD org.apache.geronimo.specs:geronimo-jpa_3.0_spec:1.1.1[./user/orm/geronimo-jpa_3.0_spec-1.1.1.jar]
GOOD org.apache.geronimo.specs:geronimo-jta_1.1_spec:1.1.1[./user/orm/geronimo-jta_1.1_spec-1.1.1.jar]
FAIL [./user/orm/jdo2-api-2.3-SNAPSHOT.jar]

There are many things I know exist but the checksums don't match. I only looked at one JAR which was the ant-launcher JAR where the Locator.class was altered slightly but the change was not indicated in the naming of the JAR. Then there's the things just missing from central. Again, not a big deal to fix.

So I have a couple of the demos converted, made some stack POMs (POM which aggregate all the dependencies for a persistence app for example), and an archetype. So if I can push all my changes into central without upsetting anyone at Google then hopefully by the end of the week I can release the Mavenized AppEngine SDK that I've put together.

If I clean this up once and the libraries are tracked then it would literally take 5 seconds to create the Maven bits. I used the same approach that we used to Mavenize the Flex SDK where we have a descriptor that describes the contents of an archive and how they are supposed to be laid out in a Maven repository. The result can either be installed locally or deployed to a remote Maven repository. You can add POMs and augment the dependency information with what you know. Here's an excerpt of what the file looks like:

<mapping>
  <version>1.0</version>
  <organization>
    <name>Sonatype Inc.</name>
    <url>http://www.sonatype.com</url>
<license>ASL 2.0</license>
  </organization>
  <defaults>
    <groupId>com.google.appengine</groupId>
    <version>1.2.0</version>
  </defaults>
  <artifacts>
    <artifact>
      <artifactId>appengine-tools-api</artifactId>
      <location>appengine-java-sdk-1.2.0/lib/appengine-tools-api.jar</location>
    </artifact>
    <artifact>
      <artifactId>appengine-api-stubs</artifactId>
      <location>appengine-java-sdk-1.2.0/lib/impl/appengine-api-stubs.jar</location>
    </artifact>
    <artifact>
      <artifactId>appengine-api</artifactId>
      <location>appengine-java-sdk-1.2.0/lib/impl/appengine-api.jar</location>
    </artifact>
    <artifact>
      <artifactId>appengine-local-runtime</artifactId>
      <location>appengine-java-sdk-1.2.0/lib/impl/appengine-local-runtime.jar</location>
    </artifact>
    <artifact>
      <artifactId>appengine-local-runtime-shared</artifactId>
      <location>appengine-java-sdk-1.2.0/lib/shared/appengine-local-runtime-shared.jar</location>
    </artifact>
    <artifact>
      <artifactId>appengine-api-sdk</artifactId>
      <location>appengine-java-sdk-1.2.0/lib/user/appengine-api-1.0-sdk-1.2.0.jar</location>
    </artifact>
  </artifacts>
</mapping>

So I'll keep plugging away here and hopefully I'll be able to get this out for Maven users by the end of the week!