Profiling Maven Tests

A few days ago we changed nexus integration tests to use an embedded version of nexus instead of using a forked VM. Before every test Nexus is started and then stopped again after the test completes. This is done once for each test class (Junit: @BeforeClass=start nexus, @AfterClass=stop nexus).

During the first run we noticed some problems.Each time Nexus started, VM would use an additional 45Mb of memory. Soon we realized that our embedder was not working right. Stopping Nexus was not removing it from memory.

Setting all fields to null after stopping helped somewhat. But now instead of 45Mb of leak we were getting 40Mb, which was still too much when we had more then 200 tests to run. It was time to profile our tests.

At Sonatype we use yourkit for profiling, so I will demonstrate how to use yourkit to profile tests executed by maven. You should be able to use the same principle with any other profile tool.

1 - First, download, get a license and install yourkit.
2 - Create a new profile to hold yourkit-related configuration:

<profiles>
...
<profile>
      <!-- http://www.yourkit.com/docs/80/help/agent.jsp -->
      <id>yourkit-profile</id>
<properties>
        <yourkit.home>C:\Arquivos de programas\YourKit Java Profiler 8.0.13</yourkit.home>
      </properties>
</profile>
  </profiles>

I put the yourkit.home in pom.xml, but you can have it in settings.xml as well.

3 - Add yourkit agent to surefire inside the profile:

...
<profile>
...
      <build>
<plugins>
<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
              <argLine>-agentpath:"${yourkit.home}\bin\win32\yjpagent.dll"</argLine>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

There are other ways to do this you can see here.

4 - Launch yourkit UI.

5 - Launch maven using yourkit profile:
$ mvn clean test -P yourkit-profile -Dyourkit.home={your yourkit installation home}

As soon as yourkit UI detects your application, it will begin collecting profiling data.

Optional, using the API

Regular profiling is not very useful to find memory leaks on unit tests. A much more efficient approach is to take memory snapshots programmatically. To do so, we need yourkit API.

On the same profile, add this dependency:

<profile>
...
      <dependencies>
        <dependency>
          <groupId>com.yourkit</groupId>
          <artifactId>yjp</artifactId>
          <version>8.0.13</version>
          <scope>system</scope>
          <systemPath>${yourkit.home}\lib\yjp.jar</systemPath>
        </dependency>
      </dependencies>
...
    </profile>

Note that you should not move yjp.jar to your local maven repository.

Now we can create a com.yourkit.api.Controller instance and take memory snapshos when we need to.

To avoid compilation problems during regular builds (when you are not profiling) I suggest doing that using reflection, like this:

    // profiling with yourkit, activate using -P youtkit-profile
    private static Object profiler;

    private static void startProfiler()
    {
        Class controllerClazz;
        try
        {
            controllerClazz = Class.forName( "com.yourkit.api.Controller" );
        }
        catch ( Exception e )
        {
            log.info( "Profiler not present" );
            return;
        }

        try
        {
            profiler = controllerClazz.newInstance();
            controllerClazz.getMethod( "captureMemorySnapshot" ).invoke( profiler );
        }
        catch ( Exception e )
        {
            fail( "Profiler was active, but failed due: " + e.getMessage() );
        }
    }

    private static void takeSnapshot()
    {
        if ( profiler != null )
        {
            try
            {
                profiler.getClass().getMethod( "forceGC" ).invoke( profiler );
                profiler.getClass().getMethod( "captureMemorySnapshot" ).invoke( profiler );
            }
            catch ( Exception e )
            {
                fail( "Profiler was active, but failed due: " + e.getMessage() );
            }
        }
    }

That way you won't need to change the code to run in non-profiling mode.

You can see the full example in place at nexus-test-harness.

If you are still reading and are curious what was causing the memory leak on Nexus tests, we discovered that jetty creates Shutdown Hook that can hold a reference to all servers (depends on how you configure it). That shutdown hook stops jetty instances when the VM is going down. So it was just a matter of removing my jetty instances from it when I manually stopped Nexus.