Enough of Cobertura, introducing Emma - another Java code coverage tool

After fooling around with Cobertura for a couple nights, I got frustrated because I couldn't figure out how to filter out my unit tests being monitored for coverage. Specifically, I had written the Cobertura ant targets to execute my suite of unit tests and monitor the coverage of the code. When reporting the results, in addition to showing the which lines were covered from my application code, the report also showed which lines from the unit tests were covered as well. I tried a multitude of different ways to try and filter out certain patterns of filenames from being reported, but nothing seemed to work. I imagine I'll revisit Cobertura in the future, since I'd really like to get the results fed into a tool like QALab. At the moment, QALab's website doesn't indicate that it supports output from Emma, only Cobertura (but there seem to be new feature requests under development for supporting Emma's format).

Emma seems to be very similar to Cobertura under the covers. Both tools do analysis and insert some markers into the compiled Java bytecode in order to monitor which lines of code are called during the coverage period. Emma's website has a lot better documentation on getting a new user up and running. With my work, I'm not even concerning myself with the command line Java execution of my code (does anyone even do this anymore besides college students writing simple programs?). Instead I like to jump right to the documentation for integrating these tools with ant, primarily because most of my code already has ant build scripts written, and that's where I'd ultimately like to integrate these tools. Also, some developers may use Emma to monitor coverage of test cases they're executing by hand, specifically QA type test cases from which they're clicking through the application. I'm not worrying about this scenario either, I just want to focus on the unit testing coverage aspect. To get Emma monitoring code coverage of your unit test cases from ant, at a high level you need:

1. A JUnit suite that will execute all of your unit test cases
2. An ant taskdef referencing the Emma jar file to load its custom targets
3. A custom ant target for compiling your test cases
4. A custom ant target for running your unit test cases while Emma monitors it

The suite is all on you to write, so no need to cover that here. The taskdef for Emma looks like this:


<!-- You'll need to define emma.home as a property pointing to the
directory where you extracted the Emma zip -->
<property name="emma.dir" value="${emma.home}/lib">
<path id="emma.lib">
<pathelement location="${emma.dir}/emma.jar">
<pathelement location="${emma.dir}/emma_ant.jar">
</path>
<taskdef classpathref="emma.lib" resource="emma_ant.properties">


The first couple lines just cleverly define some path stuff so we can reuse the variables elsewhere in our build.xml. The last line here is the interesting part, it tells ant that one of the jar files on the emma.lib path contains a file called emma_ant.properties, which contains information for tasks that can be used from ant. Troubleshooting point: If you later get an error regarding ant not being able to understand the custom emma tasks, then you most likely have a bug/typo in the above code snippet such that the Emma libraries aren't being referenced correctly.

Next we'll define a general on/off property for enabling Emma (if you understand ant, you'll later see that this really isn't necessary, but you'll need to adjust future targets accordingly)


<target name="emma">
<property name="emma.enabled" value="true"></property>
</target>


Next, we'll define the target for compiling our unit tests:


<target name="compiletests">
<javac destdir="web/WEB-INF/classes" source="1.5" target="1.5"
debug="true" deprecation="false" optimize="false" failonerror="true">
<src path="${test.dir}"></src>
<classpath refid="compile.classpath"></classpath>
</javac>
</target>


One thing you'll probably notice from this snippet is that I'm developing a web application. I really haven't variablized my ant script completely at this point, that will come later when I try to replicate the script on a separate machine. Nothing complicated, or even Emma specific, in this snippet, just compile my test cases, located in ${test.dir}

Next, we'll actually execute the unit test suite and monitor their coverage.


<target name="runtests" depends="compiletests">
<emmajava fork="yes" enabled="true" libclasspathref="emma.lib"
classname="junit.textui.TestRunner" taskname="junit" failonerror="true">
<arg value="mongo.AllMongoTests"/>
<classpath refid="compile.classpath">
<filter includes="mongo.*" />
</classpath>
</emmajava>
</target>


This target will run the custom emmajava ant task (loaded from the earlier taskdef statement) which executes my unit test suite (class is mongo.AllMongoTests), run a filter such that only classes under the mongo package are monitored for coverage (otherwise you'll see stuff like how many times you called the junit.framework.TestCase class, and who cares about that?), and finally output the results into a file called coverage.html in the same directory as the build.xml. Well the results don't end up exactly there, instead they end up in a subfolder to wherever build.xml is located called "coverage." Of course, you can customize this by just changing the value of the "outfile" attribute, and absolute paths are supported.

The html output of the tool is very pretty, although not as pretty as Cobertura (but it's just data). In my next post, I'll show an example of the output. Unfortunately, compiling and running ant all the time on your desktop gets a little slow. It's better designed for integrating and building your application.

In my next post I'll review EclEmma, a plugin for Eclipse which allows you to do all this within the context of your IDE with minimal configuration.