Please offer me some suggestions for posting XML, HTML, and other markup in my blog posts. As I found out in my previous post, Blogger is really stupid when trying to interpret any markup in the post. My intention is that the markup isn't interpreted by the browser, but instead shown as code to the reader (see the build.xml excerpts below). In Blogger's editor, I pasted the xml, and used the WYSIWYG editor to format it. The results were devastating. Both the xml became mangled, and the resulting html that Blogger generated to adjust the format of the text wasn't correct. I ultimately had to change to the "Edit Html" view and change all the > and < characters to their encoded equivalents (ampersand-lt; and ampersand-gt;).
How do you get around this problem in your blog? I guess I could merely switch over to the "Edit Html" view full time, but then, what good is the WYSIWG editor?
Stay tuned for more on Java code coverage...
Saturday, January 12, 2008
Wednesday, January 9, 2008
Enough of Cobertura, introducing Emma - another Java code coverage tool
After fooling around with Cobertura for a couple nights, I got frustrated because I couldn't figure out how to filter out my unit tests being monitored for coverage. Specifically, I had written the Cobertura ant targets to execute my suite of unit tests and monitor the coverage of the code. When reporting the results, in addition to showing the which lines were covered from my application code, the report also showed which lines from the unit tests were covered as well. I tried a multitude of different ways to try and filter out certain patterns of filenames from being reported, but nothing seemed to work. I imagine I'll revisit Cobertura in the future, since I'd really like to get the results fed into a tool like QALab. At the moment, QALab's website doesn't indicate that it supports output from Emma, only Cobertura (but there seem to be new feature requests under development for supporting Emma's format).
Emma seems to be very similar to Cobertura under the covers. Both tools do analysis and insert some markers into the compiled Java bytecode in order to monitor which lines of code are called during the coverage period. Emma's website has a lot better documentation on getting a new user up and running. With my work, I'm not even concerning myself with the command line Java execution of my code (does anyone even do this anymore besides college students writing simple programs?). Instead I like to jump right to the documentation for integrating these tools with ant, primarily because most of my code already has ant build scripts written, and that's where I'd ultimately like to integrate these tools. Also, some developers may use Emma to monitor coverage of test cases they're executing by hand, specifically QA type test cases from which they're clicking through the application. I'm not worrying about this scenario either, I just want to focus on the unit testing coverage aspect. To get Emma monitoring code coverage of your unit test cases from ant, at a high level you need:
<!-- You'll need to define emma.home as a property pointing to the
directory where you extracted the Emma zip -->
<property name="emma.dir" value="${emma.home}/lib">
<path id="emma.lib">
<pathelement location="${emma.dir}/emma.jar">
<pathelement location="${emma.dir}/emma_ant.jar">
</path>
<taskdef classpathref="emma.lib" resource="emma_ant.properties">
The first couple lines just cleverly define some path stuff so we can reuse the variables elsewhere in our build.xml. The last line here is the interesting part, it tells ant that one of the jar files on the emma.lib path contains a file called emma_ant.properties, which contains information for tasks that can be used from ant. Troubleshooting point: If you later get an error regarding ant not being able to understand the custom emma tasks, then you most likely have a bug/typo in the above code snippet such that the Emma libraries aren't being referenced correctly.
Next we'll define a general on/off property for enabling Emma (if you understand ant, you'll later see that this really isn't necessary, but you'll need to adjust future targets accordingly)
<target name="emma">
<property name="emma.enabled" value="true"></property>
</target>
Next, we'll define the target for compiling our unit tests:
<target name="compiletests">
<javac destdir="web/WEB-INF/classes" source="1.5" target="1.5"
debug="true" deprecation="false" optimize="false" failonerror="true">
<src path="${test.dir}"></src>
<classpath refid="compile.classpath"></classpath>
</javac>
</target>
One thing you'll probably notice from this snippet is that I'm developing a web application. I really haven't variablized my ant script completely at this point, that will come later when I try to replicate the script on a separate machine. Nothing complicated, or even Emma specific, in this snippet, just compile my test cases, located in ${test.dir}
Next, we'll actually execute the unit test suite and monitor their coverage.
<target name="runtests" depends="compiletests">
<emmajava fork="yes" enabled="true" libclasspathref="emma.lib"
classname="junit.textui.TestRunner" taskname="junit" failonerror="true">
<arg value="mongo.AllMongoTests"/>
<classpath refid="compile.classpath">
<filter includes="mongo.*" />
</classpath>
</emmajava>
</target>
This target will run the custom emmajava ant task (loaded from the earlier taskdef statement) which executes my unit test suite (class is mongo.AllMongoTests), run a filter such that only classes under the mongo package are monitored for coverage (otherwise you'll see stuff like how many times you called the junit.framework.TestCase class, and who cares about that?), and finally output the results into a file called coverage.html in the same directory as the build.xml. Well the results don't end up exactly there, instead they end up in a subfolder to wherever build.xml is located called "coverage." Of course, you can customize this by just changing the value of the "outfile" attribute, and absolute paths are supported.
The html output of the tool is very pretty, although not as pretty as Cobertura (but it's just data). In my next post, I'll show an example of the output. Unfortunately, compiling and running ant all the time on your desktop gets a little slow. It's better designed for integrating and building your application.
In my next post I'll review EclEmma, a plugin for Eclipse which allows you to do all this within the context of your IDE with minimal configuration.
Emma seems to be very similar to Cobertura under the covers. Both tools do analysis and insert some markers into the compiled Java bytecode in order to monitor which lines of code are called during the coverage period. Emma's website has a lot better documentation on getting a new user up and running. With my work, I'm not even concerning myself with the command line Java execution of my code (does anyone even do this anymore besides college students writing simple programs?). Instead I like to jump right to the documentation for integrating these tools with ant, primarily because most of my code already has ant build scripts written, and that's where I'd ultimately like to integrate these tools. Also, some developers may use Emma to monitor coverage of test cases they're executing by hand, specifically QA type test cases from which they're clicking through the application. I'm not worrying about this scenario either, I just want to focus on the unit testing coverage aspect. To get Emma monitoring code coverage of your unit test cases from ant, at a high level you need:
- A JUnit suite that will execute all of your unit test cases
- An ant taskdef referencing the Emma jar file to load its custom targets
- A custom ant target for compiling your test cases
- A custom ant target for running your unit test cases while Emma monitors it
<!-- You'll need to define emma.home as a property pointing to the
directory where you extracted the Emma zip -->
<property name="emma.dir" value="${emma.home}/lib">
<path id="emma.lib">
<pathelement location="${emma.dir}/emma.jar">
<pathelement location="${emma.dir}/emma_ant.jar">
</path>
<taskdef classpathref="emma.lib" resource="emma_ant.properties">
The first couple lines just cleverly define some path stuff so we can reuse the variables elsewhere in our build.xml. The last line here is the interesting part, it tells ant that one of the jar files on the emma.lib path contains a file called emma_ant.properties, which contains information for tasks that can be used from ant. Troubleshooting point: If you later get an error regarding ant not being able to understand the custom emma tasks, then you most likely have a bug/typo in the above code snippet such that the Emma libraries aren't being referenced correctly.
Next we'll define a general on/off property for enabling Emma (if you understand ant, you'll later see that this really isn't necessary, but you'll need to adjust future targets accordingly)
<target name="emma">
<property name="emma.enabled" value="true"></property>
</target>
Next, we'll define the target for compiling our unit tests:
<target name="compiletests">
<javac destdir="web/WEB-INF/classes" source="1.5" target="1.5"
debug="true" deprecation="false" optimize="false" failonerror="true">
<src path="${test.dir}"></src>
<classpath refid="compile.classpath"></classpath>
</javac>
</target>
One thing you'll probably notice from this snippet is that I'm developing a web application. I really haven't variablized my ant script completely at this point, that will come later when I try to replicate the script on a separate machine. Nothing complicated, or even Emma specific, in this snippet, just compile my test cases, located in ${test.dir}
Next, we'll actually execute the unit test suite and monitor their coverage.
<target name="runtests" depends="compiletests">
<emmajava fork="yes" enabled="true" libclasspathref="emma.lib"
classname="junit.textui.TestRunner" taskname="junit" failonerror="true">
<arg value="mongo.AllMongoTests"/>
<classpath refid="compile.classpath">
<filter includes="mongo.*" />
</classpath>
</emmajava>
</target>
This target will run the custom emmajava ant task (loaded from the earlier taskdef statement) which executes my unit test suite (class is mongo.AllMongoTests), run a filter such that only classes under the mongo package are monitored for coverage (otherwise you'll see stuff like how many times you called the junit.framework.TestCase class, and who cares about that?), and finally output the results into a file called coverage.html in the same directory as the build.xml. Well the results don't end up exactly there, instead they end up in a subfolder to wherever build.xml is located called "coverage." Of course, you can customize this by just changing the value of the "outfile" attribute, and absolute paths are supported.
The html output of the tool is very pretty, although not as pretty as Cobertura (but it's just data). In my next post, I'll show an example of the output. Unfortunately, compiling and running ant all the time on your desktop gets a little slow. It's better designed for integrating and building your application.
In my next post I'll review EclEmma, a plugin for Eclipse which allows you to do all this within the context of your IDE with minimal configuration.
Sunday, January 6, 2008
Cobertura - A tool to help you measure your Java unit test code coverage
If you've ever written unit test cases for any program language, you know how important it is to ensure that your test cases execute each line of code. This is called code coverage. The more your unit test cases exercise the logic in your application code, the less likely it is that a bug slips through your unit test cases. On a large code base (common to most applications these days), it's hard to judge how effectively your coverage statistics. There are various tools to help quantitatively measure the coverage of your unit test cases. Such vendor tools that accomplish this are Parasoft's JTest, and Quest's JProbe. But who wants to buy software? I've been trying out this open source code coverage measurement tool called Cobertura. So far I've only been using the ant tasks in combination with HTML reports. The reports it produces are really nice and informative, even allowing you to drill down into each class such that you can view the source and actually see the number of times each line of code was called from your test suite. Alternatively, red highlighting will call those lines with no coverage to your attention. There's also a command line interface if you choose to execute it outside of your build. Check it out here:
http://cobertura.sourceforge.net/
Hopefully I get the initiative to post some examples of how to include it in your ant targets.
Other tools that have my attention at the moment, and could very possibly surface in a future blog post include:
http://cobertura.sourceforge.net/
Hopefully I get the initiative to post some examples of how to include it in your ant targets.
Other tools that have my attention at the moment, and could very possibly surface in a future blog post include:
- http://cruisecontrol.sourceforge.net CruiseControl and the whole continuous integration thing
- http://qalab.sourceforge.net/index.html QALabs' is a tool to show you trends over time of your code metrics
- http://www.prweb.com/releases/2007/12/prweb591322.htm Enterprise Mashups 2.0 Conference I'll be attending on February 1st
Subscribe to:
Posts (Atom)