Code coverage is a measure used to describe the degree to which the source code of a programme is tested by a particular test suite. The higher the percentage of code coverage, the more confident we are that the code has less bugs – giving us a basis to make decisions on whether the code can be released or not. For measuring test coverage on a JAVA/Groovy project, Clover is the go-to tool.
In this blog, we’ll walk you through how to use Clover along with how it integrates with Bamboo, allowing you to see the report of your test coverage on every build. The good news is that Clover is open source!
How to configure Clover
Clover can be installed on the eclipse and idea IDE’s, on build tools like Ant, Maven and Grails, and even on the command line. Here, we will focus on using Clover for Maven 2 and 3 since our code base uses Maven primarily. As Clover is now open source, you can find the repository here. We will use OpenClover, which allows you to install Clover on your pom.xml without needing to add a license. The configuration of the Clover Maven plugin is detailed in here.
For the purpose of this blog, we will use a JAVA open source 2D chart library called jfreechart. Clone the code and follow these steps:
Install Clover in your pom.xml
<project>
...
<build>
<!-- To use the plugin goals in your POM or parent POM -->
<plugins>
<plugin>
<groupId>org.openclover</groupId>
<artifactId>clover-maven-plugin</artifactId>
<version>4.2.0</version>
</plugin>
...
</plugins>
</build>
...
<!-- To use the report goals in your POM or parent POM -->
<reporting>
<plugins>
<plugin>
<groupId>org.openclover</groupId>
<artifactId>clover-maven-plugin</artifactId>
<version>4.2.0</version>
</plugin>
...
</plugins>
</reporting>
...
</project>
|
If you have cloned the jfreechart code above, my new pom.xml looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<name>JFreeChart</name>
<artifactId>jfreechart</artifactId>
<groupId>org.jfree</groupId>
<version>1.5.0-SNAPSHOT</version>
<packaging>jar</packaging>
<organization>
<name>JFree.org</name>
<url>http://www.jfree.org/</url>
</organization>
<inceptionYear>2001</inceptionYear>
<description>
JFreeChart is a class library, written in Java, for generating charts.
Utilising the Java2D APIs, it currently supports bar charts, pie charts,
line charts, XY-plots and time series plots.
</description>
<url>http://www.jfree.org/jfreechart/</url>
<issueManagement>
<url>https://github.com/jfree/jfreechart/issues</url>
<system>GitHub Issues</system>
</issueManagement>
<scm>
<connection>scm:git:git:https://github.com/jfree/jfreechart.git</connection>
<url>https://github.com/jfree/jfreechart</url>
</scm>
<developers>
<developer>
<name>David Gilbert</name>
<email>dave@jfree.org</email>
<organization>Object Refinery Limited</organization>
<organizationUrl>http://www.object-refinery.com</organizationUrl>
</developer>
</developers>
<licenses>
<license>
<name>GNU Lesser General Public Licence</name>
<url>http://www.gnu.org/licenses/lgpl.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<dependencies>
<!-- JFreeSVG enables SVG export -->
<dependency>
<groupId>org.jfree</groupId>
<artifactId>jfreesvg</artifactId>
<version>3.2</version>
</dependency>
<!-- OrsonPDF enables PDF export -->
<dependency>
<groupId>com.orsonpdf</groupId>
<artifactId>orsonpdf</artifactId>
<version>1.7</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
<build>
<testSourceDirectory>src/test/java</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>${project.source.level}</source>
<target>${project.target.level}</target>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.10</version>
<configuration>
<includes>
<include>**/*Test.java</include>
</includes>
<excludes>
<exclude>**/JFreeChartTestSuite.java</exclude>
<exclude>**/*PackageTests.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<configuration>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
<!-- clover -->
<plugin>
<groupId>org.openclover</groupId>
<artifactId>clover-maven-plugin</artifactId>
<version>4.2.0</version>
</plugin>
<!-- <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
</plugin> -->
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.2</version>
<extensions>true</extensions>
<configuration>
<serverId>ossrh</serverId>
<nexusUrl>https://oss.sonatype.org/</nexusUrl>
<autoReleaseAfterClose>false</autoReleaseAfterClose>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>animal-sniffer-maven-plugin</artifactId>
<version>1.7</version>
<configuration>
<signature>
<groupId>org.codehaus.mojo.signature</groupId>
<artifactId>java16</artifactId>
<version>1.0</version>
</signature>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.4</version>
</plugin>
</plugins>
</build>
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.8</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jxr-plugin</artifactId>
<version>2.3</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.5.1</version>
</plugin>
<plugin>
<groupId>org.openclover</groupId>
<artifactId>clover-maven-plugin</artifactId>
<version>4.2.0</version>
</plugin>
</plugins>
</reporting>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.source.level>1.6</project.source.level>
<project.target.level>1.6</project.target.level>
</properties>
<profiles>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<inherited>true</inherited>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
<executions>
<execution>
<id>attach-javadoc</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.2.1</version>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
|
Test that Clover works from the command line
This can be done by running the following command:
mvn clean clover:setup test clover:aggregate clover:clover |
clean: Will recompile all code sources, including the instrumentation done by Clover.
clover:setup To initialise Clover and instrument code sources. Care must be taken not to use this goal in code deployment to avoid releasing your instrumented code to production.
test: To compile, run and test code coverage.
clover:aggregate: To merge together children module Clover databases.
clover:clover: To generate the Clover report.
View the Report
At this point, if everything went on smoothly, check the ${build.directory}/clover/ directory and you will see Clover dbs for all the modules, including the children. You will also notice the folders src-instrumented and test-src-instrumented which are your instrumented source code and tests, respectively.
The reports are located in the ${build.directory}/site/clover directory. You can now open the index.html in your browser to have a view of the report.
Go through all the tabs and review the metrics. You can clearly see from the dashboard that we have all our tests passing with 54.9% of coverage. You can dig in further to see which classes or methods are not tested.
Integrate Clover with Bamboo
Now that we have established what code coverage is and how to instrument our code to provide us with the code coverage, the next step will be to integrate Clover with Bamboo so that we can have insights into our build results before making decision on deployment. This is a two-step process.
Enable clover
In the miscellaneous tab of your job, you can enable Clover by checking the option Use Clover to collect Code Coverage for this build. Our configuration will look like this:
Configure Builder Task
In the task definitions, we need to define a task that clones our Jfreechart code and then add a Maven task with goals (as was described in the previous section).
You must not add any deploy related goals, like deploy or release. This is to avoid deploying instrumented code. As a rule of thumb, create a dedicated test job with Clover integration.
Run build and view Clover report
After running the build, the same set of reports we had when trying on command line is viewable within Bamboo. A Clover tab will appear in the navigation.
Clicking on the Clover Report System will show the detailed report.
That’s a wrap!
We have shown you the benefits of using code coverage as a measurement of how well your team has implemented testing.
Caution however has to be taken, as 100% coverage does not necessarily translate to bug free code. As Martin Fowler stated in his blog Test Coverage, engineers might tend to write test cases for features that rarely fail or even make AssertionFreeTesting just to attain target set. The above issues can be mitigated in teams that use test driven development (TDD), where tests are designed with thoughtfulness. Another way is to ensure strict code reviews, where at least one approver must be a software test engineer.
Clover is open source and requires little effort to get it integrated to your code and build cycle. Click below to give it a try!
Try Clover
