CruiseControl is a continuous integration tool. When any project is listed with CruiseControl, it looks for source code changes in that project at regular intervals and runs various tools everytime the codebase is updated. These tools include build tools, unit test frameworks, coding standard checker, source code documentation generator, etc. Any tool that has anything to do with source code can be in this list as long as they fulfill one criteria - the ability to generate XML output. CruiseControl is written in Java and its focus is also on the Java world. So we find Ant, JUnit, PMD as part of CruiseControl. But by including our own tools, we can use CruiseControl on any kind of source code. The PHPUnderControl project already does this for PHP by integrating PHPUnit, PHPCodeSniffer, and PHPDoc with CruiseControl. Whatever the tool is, CruiseControl runs them on the codebase and presents the result over the web. If the reports indicate any serious problem with the source code then CruiseControl can even warn people over various transports like email, Jabber, etc. Here is a typical scenario - somebody in your team mistakenly commits code that does not even pass the syntax check! It is definitely a serious error. With CruiseControl in place, you quickly become aware of it and thus the quality of the code base never goes below a certain level. Can you remember ThunderCats? Liono? And that magical sword of his? It was called the "Eye of Thundera". One of the qualities of this sword was to shine (with some background noise of course :) when any kind of danger approached the ThunderCats. Now compare this with CruiseControl, the continuous integration tool. Yes, CruiseControl is your "Eye of Thundera" - the early warning tool.
Having such a great tool in your grip is certainly very cool. But how do we integrate other tools with CruiseControl and make us more cool? That's what I'll try to explain here.
When we open CruiseControl in our browser (usually at port 8080 of the machine where CruiseControl is running), we see the list of projects that are being controlled by CruiseControl.
Click any of these projects and we see the latest summary of all the reports for this project produced by all the tools under CruiseControl's command. The tools involved are listed in the horizontal menu at the header area.
Click any of the menu items and you will see a detailed report from that tool. Basically, that's all of CruiseControl, sans the alert part. When we add our tool in CruiseControl, it will also appears in that menu bar.
CruiseControl provides us with two types of reports. One is a summary of all the reports produced by all the tools. The other type is the individual detailed report from each tool. So if you are using three tools, you get a summary report and three detailed reports. All these reports are produced from the same source - the project log file. This log file is an XML file. You can find it inside the CRUISECONTROL_HOME/logs/PROJECT_NAME/ directory. Besides other informations, this log file contains the XML output from all the tools. XSLT is a great way to process XML files and XSLT produces both the summary and detailed reports. So if you are using three tools, you will require a total of six XSLT files - three each for summary and detailed report. If this is not clear then look at the following diagram -
So, if we want to add one more tool, we will also have to create a set of two XSLT files that will produce the summary and detailed reports from the output of our tool. We will also have to modify two existing JSP and XSLT files to fit our two XSLT files inside CruiseControl. Then the summary report will be shown in CruiseControl's project summary page and the detailed report on a page of its own.
Before going ahead, I must tell that you should at least know how to use CruiseControl for your own project. You also need to know XSLT. If XSLT is foreign to you then there is no harm in learning it now. I myself learned it while I was trying to figure out how to use CruiseControl for Drupal projects. Usually it takes a day or two to learn enough XSLT to extend new tool support in CruiseControl. I studied the book Java & XSLT to learn XSLT (Thanks Kola, for lending me the book). This book has two chapers which is a great introduction to XSLT. Even if you are not interested in Java, you may find this book useful, like I did. Besides, there are lots of other books on XSLT and any of these should do.
Here are some necessary information:
- Everything in this document is true to CruiseControl version 2.7.3.
- All JSP files we are going to add/modify are in the CRUISECONTROL_HOME/webapps/cruisecontrol/ directory.
- All XSLT file we are going to add/modify are in the CRUISECONTROL_HOME/webapps/cruisecontrol/xsl directory. These files have the ".xsl" extension.
- CruiseControl stores project log files in the CRUISECONTROL_HOME/logs/PROJECT_NAME/log*.xml files. The actual log file name consists of the date, time and sequence number. Here is an example - log20080913172829Lbuild.11.xml. The "11" before ".xml" is the sequence number.
- All new and modified files mentioned in this tutorial can be found inside the attached tarball (see below).
Now I will introduce a simple tool to further describe the process of integrating a tool with CruiseControl. This tool is a small shell script (liner.sh) that counts the number of lines in source code. Here is a sample run of liner.sh where it counts the number of lines in Java source code:
$ ./liner.sh source/ *.java
<line-counts> <file name="source/a.java" size="100" /> <file name="source/b.java" size="200" /> <file name="source/c.java" size="300" /> </line-counts>
This output will be added to the project log file everytime CruiseControl runs liner.sh on the project source code. What is now left is to produce the summary and detailed report from the log file. The summary we are going to create will tell us how many source files have been processed and the total number of lines in those files.
line-count.xsl file produces this summary.
The detailed report is produced by line-count-detail.xsl. This report will not only tell us the total line count, it will also tell us the line count of individual source files:
CruiseControl comes with a demo project of its own. It is called "connectFour". So even if you are not controlling any of your projects with CruiseControl, the "connectFour" project is here to play with. We can run liner.sh on "connectFour" and see the results. In fact the screenshots are all generated from a fresh installation of CruiseControl (with our extension loaded of course). To run liner.sh on connectFour and count the number of Java source files, open connectFour's build.xml file and add the following snippet:
<target name="line-count">
<exec executable="/home/ju/liner.sh"
output="${basedir}/target/test-results/line-count.xml" error="/dev/null" >
<arg line="${basedir}/src/ '*.java'"/>
</exec>
</target>
And then include "line-count" as a top level dependency:
<target name="all" depends="clean, compile, sleep, test, jar, line-count"/>
A sample build file (connectFour-build.xml) can be found inside the attachement.
Now I'll explain how I added these two XSLT files in CruiseControl. The summary page of CruiseControl is produced by buildresults.xsl. This XSLT file includes and executes every other XSLT file responsible for generating the summary report from each tool. So we need to include line-count.xsl into buildresults.xsl. The following snippet does it:
<xsl:import href="line-count.xsl"/>
Then inside the <xsl:template match="/"> element, we add the following:
<p><xsl:apply-templates select="$cruisecontrol.list" mode="line-count"/></p>
The detailed report from liner.sh can be reached by clicking the "Line Count" menu item:
This menu item is not present in a stock installation of CruiseControl. It is one of the extensions we are creating here. We create this menu item by including line-count.jsp in main.jsp. Notice that line-count.jsp is a JSP file and not an XSLT file. line-count.jsp is included in main.jsp using the following snippet:
<cruisecontrol:tab name="line-count" label="Line Count"> <%@ include file="line-count.jsp" %> </cruisecontrol:tab>
There are some similar "<cruisecontrol:tab>" elements in main.jsp, one each for every menu item. Find those first, and add this snippet after them.
We have the menu item now but not the detailed report that the menu item will lead us to. For that purpose, we will need one more XSLT file, line-count-detail.xsl. This line-count-detail.xsl file mines the project log file and prepares a detailed report. line-count-detail.xsl is called from line-count.jsp. Here is the content of line-count.jsp:
<%@ taglib uri="/WEB-INF/cruisecontrol-jsp11.tld" prefix="cruisecontrol"%> <cruisecontrol:xsl xslFile="/xsl/line-count-detail.xsl"/>
That finishes our attempt to integrate liner.sh in CruiseControl. Here are the steps once again:
- Create line-count.xsl and line-count-detail.xsl. These XSLT files produce the summary and detailed report from liner.sh's XML output respectively.
- Modify main.xsl to add line-count.jsp. This will create the "Line Count" menu item.
- Create line-count.jsp and then include line-count-detail.xsl in line-count.jsp. As a result, when anybody clicks the "Line Count" menu item, they will see the detailed report from liner.sh
- Include line-count.xsl in buildresults.xsl. This will add the summary report from liner.sh's run into the CruiseControl summary page
Now fireup CruiseControl and check the reports. If you only have the connectFour project, then press the "build" button to run all the tools. This is because we don't know when the source code for connectFour will be updated.
I have one regret that liner.sh is a Unix shell script. But the process to integrate a tool with CruiseControl is very similar in any platform.
From outside, CruiseControl might look like a complicated piece of software which it is not. Although CruiseControl's main parts are written in Java, it relies on XSLT for log file parsing. So by adding our own XSLT files, we can add support for our tools in CruiseControl. But then we also have to modify one JSP and one XSLT file and this is ugly IMHO. The absence of any callback or hook system leaves no alternative but to edit core files. Apart from this, I like the ease of extending the software using mostly XSLT. That way even Java newbies/haters can work with CruiseControl without complaining. XSLT is also a pretty neat way of working with XML files. All these makes the extension process quite straight forward.
If you get stuck somewhere trying to extend CruiseControl, don't give up. Try to run your XSLT files on the log file outside CruiseControl with the help of an XSLT processor like xsltproc or Saxon. That should tell you if anything is wrong with your XSLT files. Before the first trial run, you also don't need to finish everything related to summary and detailed reports. You can only add the summary generation extensions and start testing. Then you can add the menu item and if that works then proceed to include the detailed report generator XSLT file. All these makes testing the extension easier. If you are still stuck, then look at the existing XSLT files (e.g. pmd.xsl and pmd-details.xsl) and how they have been included in CruiseControl.
Note: In build.xml for connectFour, the output of liner.sh is redirected to "${basedir}/target/test-result/" . This directory is also mentioned in config.xml as the directory whose files are merged with the project log file. If you want to redirect your tool's output to a different directory, make sure it is also merged with the project log file.