In the first post of this series on building polyglot persistence apps on OpenShift I talked about how we can create MongoDB backed Java application using Spring MongoDB framework. In this post we will setup Jenkins continuous integration server for our application. This blog will go beyond the Jenkins setup and will cover how to install various plugins and come up with a Quality Dashboard which shows the code coverage and test statistics as shown below.In the first post of this series on building polyglot persistence apps on OpenShift I talked about how we can create MongoDB backed Java application using Spring MongoDB framework. In this post we will setup Jenkins continuous integration server for our application. This blog will go beyond the Jenkins setup and will cover how to install various plugins and come up with a Quality Dashboard which shows the code coverage and test statistics as shown below. The plugins that we will install in this blog are Green Balls plugin, Cobertura plugin, Dashboard View plugin.
What is Continuous Integration?
Continuous Integration is one of the practices of Extreme Programming where members of a team integrate and commit their code into the code repository every few hours. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development. For more information on Continuous Integration please refer to Martin Fowler article. It is a must read if you want to learn about continuous integration.
Continuous Integration in the Cloud
Setting up a continuous integration server for your application is an infrastructure task and most developers avoid the pain of setting up a CI server. They want to write code, that is what they love to do. Isn’t it? OpenShift make it very easy for developers to set up CI server for their application. It just requires a couple of rhc commands to setup a CI server for your project. The CI server that OpenShift supports is Jenkins , a popular open-source continuous integration server written in Java. The best thing about OpenShift is that it provides you all the tools which make it very easy for a developer to follow best practices and get their job done.
Source code of Notebook Application
- I added test cases in notebook-core and notebook-web modules. For quick testing, I have used in-memory Java implementation of MongoDB called Fongo and will talk about it in my future blog.
- Modified profile openshift in pom.xml to run the tests and generate code coverage report using cobertura maven plugin. Code coverage is a measure used in software testing. to describes the degree to which the source code has been tested. The full profile is shown below.
<profile> <id>openshift</id> <modules> <module>notebook-core</module> <module>notebook-web</module> </modules> <build> <finalName>notebook</finalName> <plugins> <plugin> <artifactId>maven-war-plugin</artifactId> <version>2.1.1</version> <configuration> <outputDirectory>deployments</outputDirectory> <warName>ROOT</warName> </configuration> </plugin> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>cobertura-maven-plugin</artifactId> <version>2.5.2</version> <configuration> <check> <totalLineRate>60</totalLineRate> </check> </configuration> <executions> <execution> <id>cobertura-check</id> <phase>verify</phase> <goals> <goal>cobertura</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </profile>
Lets get started
Before we setup jenkins for our application we need to first get the latest code from github and deploy the application to OpenShift. If this is the first time you are creating application on OpenShift, then please refer to the first part for information. Please execute the following commands to get the notebook application in cloud.
rhc app create notebook jbossas-7 -l <openshift_login> rhc cartridge add mongodb-2.2 -a notebook -l <openshift_login> rhc cartridge add rockmongo-1.1 -a notebook -l <openshift_login> cd notebook git rm -rf src/ pom.xml git commit -a -m "removed default generated files" git remote add notebook -m master git://github.com/shekhargulati/notebook-part2.git git pull -s recursive -X theirs notebook master git push
This will get your own notebook application up and running and you can test it by executing couple of curl commands as covered in part1. When you push your code to the OpenShift gear, the git hooks use the maven profile to build and deploy but does not run any test cases.
Benefits of using Jenkins on OpenShift
- Archived build information : Jenkins acts as a repository for all your builds and you can view the build history using Jenkins web ui. Each build occupy some space on disk so it make sense to just keep top 5 or 10 builds and discard others as we have limited disk space on OpenShift.Later in the blog, I will show you how to do it.
- No application downtime in case of build failure : Without Jenkins support when you do git push the maven build is performed on the same node on which application is running. The application is stopped before the build is started. So, if the application build fails because of test failure or any other reason the new build will not get deployed and application will experience downtime. This can be avoided by using Jenkins as only successful builds will be deployed and application is stopped after build is successful.
- Failed builds do not get deployed (leaving the previous working version in place) .
- Jenkins builders have additional resources like memory and storage so it does not consume anything from your application resources.
- A large community of plugins make Jenkins very powerful for other tasks such as build reports, build notification,artifacts uploading,etc. You can find the full list of plugins at https://wiki.jenkins-ci.org/display/JENKINS/Plugins
Adding Jenkins Support
Now that we have done the application setup let’s add Jenkins support to the application. To add Jenkins support we first have to create Jenkins OpenShift application and then add Jenkins client to the notebook application. When we add Jenkins client to notebook application, a new Jenkins job is created. This Jenkins job creates a Node (aka Slave) which will consume one Gear. Nodes and builder Applications are automatically deleted and the corresponding Gear is freed after 15 idle minutes. The net effect of this is that using Jenkins in your application consumes 2 more gears, 1 for the persistent server and 1 for the builder. But the builder gear is only alive only for the build, and will automatically die after 15 idle minutes.
To create a Jenkins application execute the following command.
rhc app create jenkins jenkins-1.4 -l <openshift_login>
The above command will create a Jenkins instance running on OpenShift. A new directory named Jenkins will be created on your local machine in the directory where you executed the rhc command. The Jenkins directory is empty with just a README file. The first thing you should do is to change the password to something you can remember. You can change the password by log in to https://jenkins-domainname.rhcloud.com/user/admin/configure using the password given by rhc command and changing the password to something you can remember easily. Please change the domainname with your OpenShift account domain name.
After creating the Jenkins application, the next step is to add Jenkins client cartridge to the notebook application. Type the command shown below to add Jenkins client cartridge.
rhc cartridge add jenkins-client-1.4 -a notebook -l <openshift_login>
The command above create a new Jenkins job to build notebook application. You can view the job at https://jenkins-<domain name>.rhcloud.com/job/notebook-build/ and can see the job configuration by clicking on the configure link.
Look at Jenkin job configuration
I will go over few configuration options which are interesting.
Builder Configuration : The first interesting configuration is about builder as shown below. The configuration below say that create a builder with small size and with jbossas-7 cartridge and the Jenkins master will wait for 5 minutes for the slave to come online.
Git Configuration : The next configuration of interest is the git SCM. It specifies the url of git repository to use, branch to use, etc. This section is important if you want to use Jenkins to build project which exist outside of OpenShift like project hosted on github etc.
Build Configuration : The last configuration which is interesting is under Build section. Here you can specify a shell script for building the project. For our job it does the following
- Specify whether project should be built using Java 6 or Java 7
- Specify XMX memory configuration for maven and build the maven project. The memory it configures is 396M.
- Deploying the application which includes stopping the application, pushing the content back from Jenkins to application, and finally deploying the artifacts.
Run the build
Next you can start a new build by clicking “Build Now” link which will trigger the new build. The build might take couple of minutes to start as it needs to launch a new slave node based on the “Builder Configuration”. You can view the status of the Job by looking at the console log. If the next build will run in under 15 minutes then existing builder will be reused and the total build time(time to create builder and time to run project build) will be less.
Customizing Jenkins on OpenShift
So far we didn’t have to do any customization, OpenShift did everything for us, from creating a Jenkins instance in Cloud to creating the Jenkins Job to build the project. Now we will customize the Jenkins instance so that we can view the Quality dashboard which I showed in the beginning. To achieve that we have to do the following :
- Install Plugins : We will install green ball plugin, cobertura plugin , and dashboard view plugins.
- Modify the existing Jenkins job : We will modify the existing job to run tests and generate cobertura coverage report.
- Create a Quality Dashboard: We will create a new view with type as dashboard and configure it to show test and cobertura statistics.
- Finally View the Quality Dashboard.
Lets do these steps one by one.
There are couple of ways(manual and automatic) you can install plugins in Jenkins. The manual way to install the plugin is to click on “Manage Jenkins” > “Manage Plugins” > “Advanced” and upload the .hpi plugin file. You can download all the plugins from http://updates.jenkins-ci.org/download/plugins/. The automatic way to install the plugins is to go to “Manage Jenkins” > “Manage Plugins” > “Available” and select the plugins you want to install. Sometimes it might happen that the list of available plugins is empty. If that is the case, from the “Manage Plugins -> Advanced” page, click on “Check now” to forcefully check for new updates. Once that is done, you should see the list of plugins to manage/install.
In this post we will take the manual approach. Go to http://updates.jenkins-ci.org/download/plugins/ and download cobertura, green balls, dashboard plugins and then upload the artifacts to Jenkins. The plugins are not yet installed and you have to restart the Jenkins server. To do that click on installed tab and then click the button “Restart Once No Jobs Are Running” (it will exist at bottom) .
After the restart make sure to clear the browser cache to see Green Balls plugin in action. The Jenkins home page would look as shown below.
Modify the existing Jenkins job
Now lets modify the “notebook-build” https://jenkins-<domain name>.rhcloud.com/job/notebook-build/configure. There are four configurations which we will change.
- Change the “Max # of builds to keep” to 5. This will save the disk space.
- In the execute shell we will remove the -DskipTests so that it run all the tests. This is shown below.
mvn --global-settings $OPENSHIFT_MAVEN_MIRROR clean package -Popenshift
- Check Publish Cobertura Coverage Report checkbox and enter text in Cobertura xml report pattern textbox (without quotes) “target/cobertura/notebook-*/coverage.xml”
- Check Publish JUnit test result report checkbox and enter text in Test Report XMLs textbox “notebook-*/target/surefire-reports/*.xml”
Save the configuration and click the “Build Now” link.
Quality Dashboard View
The last thing left in this blog is having a “Quality Dashboard View”. You can create a Quality Dashboard View by clicking the + button on home page as shown below.
Next give view the name “Quality Dashboard” and choose radio button for Dashboard view. After you click OK, you will see a new view. Edit the view and add notebook-build job to it. In the Dashboard Portlets section check the “Show standard Jenkins list at the top of the page”. Next add Cobertura Coverage, Test Statistics Chart, and Test Statistics Grid as shown below.
Finally you can view the Quality Dashboard which will show you the code coverage and test statistics.
Viewing the Quality Dashboard
In this blog I talked about how you can setup and customize Jenkins for your application running on OpenShift. I hope this post shows you how easy it is to start using Continuous integration in your development flow. No more excuses – time to just do it!