Continuous Integration and Deployment with OpenShift v3

I’m going to diverge a bit from my usual discussion of OpenShift v3 features and function to instead walk through how easy it is to integrate Jenkins into an OpenShift v3 environment.  Please keep in mind that this is intended to demonstrate the flexibility of the image based approach OpenShift v3 is introducing and what can already be done today by taking advantage of that.  It is not intended to represent the final vision of how general continuous integration/delivery will be implemented in OpenShift or how Jenkins will be integrated in the long term.

This guide is going to illustrate setting up a Jenkins pod in OpenShift and creating a Jenkins job which will trigger an OpenShift build of an application component, deploy the component for testing, validate the new component works, and then deploy that component into production.

I’ll be utilizing the build and deployment concepts covered in my earlier posts so if you haven’t read them, it is worth doing so now:

Prerequisites

As usual you’ll need to have built the latest OpenShift binary.  You can find instructions for that here:  https://github.com/openshift/origin/blob/master/README.md

In addition you’ll need the Jenkins configuration files located here: https://github.com/openshift/origin/tree/master/examples/jenkins

You’ll also need to set selinux to permissive mode:

$ sudo setenforce 0

And if you’re on Fedora, either stop firewalld or put the docker0 interface into the trusted zone:

$ systemctl stop firewalld

Starting OpenShift

Start your openshift server by running:

 $ sudo openshift start

(sudo is needed because openshift currently needs access to iptables to configure service routing)

Launch the Docker Registry Service

As with previous examples, we’ll be pushing the built images to a local Docker registry running in OpenShift, so deploy it by running:

$ openshift kubectl apply -f docker-registry-config.json

(Note:  kubectl has replaced kubecfg in the latest OpenShift and Kubernetes code.  The functionality is largely similar but a few command names and flags have changed.  Running openshift kubectl help provides an overview of the usage.  For a detailed summary, see: https://github.com/openshift/origin/blob/master/docs/cli.md)

Launch the Jenkins Service

Next we’ll create a Pod and Service for Jenkins.  The configuration for this service is very similar to the Docker registry service.  The primary differences being that it’s based on a different image (openshift/jenkins-1-centos) and that the Service is exposed on a different port (5002).  Note that even the unique port is no longer strictly necessary as Services now each get their own unique ip address and thus can overlap ports between Services.

{

“apiVersion”:”v1beta2″,

“creationTimestamp”:null,

“id”:”jenkins”,

“kind”:”Service”,

“port”:5002,

“containerPort”:8080,

“selector”:{

“name”:”jenkinspod”

}

},
………

“podTemplate”:{

“desiredState”:{

“manifest”:{

“containers”:[

{

“image”:”openshift/jenkins-1-centos”,

“name”:”jenkins-container”,

“ports”:[

{

“containerPort”:8080,

“protocol”:”TCP”

}

],

}

],

“version”:”v1beta1″

},

“restartpolicy”:{

}

}

Start the Jenkins service by running:

$ openshift kubectl apply -f jenkins-config.json

Create the Application

Now we’re going to define our application.  This application configuration is slightly more sophisticated than earlier examples, but it doesn’t introduce any new concepts.  Specifically it creates two essentially identical deployments of the application but using different image tags.  The first tag is openshift/origin-ruby-sample:test and the second is openshift/origin-ruby-sample:prod.

In addition to the two deployments and two services for the frontend, it creates only a single build for the frontend component.  When the component is built, the new image is tagged as :test which gives Jenkins an opportunity to validate the new version is working properly.  Jenkins is then responsible for adding a :prod tag to the same image which will trigger the production deployment.  (Click here(https://blog.openshift.com/builds-deployments-services-v3/) for a refresher on how tagging images leads to deployments).

The single build configuration definition which creates a new openshift/origin-ruby-sample:test image when built:

{

“apiVersion”: “v1beta1”,

“kind”: “BuildConfig”,

“metadata”: {

“name”: “ruby-test-build”,

“labels”: {

“name”: “ruby-test-build”

}

},

“parameters”: {

“output”: {

“imageTag”: “openshift/origin-ruby-sample:test”,

“registry”: “172.121.17.3:5001”

},

“source”: {

“git”: {

“uri”: “git://github.com/openshift/ruby-hello-world.git”

},

“type”: “Git”

},

“strategy”: {

“stiStrategy”: {

“builderImage”: “openshift/ruby-20-centos”

},

“type”: “STI”

}

}

}

Container configuration for the production deployment.  Note the use of the RAILS_ENV and RACK_ENV variables which control which database service the frontend connects to.  The test deployment uses a different value for these variables.

“containers”: [

{

“env”: [

{

“name”: “ADMIN_USERNAME”,

“value”: “${ADMIN_USERNAME}”

},

{

“name”: “ADMIN_PASSWORD”,

“value”: “${ADMIN_PASSWORD}”

},

{

“name”: “MYSQL_ROOT_PASSWORD”,

“value”: “${MYSQL_ROOT_PASSWORD}”

},

{

“name”: “MYSQL_DATABASE”,

“value”: “${MYSQL_DATABASE}”

},

{

“name”: “RACK_ENV”,

“value”: “production”

},

{

“name”: “RAILS_ENV”,

“value”: “production”

}

],

“image”: “172.121.17.3:5001/openshift/origin-ruby-sample:prod”,

“name”: “ruby-helloworld-prod”,

“ports”: [

{

“containerPort”: 8080

}

]

}

]

Create the application build configuration, services, and deployment configuration by running:

$ openshift kubectl process -f application-template.json | openshift kubectl apply -f -

Create a new Jenkins job

By now your Jenkins pod should be available.  Determine what IP address the service was assigned by running:

$ openshift kubectl get service jenkins

And then pass that IP and port to curl to create the new Jenkins job:

$ JENKINS_ENDPOINT=`openshift kubectl get services | grep jenkins | awk '{print $3":"$4}'`
$ cat job.xml | curl -X POST -H "Content-Type: application/xml" -H "Expect: " --data-binary @- http://$JENKINS_ENDPOINT/createItem?name=rubyJob

This will create a new job in your Jenkins server named rubyJob.

Examining the job definition, you can see it’s a basic shell script that performs some OpenShift operations.  (The openshift binary is part of the Jenkins image so it is able to run openshift commands).

First we determine the endpoint of the test version of the application and the docker registry:

TEST_ENDPOINT=`openshift kubectl get services -s $OPENSHIFT_HOST| grep frontend-test | awk '{print $4":"$5}'`
REGISTRY_ENDPOINT=`openshift kubectl get services -s $OPENSHIFT_HOST| grep docker-registry | awk '{print $3":"$4}'`

Next we remove the existing test deployment, if there is one, by removing the ReplicationController and Pod.  Old deployments will automatically be replaced during deployment when the build is run, but we want to make sure we’re testing the new version and not the old one, so it’s easiest to remove it ourselves before we start testing:

TEST_RC_ID=`openshift kubectl get replicationController -s $OPENSHIFT_HOST | grep frontend-test | awk '{print $1}'`
openshift kubectl delete replicationController $TEST_RC_ID -s $OPENSHIFT_HOST
TEST_POD_ID=`openshift kubectl get pod -s $OPENSHIFT_HOST | grep frontend-test | awk '{print $1}'`
openshift kubectl delete pod $TEST_POD_ID -s $OPENSHIFT_HOST

Now we start a new build of the frontend component which, based on the BuildConfig, will pick up the latest source in our repository:

$ openshift kubectl start-build ruby-test-build

This starts a build in OpenShift which ultimately leads to a deployment of the test application, so once we start the build, we can start polling the test application to confirm it gets deployed successfully and works as expected:

while [ $rc -ne 0 -a $count -lt $attempts ]; do
    curl -s --connect-timeout 2 $TEST_ENDPOINT
    rc=$?
    if [ $rc -ne 0 ]; then
        count=$(($count+1))
        echo "Attempt $count/$attempts"
        sleep 5
    fi
done
if [ $count -eq $attempts ]; then
    echo "Failed to reach test deployment"
    exit 1
fi

Finally, assuming the test was successful, we tag the image with the :prod which triggers a deployment of the production application.  First we look up the Docker image id of the test image tag, and then we create a new tag named prod for that same image id.  This will automatically trigger a deployment of the production application using the image we just validated in test.

COMMIT=`curl -s http://$REGISTRY_ENDPOINT/v1/repositories/openshift/origin-ruby-sample/tags/test`
curl -s -X PUT  http://$REGISTRY_ENDPOINT/v1/repositories/openshift/origin-ruby-sample/tags/prod -d "$COMMIT"

Run the Build

Now that we’ve created the Jenkins job, we just have to execute it.  Normally this could be triggered via GitHub webhook or git polling, but for simplicity in this case you’ll just trigger it by hand.  Log into the Jenkins console in your web browser.  You can get the ip/port again by running:

$ openshift kubectl get services | grep jenkins | awk '{print $3":"$4}'

Once you navigate to the Jenkins console, select the rubyJob.

Figure 1: Jenkins console
Figure 1: Jenkins console

 

Next select Build with Parameters.  The only parameter is the endpoint of your OpenShift API server.  The default value (which is the default docker bridge ip address) should not need to be changed.

 

Figure 2: Jenkins job page
Figure 2: Jenkins job page

 

Finally you can monitor the console output of the running job to watch as the new application is built, tested, and rolled into production.

To validate that both versions of the application are deployed, determine their endpoints:

$ openshift kubectl get services | grep frontend

Now use your browser to access both applications.  If you try to access either one before the Jenkins job has been run (assuming you started from a clean state) neither will be accessible.  If you attempt to access the production application and test application repeatedly while the Jenkins job is running, you’ll see that the test application becomes available before the production application.  You may find it interesting to add a sleep to the job script before the production tag is performed, so you can more easily confirm that the production deployment only happens after the test validation is complete.

Lastly, if you update the source repository being used by the build, you can push your own changes to the application and confirm that, on repeated builds, the production application continues to run the old version of the application until the test deployment is validated, and only then gets updated to the new version.

Conclusion

I hope this exercise demonstrated to you just how easy it is to integrate relatively sophisticated continuous integration/deployment flows using the capabilities already present in OpenShift.  Moving forward, we intend to create much tighter integration with Jenkins, similar to what is available in OpenShift v2 (e.g. automatic creation of Jenkins jobs for your application), but there’s no need to wait for us, you can get started today!

 

Categories
Jenkins, OpenShift Container Platform, OpenShift Online, OpenShift Origin
Tags
,
Comments are closed.