CloudFoundry: Using Blue-Green deloyment and route mapping for continuous deployment

One of the goals of Continuous Deployment (CD) is an automated deployment and patching process.  And as your  organization becomes more efficient at deployment and velocity increases, you want to ensure that availability is maintained while keeping risk to a minimum.

Cloud Foundry supports route mapping that allows you to stand up a new live version of a service, validate its functionality, and only then switch your users over to the new service – all transparent to the end user.  This type of deployment is typically referred to as Blue-Green deployment as popularized by Martin Fowler.

Standing up a new, live service into a production environment allows you to validate the service fully before switching users over.  And if there are issues discovered, reversion back to the older service is just a matter of switching the mapping back.

Concept Overview

The general idea is that we will push my “hello-spring” application as a base version “Blue1.0”.  The URL of this application will be the only one ever seen by end users.

Then we emulate a change that requires a new version of this application, “Green1.1”.  We stand up a live version of this service at a new URL, one only network accessible or known to internal testers.

If the new service passes quality checks, then we will append a mapping to the original end user URL to this new service.  Users will then be load balanced between both service instances, Blue1.0 and Green1.1.

If there are no end-user issues, then the original service can be retired and the route to the original “Blue1.0” version will be removed.  At that point, end users will only be routed to the “Green1.1” version.   This all happens with no downtime or interruption to service.

If severe issues in the new code base are discovered, switching the mapping back to the original service can also be done seamlessly.

Cloud Foundry Provider and CLI

All the concepts and commands in this article apply to any Cloud Foundry instance, whether on a public or local CF provider.  But in this article, I’m going to assume you are using Pivotal Cloud Foundry, and this is reflected in the URL domain (cfapps.io).  Translate as necessary.

If you don’t have a Cloud Foundry provider and the CLI tool installed, read my article here which goes through the full instructions for Ubuntu.

Before going on, make sure you have the CF CLI tool installed:

$ cf --version

And that you are logged in to your CF provider (URL is based on Pivotal):

$ cf login -a https://api.run.pivotal.io

Simple Project

In order to test the Blue-Green deployment, we are going to use a very simple Java project that displays a JVM system property named “MYVERSION”.

We first install the Java8 OpenJDK:

$ sudo add-apt-repository ppa:openjdk-r/ppa -y
$ sudo apt-get update
$ sudo apt-cache policy openjdk-8-jdk
$ sudo apt-get install openjdk-8-jdk -y
$ sudo update-alternatives --config java
$ sudo update-alternatives --config javac

$ /usr/bin/java -version
openjdk version "1.8.0_141"
OpenJDK Runtime Environment (build 1.8.0_141-8u141-b15-3~14.04-b15)
OpenJDK 64-Bit Server VM (build 25.141-b15, mixed mode)

Then we grab the source code of the project from github and package it using Maven:

$ sudo apt-get install git maven -y
$ git clone https://github.com/fabianlee/hello-spring.git
$ cd hello-spring
$ mvn package

Look at the manifest.yml file before moving on, because this defines the default settings and environment values used when pushing this application to CF.

---
applications:
- name: hello-spring
  memory: 768M
  instances: 1
  random-route: true
  path: target/hello-spring-1.0.0-BUILD-SNAPSHOT
  env:
    MYVERSION: Blue1.0

Of particular note is the environment variable “MYVERSION”.  Unless we override this value, the value on deployments will be “Blue1.0”.

Push original version of App, Blue1.0

Deploy the application using the name “hello-spring-blue”, and then view the details of the application.

$ cf push hello-spring-blue10
$ cf app hello-spring-blue10
$ cf env hello-spring-blue10 | grep MYVERSION

The last environment check will show you that the “MYVERSION” property is default to what is found in the manifest file “Blue1.0”.

The ‘random-route’ property in the manifest creates a unique URL for our application that won’t conflict with other users of the CF PaaS.  Run the commands below to parse out the URL for use later in our remapping commands.

$ blueHostname=`cf app hello-spring-blue10 | grep routes | awk {'print $2'} | cut -d'.' -f 1`

$ blueDomain=`cf app hello-spring-blue10 | grep routes | awk {'print $2'} | cut -d'.' -f 2,3,4,5,6`

$ echo end user URL will always be http://$blueHostname.$blueDomain/myversion

Go ahead and point your browser at the suggested URL or use curl, and you should see that it returns back a simple text page containing “MYVERSION: Blue1.0”.

$ curl http://$blueHostname.$blueDomain/myversion

Push new version of App, green1.1

Now deploy a new version of the same application, but name it “hello-spring-green”.  We want to set the MYVERSION environment variable before it starts.

$ cf push hello-spring-green11 --no-start

$ cf set-env hello-spring-green11 MYVERSION Green1.1

$ cf start hello-spring-green11

Then view the details of the deployed “hello-spring-green” application.

$ cf app hello-spring-green11

$ cf env hello-spring-green11 | grep MYVERSION

The last environment check will show you that the “MYVERSION” property is set to “Green1.1”.  Run the commands below to parse out the URL for use later in our remapping commands.

$ greenHostname=`cf app hello-spring-green11 | grep routes | awk {'print $2'} | cut -d'.' -f 1`

$ greenDomain=`cf app hello-spring-green11 | grep routes | awk {'print $2'} | cut -d'.' -f 2,3,4,5,6`

$ echo temporary URL for testing new release is at http://$greenHostname.$greenDomain/myversion

Point your browser at the suggested URL or use curl, and you should see that it returns back a simple text page containing “MYVERSION: Green1.1”.

$ curl http://$greenHostname.$greenDomain/myversion

Original URL shared with both blue and green apps

Now we want the original URL ($blueHostname) to direct users not only to the original Blue1.0 application, but also the Green1.1 application.  This is done with the following command.

$ cf map-route hello-spring-green11 $blueDomain --hostname $blueHostname

Now the original end user URL is mapped to both Blue1.0 and Green1.1.  It is load balanced, and if you refresh your browser or curl multiple times, you will see the page delivered from the Blue and Green backend randomly.

$ curl http://$blueHostname.$blueDomain/myversion

Unmap route to blue

After a period of validation, you can go ahead and unmap the route from the original Blue1.0 application altogether.

$ cf unmap-route hello-spring-blue10 $blueDomain --hostname $blueHostname

Pulling from the original Blue URL now will only deliver back “MYVERSION: Green1.0”.  This is the only backend servicing this route.

$ curl http://$blueHostname.$blueDomain/myversion

The deployment is now complete, and end users have transparently been migrated to a new version of the service without any interruption in service.

Cleanup

The only thing left to do is delete the URL route created by the green1.1 deployment.  This was used for internal testing, but was never used by end users.  You can delete it explicitly:

$ cf delete-route $greenDomain --hostname $greenHostname

If you do not need the Blue1.0 service anymore, and won’t need it for reversion in case of critical issues, you can delete it.

$ cf delete hello-spring-blue10

If after deletion of the original service, it makes it easier to think about “Blue” always being live, then you can always rename the CF app name.  This does not affect the route servicing customers.

$ cf rename hello-spring-green11 hello-spring-blue11
$ cf apps

 Reversion

If you need a reversion plan in case severe issues are introduced in the codebase, you can always keep the older service version running.  And then if you needed to revert, that would be a simple matter of mapping the route back to blue10, and unmapping from green11.

Here is the command for adding a mapping back to blue10.

$ cf map-route hello-spring-blue10 $blueDomain --hostname $blueHostname

Now the end user URL once again points at both services, which can be verified by running curl multiple times.

$ curl http://$blueHostname.$blueDomain/myversion

And finally, remove the route to the green11 service.

$ cf unmap-route hello-spring-green11 $blueDomain --hostname $blueHostname

The end user URL once again only delivers requests from the blue10 service.

$ curl http://$blueHostname.$blueDomain/myversion

 

 

REFRENCES

https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html

http://www.mattstine.com/2013/07/10/blue-green-deployments-on-cloudfoundry/

https://martinfowler.com/bliki/BlueGreenDeployment.html

http://cli.cloudfoundry.org/en-US/cf/ (CLI reference)

https://github.com/cloudfoundry-samples/hello-spring (simple app, shows system env, data bindings, cloud properties)

https://www.digitalocean.com/community/tutorials/how-to-use-blue-green-deployments-to-release-software-safely

NOTES

cf login -a https://api.run.pivotal.io -o <orgname>

cf delete-orphaned-routes