CloudFoundry: Scaling the spring-music webapp, Part 3

Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. This article is part of a series that explores different facets of a Cloud Foundry deployment using the spring-music project as an example.

This article is Part 3 of a series on Cloud Foundry concepts:

Specifically in this article, we will horizontally and vertically scale up the spring-music application and show how this affects the routing and logging.

Previous articles in the series describe basic deployment of the spring-music application, and we pick up from there.

Vertical Scaling

The simplest method for scaling is vertical, adding memory/disk resources to a single instance of the application in order to accommodate more processing or simultaneous users. This can be done with the “cf scale” command.

If we have deployed spring-music as detailed in part 1 of this series, then we have a single instance already deployed:

$ cf apps
Getting apps in org fleetest1 / space development as fabian.lee@test1.com...
OK

name requested state instances memory disk urls spring-music started 1/1 1.2G 1G spring-music-monarchal-mass.cfapps.io

And that single instance is using ‘1G’ of memory (1024Mb). If we want to scale that up to 1280Mb:

$ cf scale spring-music -m 1280M -f

$ cf apps
Getting apps in org fleetest1 / space development as fabian.lee@test1.com...
OK
name requested state instances memory disk urls spring-music started 1/1 1.2G 1G spring-music-monarchal-mass.cfapps.io

Now checking ‘cf apps’ again shows that there is 1.2G (1280Mb) available for this instance.

Note that this restarts the application, so any in-memory persistence will be lost.

Horizontal Scaling

Vertical scaling only goes so far before you need to horizontally scale, creating multiple instances of the application. Incoming requests are then distributed across the instances.

To increase the number of instances of our application, use the scale command with the ‘-i’ switch.

$ cf scale spring-music -m 1024M -i 2 -f

$ cf apps
Getting apps in org fleetest1 / space development as fabian.lee@test1.com...
OK

name requested state instances memory disk urls 
spring-music started 2/2 1G 1G spring-music-monarchal-mass.cfapps.io

Now we see “2/2” instances indicating we are running two instances of the same application, each using 1G of memory.

Incoming requests and GoRouter

The component that distributes the incoming requests to the available instances of the application is called the GoRouter. By default it uses round-robin load balancing, and it can itself be horizontally scaled in order to accommodate higher usage.

Typically, the GoRouter is fronted by a highly available load balancer like an F5 or AWS ELB depending on your supporting infrastructure.

You can get a map of the GoRouter’s URL to application definitions by using ‘cf routes’:

$ cf routes
Getting routes for org fleetest1 / space development as fabian.lee@test1.com ...

space         host                          domain      port   path   type   apps           service
development   spring-music-monarchal-mass   cfapps.io                        spring-music

Instance serving the request

Now that we have two instances of the application, tail the logs of the spring-music application. First we will get the guid, which will make more sense in a second when we look at the log output.

$ cf app spring-music --guid
5fd9e495-029b-4758-b68f-d02be86d8517

$ cf logs spring-music

Press refresh on the main application screen in your browser several times. Requests to both instance #0 as well as instance #1 will be sent to your stdout.

This log output should look similar to the line below, with the ‘vcap_request_id’ being unique for each request, the ‘app_id’ being the guid of the application we retrieved earlier, and ‘app_index’ indicating which application instance served the request (#0 or #1).

2017-11-30T17:54:53.48-0500 [RTR/0] OUT spring-music-monarchal-mass.cfapps.io - [2017-11-30T22:54:53.476+0000] "GET /albums HTTP/1.1" 200 0 4678 "http://spring-music-monarchal-mass.cfapps.io/" "...<useragent>..." "10.10.2.186:50894" "10.10.149.149:61010" x_forwarded_for:"100.1.237.157, 10.10.2.186" x_forwarded_proto:"http" vcap_request_id:"fc7ac3f8-65ed-47c5-6be9-51f6a3378506" response_time:0.011157593 app_id:"5fd9e495-029b-4758-b68f-d02be86d8517" app_index:"0" x_b3_traceid:"cfdfecf60080bae1" x_b3_spanid:"cfdfecf60080bae1" x_b3_parentspanid:"-"

The app_index will continually change as the GoRouter sends the request to either instance #0 or #1 of the spring-music application.

REFERENCES

https://docs.cloudfoundry.org/devguide/deploy-apps/cf-scale.html

https://docs.cloudfoundry.org/concepts/http-routing.html#round-robin

https://github.com/cloudfoundry/gorouter

https://github.com/cloudfoundry/gorouter#load-balancing (default_balancing_algorithm: round-robin|least-connection)

http://docs.pivotal.io/pivotalcf/1-12/concepts/high-availability.html (recommended # instances for HA)

https://www.cloudfoundry.org/routing-performance/

https://docs.cloudfoundry.org/adminguide/troubleshooting_slow_requests.html

https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html (common spring properties, can turn off ‘X-Application-Context’ header using ‘management.add-application-context-header=false)

https://www.dontpanicblog.co.uk/2017/04/14/spring-boot-actuator-trace/ (spring Boot actuator provides advanced logging and health)