Persisting spring-music data using Postgres service, Part 2

Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale.  This article is part of a series that explores different facets of a Cloud Foundry deployment using the spring-music project as an example.

This article is Part 2 of  a series on Cloud Foundry concepts:

In this particular article, we will create a Cloud Foundry Postgres service to externalize the persistent store instead of using the default in-memory H2 database which is destroyed every time the application is restarted or restaged.

Continue reading “Persisting spring-music data using Postgres service, Part 2”

CloudFoundry: PCF Dev for local development on Ubuntu

PCF Dev is a distribution of Cloud Foundry that has a minimal footprint and is designed to run locally on a developer’s machine.  Using this lightweight distribution of Cloud Foundry, a developer can debug and deploy applications locally.

In this article, we’ll go through the installation of PCF Dev on an Ubuntu development host.

Continue reading “CloudFoundry: PCF Dev for local development on Ubuntu”

CloudFoundry: Deploying the spring-music webapp, Part 1

Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale.  It supports multiple infrastructure platforms, and is able to standardize deployment, logging,  scaling, and routing in a way friendly to a continuous delivery pipeline.

This article is Part 1 of  a series on Cloud Foundry concepts:

In this particular article, we will install the command line interface for Cloud Foundry on Ubuntu and then use that to deploy the Spring Boot based spring-music project to a CF provider.

Continue reading “CloudFoundry: Deploying the spring-music webapp, Part 1”

HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 16.04 with Systemd

The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was typically only done during non-peak traffic times.

But with the popularity of microservices, containerization, continuous deployment, and dynamically scalable architecture, it has become critical for our load balancers to provide zero downtime reloads because reloading can potentially happen every few seconds even during peak production load.

There have been some seminal pieces written on how to achieve this level of availability with HAProxy. Yelp Engineering wrote up how to use qdiscs to delay the SYN packets, then followed up with using a combination of Nginx and HAProxy communicating over unix sockets. An alternative solution used two instances of HAProxy with an iptables flip.

But now with the ability in HAProxy 1.8 to pass listening sockets from the old process, along with Linux kernel 3.9 support of SO_REUSEPORT we finally have a solution that doesn’t feel like an ingenious hack of the Linux kernel and networking stack.

Continue reading “HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 16.04 with Systemd”

HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 14.04

The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was typically only done during non-peak traffic times.

But with the popularity of microservices, containerization, continuous deployment, and dynamically scalable architecture, it has become critical for our load balancers to provide zero downtime reloads because reloading can potentially happen every few seconds even during peak production load.

There have been some seminal pieces written on how to achieve this level of availability with HAProxy.  Yelp Engineering wrote up how to use qdiscs to delay the SYN packets, then followed up with using a combination of Nginx and HAProxy communicating over unix sockets. An alternative solution used two instances of HAProxy with an iptables flip.

But now with the ability in HAProxy 1.8 to pass listening sockets from the old process, along with Linux kernel 3.9 support of SO_REUSEPORT we finally have a solution that doesn’t feel like an ingenious hack of the Linux kernel and networking stack.

Continue reading “HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 14.04”

Docker: Visualizing image hierarchy and container dependency using dockviz

The Docker console commands for listing and viewing containers and images (ps, images, history, inspect) provides a wealth of information, but when you are managing hundreds of containers, a graph view of the container inventory and their dependencies can be critical for operations.

Dockviz can help you visualize your containers and images by creating an PNG image representing the container links and image lineage.

Continue reading “Docker: Visualizing image hierarchy and container dependency using dockviz”

Docker: logspout for Docker log collection

Docker log collection can be done using various methods, one method that is particularly effective is having a dedicated container whose sole purpose is to automatically sense other deployed containers and aggregate their log events.

This is the architectural model of logspout, an open-source project that acts as a router for the stdout/stderr logs of other containers.

If you do not have docker installed yet, see my article here.  Before moving on, you should be able to run the hello-world container.

Continue reading “Docker: logspout for Docker log collection”

Docker: Sending Spring Boot logging to syslog

Building services using Spring Boot gives a development team a jump start on many production concerns, including logging.  But unlike a standard deployment where logging to a local file is where the developer’s responsibility typically ends, with Docker we must think about how to log to a public space outside our ephemeral container space.

The Docker logging drivers capture all the output from a container’s stdout/stderr, and can send a container’s logs directly to most major logging solutions (syslog, Logstash, gelf, fluentd).

As an added benefit, by making the logging implementation a runtime choice for the container, it provides flexibility to use a simpler implementation during development but a highly-available, scalable logging solution in production.

Continue reading “Docker: Sending Spring Boot logging to syslog”

Docker: Installing Docker CE on Ubuntu 14.04 and 16.04

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization.

These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu 14.04 and 16.04.

Continue reading “Docker: Installing Docker CE on Ubuntu 14.04 and 16.04”