Volumes are the preferred method for persisting data for Docker containers. In this article, I will show how to create and use volumes for persistence, as well as tmpfs for temporary storage. At its absolute simplest, creating and mounting a volume backed by a local directory looks like: # make host directory mkdir -p /data … Docker: Working with local volumes and tmpfs mounts
Containers themselves are light, but by default a container has access to all the CPU resources the Docker host kernel scheduler will allow. Internally Docker uses cgroups to limit CPU resources, and this is exposed as the flag “–cpus” when bringing up a docker container: sudo docker run -it –cpus=1.0 alpine:latest /bin/sh This will limit … Docker: Placing limits on cpu usage in containers
Containers themselves are light, but by default a container has access to all the memory resources of the Docker host. Internally Docker uses cgroups to limit memory resources, and in its simplest form is exposed as the flags “-m” and “–memory-swap” when bringing up a docker container. sudo docker run -it -m 8m –memory-swap 8m … Docker: Placing limits on container memory using cgroups
If you are using the overlay2 storage driver, you can place limits on the rootfs within a container but only if using an xfs backing filesystem (not ext4). As a quick test of your Docker install, check your Docker storage driver and backing filesystem, then attempt to spin up a small alpine image with a … Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size
Update Jan 2022: If you would rather have a mail server running in Kubernetes, see my article here. There are many reasons you might want your own Postfix server: maybe you need to test various relay settings, or validate the ability of your mail script to use TLS, or maybe you are a developer that … Docker: Running a Postfix container for testing mail during development
Zabbix distributes Docker images for each component. Not only does this mean you can quickly standup the monitoring solution, but upgrades also become a simple matter of trading up images. In this article, I will show how to stand up and then upgrade a zabbix installation using docker-compose.
Update Dec 2021: I have written an updated article for installing Docker on focal 20.04 Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those … Docker: Installing Docker CE on Ubuntu bionic 18.04
If you are going to implement major changes to your publicly hosted WordPress site, it can be beneficial to create a local sandbox where those changes can be tested ahead of time. In this article, I will show how to use Docker Compose to create a local instance of WordPress, and the “All-in-One WP Migration” … WordPress: Cloning your WordPress site locally using Docker Compose
Updated Aug 2023: tested on Ubuntu 22 with QEMU 6.2.0/libvirt 8.0.0 and with Minikube v1.31.2 Minikube is a tool that runs a full Kubernetes stack on a host, making it ideal for local development and experimentation. We are taking it one step further by placing Minikube into a virtual machine run by the KVM virtualization … Kubernetes: running Minikube locally on Ubuntu using KVM
If you receive this error upon trying to run commands from the CloudFoundry CLI: “Error read/writing config: unexpected end of JSON input” This is most likely due to an error in the “$HOME/.cf/config.json” file. Make sure the user has full ownership: ls -l ~/.cf/config.json chown $USER:$USER ~/.cf/config.json chmod 755 ~/.cf/config.json And if that does not … CloudFoundry: CLI error, unexpected end of JSON input
BOSH is a project that unifies release, deployment, and lifecycle management of cloud based software. It also serves as the underlying infrastructure for deploying the CloudFoundry PaaS. In this article, I will lead you through deploying the BOSH Director to Amazon EC2 which is the first step in deploying both CloudFoundry CFAR as well as CFCR … CloudFoundry: Installing a BOSH Director on AWS
Updated article to latest Minikube, Feb 2019 Minikube is a tool that runs a Kubernetes stack inside a single VM being run by a local virtualization engine such as VirtualBox. This makes it ideal for local development and experimentation. In this article we’ll be going through installation and validation of a Minikube installation on Ubuntu … Kubernetes: running Minikube locally on Ubuntu using VirtualBox
In earlier versions of Docker, links were used to connect two containers by enabling network access as well as sharing environment variables. This approach is being deprecated in favor of a shared network between the services. And instead of environment propagation, environment values will need to be explicitly added to each service that requires them. … Docker: Using docker-compose and networking to link a Spring Boot app to an external service dependency
Docker Compose gives us the ability to define services, ports, volumes, and networks in a single file. This file provides a convenient and unified view into what would otherwise be a long list of docker commands. In this article, you will run the Spring Boot based spring-music app in a Docker container using docker-compose.
Spring Boot is an easy way to create production grade Spring based web applications. In this article, I want to illustrate how easily a Spring Boot application can be deployed to run as a Docker container. Specifically, we will build the spring-music project into a single executable jar that will then be copied into a … Docker: Running a Spring Boot based app in a Docker container
Docker Compose gives us the ability to define and orchestrate multiple containers in order to construct a service. In this article, we will use Docker Compose to create a MongoDB server and then another container that is used exclusively as a MongoDB client. While it is entirely possible to manually create these containers, links, and … Docker: Using docker-compose to link a MongoDB server and client
Update Oct 2020: multi-stage builds now provide a standard way to leverage a fat build image, while allowing your published image to remain small. This article is still useful for comparing base image sizes. GoLang applications are a great architectural fit for a Docker container because of their single binary executable. But you need to … Docker: Base image when deploying a GoLang binary in a container
Cloud Foundry deployed Java applications can send log events to stdout/stderr and then a bound syslog drain can send this to a logging solution like ELK for ingestion. One area that has always been tricky when dealing with logging is multi-line Java stack traces. By default, because each line in the stack trace has a … CloudFoundry: Extracting Java multiline exception stack traces from Logback and Log4j2 using Logstash
Update Jan 2019: Now using CredHub instead of ‘–vars-store’ (which will be deprecated in CF 3) Even if you are developing a service or application that will ultimately be deployed to a private Cloud Foundry instance, having a local CF instance for development work is still an ideal development workflow. There is a local CF … CloudFoundry: Deploy Cloud Foundry locally using BOSH Lite on Ubuntu
BOSH is a project that unifies release, deployment, and lifecycle management of cloud based software. In this article I will describe how to install BOSH unto VirtualBox. This implementation is often called “BOSH Lite” because it internally uses containers to emulate VMs.
Update Jan 2019: Instead of these instructions, please see my newer article for local BOSH installation on VirtualBox using CLI v2. BOSH is a project that unifies release, deployment, and lifecycle management of cloud based software. The BOSH CLI v1 is now older and has been replaced with CLI v2, so if you want … CloudFoundry: Install BOSH CLI v1 and BOSH Lite on Ubuntu
The Cloud Foundry CLI makes it easy to authenticate into a target API endpoint to consume the Cloud Controller API (CAPI). The “cf login” command allows you to pass or interactively provide the user, password, org, and space. Once logged in, if you want to run raw commands, the “cf curl” can execute a request … CloudFoundry: Authenticating and running CF API commands using curl
For most Cloud Foundry applications and services, you can avoid downtime for maintenance with a combination of following best practices for 12 factor app development and taking advantage of Cloud Foundry’s scaling and flexible routing to implement Blue-Green deployment. But there will still be times where an application, service, or shared backend component does not … CloudFoundry: Beyond the maintenance page, delivering a response during service unavailability
One of the goals of Continuous Deployment (CD) is an automated deployment and patching process. And as your organization becomes more efficient at deployment and velocity increases, you want to ensure that availability is maintained while keeping risk to a minimum. Cloud Foundry supports route mapping that allows you to stand up a new live … CloudFoundry: Using Blue-Green deloyment and route mapping for continuous deployment
Enabling the use of real-time JVM monitoring tools like jconsole and VisualVM can be extremely beneficial when troubleshooting issues. These tools work by enabling a JMX/RMI communication channel to the JVM. These are typically thought of as local development tools, but they can also be used on remote CF containers running Java. In this article, … CloudFoundry: Enabling Java JMX/RMI access for remote containers
Java thread and heap dumps are valuable tools for troubleshooting local development, but they can also be used on remote CF containers running a JVM. In this article, we’ll go through various method of gathering this data from a Cloud Foundry container and then tools for analyzing this data. Now matter how uniform your environments, … CloudFoundry: Java thread and heap dump analysis on remote containers
Remote debugging of Java applications from an IDE can be essential when debugging difficult issues. There is no reason to give this functionality up just because you are deploying to a container in Cloud Foundry. In this article we’ll go over how to enable remote debugging from a local Eclipse IDE to a public CF … CloudFoundry: Enabling Java remote debugging with Eclipse
Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. This article is part of a series that explores different facets of a Cloud Foundry deployment using the spring-music project as an example. This article is Part 5 of a series on Cloud Foundry concepts: Deploying the spring-music webapp, Part 1 Persisting spring-music data … CloudFoundry: Monitoring the spring-music webapp, Part 5
Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. This article is part of a series that explores different facets of a Cloud Foundry deployment using the spring-music project as an example. This article is Part 4 of a series on Cloud Foundry concepts: Deploying the spring-music webapp, Part 1 Persisting spring-music data … CloudFoundry: Logging for the spring-music webapp, Part 4
Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. It supports multiple infrastructure platforms (EC2, VMware, OpenStack), and is able to standardize deployment, logging, scaling, and routing in a way that is friendly to a continuous delivery pipeline. In this series of articles, we will use the spring-music web application … CloudFoundry: Exploring Cloud Foundry using the spring-music application
Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. This article is part of a series that explores different facets of a Cloud Foundry deployment using the spring-music project as an example. This article is Part 3 of a series on Cloud Foundry concepts: Deploying the spring-music webapp, Part 1 Persisting … CloudFoundry: Scaling the spring-music webapp, Part 3
Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. This article is part of a series that explores different facets of a Cloud Foundry deployment using the spring-music project as an example. This article is Part 2 of a series on Cloud Foundry concepts: Deploying the spring-music webapp, Part 1 Persisting … CloudFoundry: Persisting spring-music data using Postgres service, Part 2
Update Jan 2019: The latest PCF Dev 2.x releases only support Windows/Mac and not Linux. This article describes the older v0.28 PCF Dev is a distribution of Cloud Foundry that has a minimal footprint and is designed to run locally on a developer’s machine. Using this lightweight distribution of Cloud Foundry, a developer can debug … CloudFoundry: PCF Dev 0.28 for local development on Ubuntu
Cloud Foundry is an opinionated Platform-as-a-Service that allows you to manage applications at scale. It supports multiple infrastructure platforms, and is able to standardize deployment, logging, scaling, and routing in a way friendly to a continuous delivery pipeline. This article is Part 1 of a series on Cloud Foundry concepts: Deploying the spring-music webapp, Part 1 … CloudFoundry: Deploying the spring-music webapp, Part 1
This article has been updated in October 2018 and is now tested for HAProxy 1.8.14. The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was … HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 16.04 with Systemd
This article has been updated in October 2018 and is now tested for HAProxy 1.8.14. The reload functionality in HAProxy till now has always been “not perfect but good enough”, perhaps dropping a few connections under heavy load but within parameters everyone was willing to accept. And because of the potential impact, a reload was … HAProxy: Zero downtime reloads with HAProxy 1.8 on Ubuntu 14.04
The Docker console commands for listing and viewing containers and images (ps, images, history, inspect) provides a wealth of information, but when you are managing hundreds of containers, a graph view of the container inventory and their dependencies can be critical for operations. Dockviz can help you visualize your containers and images by creating an PNG image … Docker: Visualizing image hierarchy and container dependency using dockviz
Docker log collection can be done using various methods, one method that is particularly effective is having a dedicated container whose sole purpose is to automatically sense other deployed containers and aggregate their log events. This is the architectural model of logspout, an open-source project that acts as a router for the stdout/stderr logs of other containers. If you do … Docker: logspout for Docker log collection
Building services using Spring Boot gives a development team a jump start on many production concerns, including logging. But unlike a standard deployment where logging to a local file is where the developer’s responsibility typically ends, with Docker we must think about how to log to a public space outside our ephemeral container space. The … Docker: Sending Spring Boot logging to syslog
Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu 14.04 and 16.04.