docker

GitLab: self-managed runner for CI/CD jobs on GCP VM instances


Categories: DevOps

The globally shared set of GitLab runners for CI/CD jobs works well for building binaries, publishing images, and reaching out to publicly available endpoints for services and infrastructure building. But the ability to run a private, self-managed runner can grant pipelines entirely new levels of functionality on several fronts: Can communicate openly to private, internal … GitLab: self-managed runner for CI/CD jobs on GCP VM instances

Github: automated build and publish of multi-platform container image with Github Actions


Categories: Containers, DevOps

Github Actions provide the ability to define a build workflow based on Github repository events.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article, I will show how to define workflow steps that build and push a … Github: automated build and publish of multi-platform container image with Github Actions

Docker: building multi-platform images that use fat manifest list/index


Categories: Containers, Development

Docker can build multi-platform images that use a manifest index (fat manifest list) by using the Docker buildx command with backing containerd runtime and QEMU for cross-platform emulation. Using a manifest index for multi-platform images simplifies application level orchestration by using the same name and version for all architectures.  For example: # same image name … Docker: building multi-platform images that use fat manifest list/index

Docker: installing Docker CE on Ubuntu


Categories: Containers, Linux

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken from the official Docker for Ubuntu page, but I fine-tuned them per Ubuntu22+ standards.

Github: automated build and publish of containerized GoLang app with Github Actions


Categories: Containers, Development

Github Actions provide the ability to define a build workflow based on Github repository events.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a statically-linked GoLang binary that when … Github: automated build and publish of containerized GoLang app with Github Actions

Github: automated build and publish of containerized Spring Boot app using GitHub Actions


Categories: Containers, Development, Java

Github Actions provide the ability to define a build workflow directly in Github.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a simple Spring Boot web application that when … Github: automated build and publish of containerized Spring Boot app using GitHub Actions

Github: locally invoked release process for a Gradle built Java Spring Boot project


Categories: Containers, Development, Java

The GitHub “Release” page for a repository can provide your consumers a convenient way to download a binary version of your software as well as track the latest changes and enhancements. In this article, I will show how to invoke a local release process for a Java Spring Boot jar built with Gradle.  A new … Github: locally invoked release process for a Gradle built Java Spring Boot project

Java: build OCI compatible image for Spring Boot web app using jib


Categories: Containers, Java, Kubernetes

While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator … Java: build OCI compatible image for Spring Boot web app using jib

Java: Creating Docker image for Spring Boot web app using gradle


Categories: Containers, Development, Java

While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator … Java: Creating Docker image for Spring Boot web app using gradle

Python: Building an image for a Flask app served from Gunicorn


Categories: Containers, Python

Gunicorn is a WSGI HTTP server commonly used to run Flask applications in production.  Running Flask applications directly is great for development and testing of the basic request/response flow, but you need gunicorn to handle production level loads,  concurrency, logging, and timeouts. In this article, I will show you how to build a Docker image … Python: Building an image for a Flask app served from Gunicorn

Kubernetes: volumeMount, emptyDir, and env equivalents during local Docker development


Categories: Containers, Kubernetes

Kubernetes has a rich way of expressing volumes/ volumeMounts for mounting files, emptyDir for ephemeral directories, and env/envFrom for adding environment variables to your container definition running on a Kubernetes cluster. However, if you are actively iterating on the development of an image, it may slow you down to require a deployment to a remote … Kubernetes: volumeMount, emptyDir, and env equivalents during local Docker development

Docker: Installing Docker CE on Ubuntu focal 20.04


Categories: Containers, Linux

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want … Docker: Installing Docker CE on Ubuntu focal 20.04

Docker: building an ntp server image with Alpine and chrony


Categories: Containers

If you need a lightweight NTP server, an Alpine based container image with a chrony daemon takes up minimal runtime resources and is about 8Mb in size. I have pushed ‘fabianlee/docker-chrony-alpine‘ to docker hub.  The run command requires that you specify linux capabilities and a volume for the chrony.conf file, so the easiest way to … Docker: building an ntp server image with Alpine and chrony

GCP: pushing GKE images into gcr.io to avoid pull rate limits


Categories: Containers

Docker hub now enforces pull rate limits (since November 2020).  And unfortunately, this limit is often reached at critical moments such as upgrades or infrastructure events when bulk pod recreation is happening. One way to avoid this problem is to place your images into an alternate image registry.  This could mean a lot of work … GCP: pushing GKE images into gcr.io to avoid pull rate limits

Docker: determining container responsible for largest overlay directories


Categories: Virtualization

Whether you are running a docker daemon on a development host or a GKE worker node using Docker as the container engine, it is important to understand the amount of disk storage being utilized by the containers. If you navigate into the ‘/var/lib/docker/overlay2’ directory, you will  see cryptic hashed ids representing the containers layers instead … Docker: determining container responsible for largest overlay directories

GoLang: Using multi-stage builds to create clean Docker images


Categories: Containers, Development

The Go programming language is a natural fit for containers because it can compile down to a single statically-linked binary.  And if you place that single executable on top of scratch, a distroless image, or a small image like alpine, your final image has a minimal footprint which is great for consumption and reuse. But the … GoLang: Using multi-stage builds to create clean Docker images

Docker: Placing limits on cpu usage in containers


Categories: Containers, Linux

Containers themselves are light, but by default a container has access to all the CPU resources the Docker host kernel scheduler will allow. Internally Docker uses cgroups to limit CPU resources, and this is exposed as the flag “–cpus” when bringing up a docker container: sudo docker run -it –cpus=1.0 alpine:latest /bin/sh This will limit … Docker: Placing limits on cpu usage in containers

Docker: Placing limits on container memory using cgroups


Categories: Containers, Linux

Containers themselves are light, but by default a container has access to all the memory resources of the Docker host. Internally Docker uses cgroups to limit memory resources, and in its simplest form is exposed as the  flags “-m” and “–memory-swap” when bringing up a docker container. sudo docker run -it -m 8m –memory-swap 8m … Docker: Placing limits on container memory using cgroups

Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size


Categories: Containers, Linux

If you are using the overlay2 storage driver, you can place limits on the rootfs within a container but only if using an xfs backing filesystem (not ext4). As a quick test of your Docker install, check your Docker storage driver and backing filesystem, then attempt to spin up a small alpine image with a … Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size

Linux: Using xfs project quotas to limit capacity within a subdirectory


Categories: Linux

XFS is a journaled filesystem that has excellent parallel performance, and is licensed under the GPL which means it has been included in many Linux distributions. One of the features of XFS is the ability to enforce quotas based on user, group, and project.  In this article, I will show how to assign filesize quotas … Linux: Using xfs project quotas to limit capacity within a subdirectory

Zabbix: Using Docker Compose to install and upgrade Zabbix


Categories: Containers, Monitoring

Zabbix distributes Docker images for each component.  Not only does this mean you can quickly standup the monitoring solution, but upgrades also become a simple matter of trading up images. In this article, I will show how to stand up and then upgrade a zabbix installation using docker-compose.

Docker: Installing Docker CE on Ubuntu bionic 18.04


Categories: Containers, Linux

Update Dec 2021: I have written an updated article for installing Docker on focal 20.04 Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those … Docker: Installing Docker CE on Ubuntu bionic 18.04

Docker: Using docker-compose and networking to link a Spring Boot app to an external service dependency


Categories: Containers

In earlier versions of Docker,  links were used to connect two containers by enabling network access as well as sharing environment variables.  This approach is being deprecated in favor of a shared network between the services.  And instead of environment propagation, environment values will need to be explicitly added to each service that requires them. … Docker: Using docker-compose and networking to link a Spring Boot app to an external service dependency

Docker: Running a Spring Boot based app using docker-compose


Categories: Containers

Docker Compose gives us the ability to define services,  ports, volumes, and networks in a single file.  This  file provides a convenient and unified view into what would otherwise be a long list of docker commands. In this article, you will run the Spring Boot based spring-music app in a Docker container using docker-compose.

Docker: Using docker-compose to link a MongoDB server and client


Categories: Containers, Linux

Docker Compose gives us the ability to define and orchestrate multiple containers in order to construct a service.  In this article, we will use Docker Compose to create a MongoDB server and then another container that is used exclusively as a MongoDB client. While it is entirely possible to manually create these containers, links, and … Docker: Using docker-compose to link a MongoDB server and client

Docker: Base image when deploying a GoLang binary in a container


Categories: Containers, Linux, Scripting

Update Oct 2020: multi-stage builds now provide a standard way to leverage a fat build image, while allowing your published image to remain small.  This article is still useful for comparing base image sizes. GoLang applications are a great architectural fit for a Docker container because of their single binary executable. But you need to … Docker: Base image when deploying a GoLang binary in a container

Docker: Visualizing image hierarchy and container dependency using dockviz


Categories: Containers, Scripting

The Docker console commands for listing and viewing containers and images (ps, images, history, inspect) provides a wealth of information, but when you are managing hundreds of containers, a graph view of the container inventory and their dependencies can be critical for operations. Dockviz can help you visualize your containers and images by creating an PNG image … Docker: Visualizing image hierarchy and container dependency using dockviz

Docker: logspout for Docker log collection


Categories: Containers, Logging

Docker log collection can be done using various methods, one method that is particularly effective is having a dedicated container whose sole purpose is to automatically sense other deployed containers and aggregate their log events. This is the architectural model of logspout, an open-source project that acts as a router for the stdout/stderr logs of other containers. If you do … Docker: logspout for Docker log collection

Docker: Sending Spring Boot logging to syslog


Categories: Containers, DevOps, Java, Monitoring

Building services using Spring Boot gives a development team a jump start on many production concerns, including logging.  But unlike a standard deployment where logging to a local file is where the developer’s responsibility typically ends, with Docker we must think about how to log to a public space outside our ephemeral container space. The … Docker: Sending Spring Boot logging to syslog

Docker: Installing Docker CE on Ubuntu 14.04 and 16.04


Categories: Containers, Linux

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu 14.04 and 16.04.