docker

GitLab: self-managed runner for CI/CD jobs on GCP VM instances

The globally shared set of GitLab runners for CI/CD jobs works well for building binaries, publishing images, and reaching out to publicly available endpoints for services and infrastructure building. But the ability to run a private, self-managed runner can grant pipelines entirely new levels of functionality on several fronts: Can communicate openly to private, internal GitLab: self-managed runner for CI/CD jobs on GCP VM instances

Github: automated build and publish of multi-platform container image with Github Actions

Github Actions provide the ability to define a build workflow based on Github repository events.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article, I will show how to define workflow steps that build and push a Github: automated build and publish of multi-platform container image with Github Actions

Docker: building multi-platform images that use fat manifest list/index

Docker can build multi-platform images that use a manifest index (fat manifest list) by using the Docker buildx command with backing containerd runtime and QEMU for cross-platform emulation. Using a manifest index for multi-platform images simplifies application level orchestration by using the same name and version for all architectures.  For example: # same image name Docker: building multi-platform images that use fat manifest list/index

Docker: installing Docker CE on Ubuntu

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken from the official Docker for Ubuntu page, but I fine-tuned them per Ubuntu22+ standards.

Github: automated build and publish of containerized GoLang app with Github Actions

Github Actions provide the ability to define a build workflow based on Github repository events.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a statically-linked GoLang binary that when Github: automated build and publish of containerized GoLang app with Github Actions

Github: automated build and publish of containerized Spring Boot app using GitHub Actions

Github Actions provide the ability to define a build workflow directly in Github.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a simple Spring Boot web application that when Github: automated build and publish of containerized Spring Boot app using GitHub Actions

Github: locally invoked release process for a Gradle built Java Spring Boot project

The GitHub “Release” page for a repository can provide your consumers a convenient way to download a binary version of your software as well as track the latest changes and enhancements. In this article, I will show how to invoke a local release process for a Java Spring Boot jar built with Gradle.  A new Github: locally invoked release process for a Gradle built Java Spring Boot project

Java: build OCI compatible image for Spring Boot web app using jib

While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator Java: build OCI compatible image for Spring Boot web app using jib

Java: Creating Docker image for Spring Boot web app using gradle

While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator Java: Creating Docker image for Spring Boot web app using gradle

Python: Building an image for a Flask app served from Gunicorn

Gunicorn is a WSGI HTTP server commonly used to run Flask applications in production.  Running Flask applications directly is great for development and testing of the basic request/response flow, but you need gunicorn to handle production level loads,  concurrency, logging, and timeouts. In this article, I will show you how to build a Docker image Python: Building an image for a Flask app served from Gunicorn

Kubernetes: volumeMount, emptyDir, and env equivalents during local Docker development

Kubernetes has a rich way of expressing volumes/ volumeMounts for mounting files, emptyDir for ephemeral directories, and env/envFrom for adding environment variables to your container definition running on a Kubernetes cluster. However, if you are actively iterating on the development of an image, it may slow you down to require a deployment to a remote Kubernetes: volumeMount, emptyDir, and env equivalents during local Docker development

Docker: Installing Docker CE on Ubuntu focal 20.04

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want Docker: Installing Docker CE on Ubuntu focal 20.04

Docker: building an ntp server image with Alpine and chrony

If you need a lightweight NTP server, an Alpine based container image with a chrony daemon takes up minimal runtime resources and is about 8Mb in size. I have pushed ‘fabianlee/docker-chrony-alpine‘ to docker hub.  The run command requires that you specify linux capabilities and a volume for the chrony.conf file, so the easiest way to Docker: building an ntp server image with Alpine and chrony

GCP: pushing GKE images into gcr.io to avoid pull rate limits

Docker hub now enforces pull rate limits (since November 2020).  And unfortunately, this limit is often reached at critical moments such as upgrades or infrastructure events when bulk pod recreation is happening. One way to avoid this problem is to place your images into an alternate image registry.  This could mean a lot of work GCP: pushing GKE images into gcr.io to avoid pull rate limits

Docker: determining container responsible for largest overlay directories

Whether you are running a docker daemon on a development host or a GKE worker node using Docker as the container engine, it is important to understand the amount of disk storage being utilized by the containers. If you navigate into the ‘/var/lib/docker/overlay2’ directory, you will  see cryptic hashed ids representing the containers layers instead Docker: determining container responsible for largest overlay directories

GoLang: Using multi-stage builds to create clean Docker images

The Go programming language is a natural fit for containers because it can compile down to a single statically-linked binary.  And if you place that single executable on top of scratch, a distroless image, or a small image like alpine, your final image has a minimal footprint which is great for consumption and reuse. But the GoLang: Using multi-stage builds to create clean Docker images

Docker: Placing limits on cpu usage in containers

Containers themselves are light, but by default a container has access to all the CPU resources the Docker host kernel scheduler will allow. Internally Docker uses cgroups to limit CPU resources, and this is exposed as the flag “–cpus” when bringing up a docker container: sudo docker run -it –cpus=1.0 alpine:latest /bin/sh This will limit Docker: Placing limits on cpu usage in containers

Docker: Placing limits on container memory using cgroups

Containers themselves are light, but by default a container has access to all the memory resources of the Docker host. Internally Docker uses cgroups to limit memory resources, and in its simplest form is exposed as the  flags “-m” and “–memory-swap” when bringing up a docker container. sudo docker run -it -m 8m –memory-swap 8m Docker: Placing limits on container memory using cgroups

Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size

If you are using the overlay2 storage driver, you can place limits on the rootfs within a container but only if using an xfs backing filesystem (not ext4). As a quick test of your Docker install, check your Docker storage driver and backing filesystem, then attempt to spin up a small alpine image with a Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size

Linux: Using xfs project quotas to limit capacity within a subdirectory

XFS is a journaled filesystem that has excellent parallel performance, and is licensed under the GPL which means it has been included in many Linux distributions. One of the features of XFS is the ability to enforce quotas based on user, group, and project.  In this article, I will show how to assign filesize quotas Linux: Using xfs project quotas to limit capacity within a subdirectory

Zabbix: Using Docker Compose to install and upgrade Zabbix

Zabbix distributes Docker images for each component.  Not only does this mean you can quickly standup the monitoring solution, but upgrades also become a simple matter of trading up images. In this article, I will show how to stand up and then upgrade a zabbix installation using docker-compose.

Docker: Installing Docker CE on Ubuntu bionic 18.04

Update Dec 2021: I have written an updated article for installing Docker on focal 20.04 Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those Docker: Installing Docker CE on Ubuntu bionic 18.04

Docker: Using docker-compose and networking to link a Spring Boot app to an external service dependency

In earlier versions of Docker,  links were used to connect two containers by enabling network access as well as sharing environment variables.  This approach is being deprecated in favor of a shared network between the services.  And instead of environment propagation, environment values will need to be explicitly added to each service that requires them. Docker: Using docker-compose and networking to link a Spring Boot app to an external service dependency

Docker: Running a Spring Boot based app using docker-compose

Docker Compose gives us the ability to define services,  ports, volumes, and networks in a single file.  This  file provides a convenient and unified view into what would otherwise be a long list of docker commands. In this article, you will run the Spring Boot based spring-music app in a Docker container using docker-compose.

Docker: Using docker-compose to link a MongoDB server and client

Docker Compose gives us the ability to define and orchestrate multiple containers in order to construct a service.  In this article, we will use Docker Compose to create a MongoDB server and then another container that is used exclusively as a MongoDB client. While it is entirely possible to manually create these containers, links, and Docker: Using docker-compose to link a MongoDB server and client

Docker: Base image when deploying a GoLang binary in a container

Update Oct 2020: multi-stage builds now provide a standard way to leverage a fat build image, while allowing your published image to remain small.  This article is still useful for comparing base image sizes. GoLang applications are a great architectural fit for a Docker container because of their single binary executable. But you need to Docker: Base image when deploying a GoLang binary in a container

Docker: Visualizing image hierarchy and container dependency using dockviz

The Docker console commands for listing and viewing containers and images (ps, images, history, inspect) provides a wealth of information, but when you are managing hundreds of containers, a graph view of the container inventory and their dependencies can be critical for operations. Dockviz can help you visualize your containers and images by creating an PNG image Docker: Visualizing image hierarchy and container dependency using dockviz

Docker: logspout for Docker log collection

Docker log collection can be done using various methods, one method that is particularly effective is having a dedicated container whose sole purpose is to automatically sense other deployed containers and aggregate their log events. This is the architectural model of logspout, an open-source project that acts as a router for the stdout/stderr logs of other containers. If you do Docker: logspout for Docker log collection

Docker: Sending Spring Boot logging to syslog

Building services using Spring Boot gives a development team a jump start on many production concerns, including logging.  But unlike a standard deployment where logging to a local file is where the developer’s responsibility typically ends, with Docker we must think about how to log to a public space outside our ephemeral container space. The Docker: Sending Spring Boot logging to syslog

Docker: Installing Docker CE on Ubuntu 14.04 and 16.04

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu 14.04 and 16.04.