Docker: building multi-platform images that use fat manifest list/index

Docker can build multi-platform images that use a manifest index (fat manifest list) by using the Docker buildx command with backing containerd runtime and QEMU for cross-platform emulation.

Using a manifest index for multi-platform images simplifies application level orchestration by using the same name and version for all architectures.  For example:

# same image name and version, but works across architectures
docker run --rm -i "fabianlee/tiny-tools-multi-arch:2.0.0" uname -m

The same name and version (fabianlee/tiny-tools-multi-arch:2.0.0) can be used regardless of whether I am on my Intel host (amd64), Mac M2 (arm64), or Amazon EKS using Graviton (arm64).  The OCI client pulling the image parses the manifest list and selects the correct image based on architecture.

Prerequisites

I will be assuming a host server running Ubuntu 22+ with an amd64 native architecture.

# host architecture reporting "x86_64" (which is amd64)
uname -m

Additionally Docker and QEMU need to be configured as described in my articles below:

If this is configured properly, then you should be able to run an arm64 Docker image via QEMU software emulation like below.

# ARM64 image should run now (shows 'aarch64' with is arm64 family)
docker run --rm -t arm64v8/alpine uname -m

Because the ‘docker buildx’ command will also need to push the image to hub.docker.com, establish a login before continuing.

# login to Docker Hub, used to push multi-platform images
docker login

Download example project

I have an example project on github, tiny-tools-multi-arch, with a Dockerfile that can be used for testing multi-platform building.

# download example project
sudo apt install git curl -y
git clone https://github.com/fabianlee/tiny-tools-multi-arch
cd tiny-tools-multi-arch

Create multi-platform builder

# currently only 'default' builder, but this cannot do multi-platform builds
docker buildx ls

# create new builder backed by 'docker-container' driver that can do multi-platform builds
docker buildx create --name mybuilder --driver docker-container

# set buildx to use new builder
docker buildx use mybuilder

Build and push multi-platform image

# https://hub.docker.com/repositories/<yourDockerId>
OWNER=<yourDockerId>
# docker tag for owner/platform:version
OPV="$OWNER/tiny-tools-multi-arch:1.0.0"
# CSV list of platforms we are building for
PLATFORMS_LIST="linux/amd64,linux/arm64,linux/arm/v7"

# build for multiple platforms, push to Hub
docker buildx build --platform $PLATFORMS_LIST -f Dockerfile -t $OPV --push .

# show manifest index just created
docker manifest inspect $OPV | head

Run image on various architectures using QEMU emulation

# shows "x86_64" corresponding to amd64 family
docker run -it --platform linux/amd64 --network host --rm $OPV uname -m

# shows "aarch64" corresponding to arm64 family
docker run -it --platform linux/arm64 --network host --rm $OPV uname -m

# shows "armv71" corresponding to arm32 family
docker run -it --platform linux/arm/v7 --network host --rm $OPV uname -m

Convert to Docker v2.2 fat manifest list

The ‘docker buildx’ tool creates the latest OCI manifest index schema.  But some older container registry servers do not support this, and instead support the Docker fat manifest list schema v2.2.  If you need to convert, use the regctl tool.

# download regctl tool
curl -L https://github.com/regclient/regclient/releases/latest/download/regctl-linux-amd64 >regctl
chmod 755 regctl

# run tool to covert
OPV22="$OWNER/tiny-tools-multi-archv22:1.0.0"
echo "converting from OCI index schema: $OPV to Docker v2.2 manifest list schema: $OPV22"
./regctl image mod $OPV --to-docker --create $OPV22

# show Docker v2.2 manifest list
docker manifest inspect $OPV22 | head

Since regctl is pushing to Docker Hub, it can reuse the existing login context.  If you were using regctl to push to a different repository, you would need to use ‘regctl registry login <registry>’ so it had permission to push into that remote repository.

 

REFERENCES

github docker buildx plugin

docker, buildx command reference

github buildkit, configuration that can be specified via buildx ‘config’ flag

buildkit, capturing build information

docker, multi-platform building with QEMU

docker, multi-platform not necessary if your language provides cross-compliation

ubuntu.com, qemu-system-arm on Ubuntu20

hub.docker.com, arm64v8/alpine

hub.docker.com, tiny-tools-multi-arch

hub.docker.com, tiny-tools-multi-archv22

github fabianlee, tiny-tools-multi-arch

danmanners.com, buildah for multi-arch

github buildah, docs for create manifest

andrewlock.net, ‘load’ can be used to pull up buildx images, but not local pull

aarch64 and arm64 are same family

jfrog, buildx and Artifactory

github, OCI manifest index schema

github, Docker v2.2 manifest list schema

stackoverflow, differences between OCI index and Docker manifest list

docker.com, cross-platform with docker the manual way with build for each arch and ‘docker manifest’

github docker, platform list values

github containerd, explanation in code of platform list values and normalization

stackoverflow, explains platform string and normalization of the value

BretFisher, discussion about platform naming for Docker

digilution.io, shows how to use ‘load’ flag to run multi-platform image

docker.com, github-actions for multi-platform