With kustomize built into the kubectl CLI since version 1.14, there is little reason not to take advantage of this overlay system to deploy components to your Kubernetes cluster.
Kustomize has the advantage that it is purpose built to understand and validate yaml and Kubernetes CRD, as opposed to bespoke templating solutions using sed/envsubst, Ansible, or yq. And Helm is a great tool, but is heavier and comes with requirements for packaging and building.
In this article, we will build a base NGINX deployment and then enrich the definition using an overlay that can dynamically restart when its backing configmap changes.
Prerequisites
The only environmental prerequisites are that you have kubectl version 1.14+ (which has kustomize built into the CLI commands), and that your KUBECONFIG environment variable is pointed at a Kubernetes config.
Then download my project from github for use in the sections below.
kubectl version --short # grab project from github sudo apt install git -y git clone https://github.com/fabianlee/kustomize-overlays-with-reload.git cd kustomize-overlays-with-reload root_dir=$(realpath .)
kustomize base
The first thing we will do is examine the base resource for ‘nginx-index-html’. This is a minimalist definition for an NGINX container that returns an index.html that is sourced from a configmap.
cd $root_dir/base/nginx-index-html $ cat kustomization.yaml namespace: nginx-index resources: - namespace.yaml - deployment.yaml - service.yaml configMapGenerator: - name: nginx-cm namespace: default files: - cm-index.html generatorOptions: disableNameSuffixHash: true
The resource components create a namespace “nginx-index”, then place the deployment and service inside this namespace. The content delivered from NGINX comes from the configmap based on the local file “cm-index.html”.
<html> <head></head> <body>Hello, World!</body> </html>
The deployment.yaml defines the volume and volumeMount that place this configmap at /usr/share/nginx/html/index.html and therefore delivered whenever a GET HTTP request is invoked.
Deploying this to your Kubernetes cluster is as simple as using the normal apply command but adding the “-k” flag to indicate the use of kustomize.
# view the yaml that will be created kubectl kustomize # now deploy yaml kubectl apply -k . # test deployment using curl located inside container kubectl exec -it deployment/nginx-deployment -n nginx-index -- curl http://localhost <html> <head></head> <body>Hello, World!</body> </html>
kustomize overlay
To realize the power of kustomize, we will now take that same base NGINX resource, but now enrich it using an overlay.
Let’s use stakater’s Reload operator to watch for changes in the index.html configmap, and automatically do a rolling restart if we modify the contents. The Reload operator can be deployed like below.
cd $root_dir/base/reloader kubectl apply -k .
To use this Reload operator, we need to add an annotation to the deployment indicating that changes in its configmap should trigger a rolling restart. Instead of modifying the base, we will use an overlay to add this feature.
cd $root_dir/overlays/nginx-index-with-reloader cat kustomization.yaml resources: - ../../base/nginx-index-html namespace: nginx-reloader nameSuffix: "-reloader" # add annotation that allows configmap reloading # 'nginx-cm' is the name of configmap, BUT with nameSuffix above, need to tack on '-reloader' commonAnnotations: configmap.reloader.stakater.com/reload: nginx-cm-reloader configMapGenerator: - name: nginx-cm behavior: replace files: - cm-index.html generatorOptions: disableNameSuffixHash: true
Notice that we refer back to the base definition, “../../base/nginx-index-html” to pull our original definition but in a unique namespace of “nginx-reloader”. Then we redefine the configmap with a replace behavior so we can use our own version of “cm-index.html”.
And to enroll in the Reload operator’s logic, we add the “configmap.reloader.stakater.com/reload” annotation to the overlay definition.
To test this new overlay, use the commands below.
# view new overlay definition kubectl kustomize # apply overlay definition kubectl apply -k . # test using internal curl kubectl exec -it deployment/nginx-deployment-reloader -n nginx-reloader -- curl http://localhost <html><head></head> <body> <h1>nginx-index-html enhanced with reloader</h1> </body> </html>
You can see that the index.html content now comes from the overlay’s cm-index.html.
But we also need to test the auto redeployment if the configmap changes.
# change the overlay contents sed -i 's/reloader/reloader-updated1/' cm-index.html # reapply the overlay # the Reload operator will sense the configmap change kubectl apply -k . # test using internal curl, notice the content changed kubectl exec -it deployment/nginx-deployment-reloader -n nginx-reloader -- curl http://localhost <html><head></head> <body> <h1>nginx-index-html enhanced with reloader-updated1</h1> </body> </html>
This is just one example of how overlays can be used to enrich and transform base definitions. The addition, removal, and replacement of entire sections can be performed using the various kustomize patch types.
REFERENCES
kubectl, three inline patch types (patchesStrategicMerge,patchesJSON6902,patches)