Kubernetes: kustomize with Helm charts


Categories: Kubernetes

kustomize is typically used to overlay a base set of yaml, but it also has the ability to leverage existing Helm charts, and overlay a set of custom values with HelmChartInflationGenerator.

In this article, I will use kustomize to deploy the Bitnami NGINX Helm chart with overridden values that provide a customized nginx.conf and custom index page.

Prerequisite

I’ll assume you have an existing Kubernetes cluster, the kubectl binary already installed, and a valid KUBECONFIG environment variable.

Now just make sure you have the Helm binary installed.  Here is an article where I describe installation of Helm.  Then you should be able to check the version like below.

helm version --short

Example project

All the source from this article can be found in my github project.

git clone https://github.com/fabianlee/kustomize-overlays-with-reload.git
cd kustomize-overlays-with-reload

root_dir=$(realpath .)
cd $root_dir/base/nginx-bitnami-chart

kustomization.yaml

The main file we will be editing is “kustomization.yaml”, which contains our resources, configmap, and NGINX Helm chart information.

namespace: nginx-chart

# make sure namespace exists
resources:
- namespace.yaml

# custom index.html page
configMapGenerator:
- name: nginx-cm
  namespace: default
  files:
  - cm-index.html

# Bitnami NGINX helm chart
helmChartInflationGenerator:
- chartName: nginx
  chartRepoUrl: https://charts.bitnami.com/bitnami
  chartVersion: 9.5.4
  releaseName: my-nginx-release
  releaseNamespace: nginx-chart
  # override with these custom values
  values: values.yaml

values.yaml

The other file we need to look at is “values.yaml”.  The Bitnami NGINX chart contains a list of parameters that can be used to configure the deployment.  We will set custom values for the:

  • replica count
  • cpu resource limits
  • volume mount for a custom index page loaded from a configmap
  • a custom health check point created by modifying the nginx.conf
replicaCount: 1

resources:
  limits:
    cpu: 300m

# mount our own custom index.html from configmap
extraVolumes:
  - name: nginx-homedir
    configMap:
      name: nginx-cm
      items:
        - key: cm-index.html
          path: index.html
extraVolumeMounts:
  - name: nginx-homedir
    mountPath: "/usr/share/nginx/html" # /app by default
    readOnly: true

# insert custom NGINX configuration for healthcheck
serverBlock: |-
  server {
      listen       8080;
      server_name  _;
      root         /usr/share/nginx/html;

      # for kubernetes health check
      location = /healthz {
       add_header Content-Type text/plain;
       return 200 'OK';
      }
  }

Deploy using kustomize

To deploy using kustomize, use the commands below.

# view yaml manifest first
kubectl kustomize --enable-helm

# apply to kubernetes cluster
# 'apply' command does not have enable-helm flag
kubectl kustomize --enable-helm | kubectl apply -f -

Validate

You can see that our custom index page, matching the content of ‘cm-index.html’ is returned.  Also, the custom health check at ‘/healthz’ properly returns “OK”.

ns=nginx-chart

# will return our custom page from configmap
kubectl exec -it deployment/my-nginx-release -n $ns -- curl http://localhost:8080

# will return the custom healthcheck "OK" wit
kubectl exec -it deployment/my-nginx-release -n $ns -- curl http://localhost:8080/healthz

Summary

If you are already using kustomize in your deployment pipeline, it could make sense for you to deploy your Helm charts with kustomize as well given its rich overlay abilities.

 

REFERENCES

kustomize ref, HelmChartInflationGenerator

bitnami nginx chart

github, bitnami nginx chart parameters

kustomize with –enable-helm

github issue, example using bitnami and extraVolumes/extraVolumeMounts

stackoverflow, using extraVolums/extraVolumeMounts

 

NOTES

manual installation of helm chart

helm repo add bitnami https://charts.bitnami.com/bitnami

helm repo list
NAME   	URL                               
bitnami	https://charts.bitnami.com/bitnami

helm search repo bitnami/nginx
NAME                            	CHART VERSION	APP VERSION	DESCRIPTION                           
bitnami/nginx                   	9.5.4        	1.21.3     	Chart for the nginx server            
bitnami/nginx-ingress-controller	7.6.21       	0.48.1     	Chart for the nginx Ingress controller

# chart metadata and version (can be done before install)
helm show chart bitnami/nginx

# show all available values
helm show values bitnami/nginx

# install chart
helm install my-release bitnami/nginx