minikube: exposing a deployment using ingress with secure TLS


Categories: Kubernetes

minikube makes it easy to spin up a local Kubernetes cluster, and adding an Ingress is convenient with its built-in Addons.

In this article, I want to take it one step further and show how to use a custom key/certificate to expose a service using TLS (secure https).

Prerequisites

  • A container or virtual machine manager (Docker, KVM, Podman, etc.).
  • The minikube binary installed
  • ‘step‘ utility for generating custom CA and TLS certificate

Container or virtual machine manager

Here is a previous article I wrote on installing Docker CE.

minikube

Installing minikube is just making the latest binary available in the PATH.

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube

Certificate creation utility, step

# install step utility for certificate generation
wget -q https://github.com/smallstep/cli/releases/download/v0.25.0/step_linux_0.25.0_amd64.tar.gz
tar xvfz step_linux_0.25.0_amd64.tar.gz step_0.25.0/bin/step --strip-components 2
rm step*.gz

Start minikube, enable Ingress

# start minikube, by default uses docker
minikube start

# validate access to cluster
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready control-plane 6m42s v1.28.3

# enable ingress-nginx using Addon
minikube addons enable ingress

# wait for deployment readiness
kubectl rollout status deployment ingress-nginx-controller -n ingress-nginx --timeout=90s

Test Ingress with non-secure http

Before we move forward with secure Ingress over TLS, let’s prove out simple http. We will use the same docker image and instructions as described in these tutorials at k8-sdocs and w3cubdocs.

Simple example deployed at NodePort

# simple deployment
kubectl create deployment web --image=gcr.io/google-samples/hello-app:1.0 --port=8080
kubectl expose deployment web --type=NodePort --port=8080

# wait for full readiness
kubectl rollout status deployment web -n default --timeout=90s

# get NodePort info
kubectl get service web
web_url=$(minikube service web --url)
echo "about to insecurely pull from deployment at NodePort: $web_url"

# use http to pull content from service at NodePort
$ curl $web_url
Hello, world!
Version: 1.0.0
Hostname: web-5d76dc856d-6xhh6

Exposed via Ingress

# set variable used for domain name
domain=hello-world.info

# create Ingress using host 'hello-world.info'
wget https://k8s.io/examples/service/networking/example-ingress.yaml
kubectl apply -f example-ingress.yaml

# wait for ingress to get assigned IP address (will match 'minikube ip')
kubectl get ingress
sleep 60
lb_ip=$(kubectl get ingress -o=jsonpath="{.items[].status.loadBalancer.ingress[0].ip}")
echo "loadbalancer IP for ingress: $lb_ip"

# adding loadbalancer IP to local hosts file
# this way we do not need minikube tunnel
echo $lb_ip $domain | sudo tee -a /etc/hosts

# prove that service is available at Ingress
$ curl http://$domain
Hello, world!
Version: 1.0.0
Hostname: web-5d76dc856d-6xhh6

Create CA cert and TLS certificate

Using the step utility, we can create a custom root CA, intermediate, and leaf certificate for secure TLS at the URL “https://hello-world.info”.

# set variable
domain=hello-world.info

# create root CA
./step certificate create --no-password --insecure --profile root-ca "Example Root CA" root_ca.crt root_ca.key

# intermediate cert
./step certificate create "Example Intermediate CA 1" intermediate_ca.crt intermediate_ca.key --profile intermediate-ca --ca ./root_ca.crt --ca-key ./root_ca.key --no-password --insecure

# leaf cert
./step certificate create $domain $domain.crt $domain.key --profile leaf --not-after=8760h --ca ./intermediate_ca.crt --ca-key ./intermediate_ca.key --bundle --no-password --insecure

# show results
./step certificate inspect $domain.crt --short

Load TLS cert and key into cluster

# load key+cert into secret in kube-system namespace
kubectl -n kube-system create secret tls ingress-tls-cert --key $domain.key --cert $domain.crt

# do direct patch to set default TLS certificate (had problem setting with 'minikube addons configure ingress')
kubectl patch deployment ingress-nginx-controller -n ingress-nginx --type "json" --patch '[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--default-ssl-certificate=kube-system/ingress-tls-cert"}]'

# wait for readiness
kubectl rollout status deployment ingress-nginx-controller -n ingress-nginx --timeout=90s

# ensure that secret was loaded by looking at log
$ kubectl logs deployment/ingress-nginx-controller -n ingress-nginx | grep ingress-tls-cert
...
backend_ssl.go:67] "Adding secret to local store" name="kube-system/ingress-tls-cert"

Validate ingress with secure TLS

With the TLS key+cert now loaded, we should be able to pull from the service using secure https.

# pull using TLS (trusted CA cert required)
curl --cacert root_ca.crt https://$domain

# show leaf certificate
./step certificate inspect --roots=root_ca.crt --short https://$domain

 

REFERENCES

minikube, introduction

minikube, custom TLS cert with ingress 

stackoverflow, patch command to set ingress-nginx default certificate

k8s-docs, minikube with ingress nginx

kubernetes.io, minikube and Ingress

github container-hello-app, source code for image gcr.io/google-samples/hello-app

kubernetes.github.io, TLS and default cert

NOTES

If you have a problem where the ingress components never show up, try starting fresh and enabling the ‘ingress-dns’ addon first.

minikube delete
minikube start
minikube addons enable ingress-dns
minikube addons enable ingress

If you have another virtual machine manager (like KVM), you can start minikube like:

minikube start --driver=kvm2