Istio Ambient mode is a data plane model that eliminates the need for an envoy sidecar proxy on each of your workloads. This reduces resource overhead, timing issues between sidecar lifecycle and your containers, and the need to restart your workloads to upgrade proxy versions.
In this article, I will show you how to install and test Ambient mode on a local Minikube cluster on Ubuntu 22.04/Debian 12 bookworm.
Install required tools and Minikube
Install Docker
See my article here on Installing Docker on Ubuntu/Debian.
Install kubectl
kubectl is the command line tool used to deploy and manage applications.
curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/bin/.
You can verify the version using:
# check path and version which kubectl kubectl version
Install Minikube
Now download and put the latest minikube binary into /usr/local/bin.
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/ # check version minikube version
Start Minikube
Start Minikube explicitly with the Docker container runtime/driver and the ‘kindnet‘ Network Policy.
minikube start --container-runtime=docker --driver=docker --cni=kindnet # check status minikube status # logs should reflect docker driver minikube logs | grep -i "selected driver:" # logs should reflect kindnet CNI minikube logs | grep "Creating CNI manager" # logs should reflect docker runtime minikube logs | grep -i "container runtime version" # test kubectl against cluster kubectl get nodes
Note that I have tested the installation successfully with CNI=kindnet|calico, driver=docker|kvm2, and container runtime=docker|containerd. Although the kvm2 driver does require that we install istio in IPv4 mode in a later section.
# starting minikube with alternate driver and container runtime minikube -c containerd start --driver=kvm2 --cni=kindnet
Install Istio
Per the official documentation, download and install Istio along with the newer Gateway CRD (Gateway API replaces the Istio API e.g. VirtualService).
curl -L https://istio.io/downloadIstio | sh -
cd istio-*
# if using KVM2 container runtime add: --set values.cni.ambient.ipv6=false
bin/istioctl install --set profile=ambient --skip-confirmation
bin/istioctl verify-install
# deploy Gateway CRD
kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || \
{ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml; }
If using the KVM2 driver for Minikube, you need to install Istio in IPv4 mode to avoid errors during istio-cni-node startup, “ip6tables-restore unable to initialize table ‘nat'”. This sets AMBIENT_IPV6=false in the configmap/istio-cni-config.
Deploy BookInfo sample app
Now we are going to deploy the small “Book Info” web application that comes with Istio. This is creating the deployments and services, but does NOT put the apps under Ambient mesh control.
# deploy bookinfo solution
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.23/samples/bookinfo/platform/kube/bookinfo.yaml
# use multiple versions of services
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.23/samples/bookinfo/platform/kube/bookinfo-versions.yaml
# wait for deployments to be ready
kubectl rollout status deployment productpage-v1 ratings-v1 reviews-v1 -n default --timeout=90s
# verify bookinfo app returning page with title (internal, from inside pod)
kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -sS productpage:9080/productpage | grep "<title>"
Create Gateway
Per the newer Gateway API, we now want to create the Gateway object that exposes the service. Once again, this does not put the app under Ambient mesh control, that will happen later when we properly label the default namespace.
# create 'bookinfo-gateway' Gateway and 'bookinfo' HttpRoute kubectl apply -f samples/bookinfo/gateway-api/bookinfo-gateway.yaml # wait for auto-created 'bookinfo-gateway-istio' waypoint proxy kubectl rollout status deployment bookinfo-gateway-istio -n default --timeout=90s # by default creates loadbalancer service, no address available $ kubectl get gateway NAME CLASS ADDRESS PROGRAMMED AGE bookinfo-gateway istio False 109s # this loadbalancer service is <pending>, but not necessary for us $ kubectl get service bookinfo-gateway-istio NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE bookinfo-gateway-istio LoadBalancer 10.96.238.79 <pending> 15021:32380/TCP,80:31433/TCP 33s # so change the service type to ClusterIP kubectl annotate gateway bookinfo-gateway networking.istio.io/service-type=ClusterIP --namespace=default # loadbalancer IP no longer awaiting creation with <pending> $ kubectl get service bookinfo-gateway-istio NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE bookinfo-gateway-istio ClusterIP 10.96.238.79 <none> 15021/TCP,80/TCP 2m18s # gateway now was internal service name $ kubectl get gateway NAME CLASS ADDRESS PROGRAMMED AGE bookinfo-gateway istio bookinfo-gateway-istio.default.svc.cluster.local True 2m44s
Test Gateway API
Istio waypoint proxy are deployed by Gateway resources. Use this to reach the service locally.
# forward Istio waypoint proxy to local port 8080 kubectl port-forward svc/bookinfo-gateway-istio 8080:80 # check with local browser, or another console curl -s http://localhost:8080/productpage | grep '<title>'
Looks like this in the browser:
Kiali, the Istio observability tool
Deploy Kiali
In order to better understand the application and its participation in the Ambient mode mesh, we will install Kiali the Istio observability dashboard.
# install kiali dashboard kubectl apply -f samples/addons kubectl rollout status deployment/kiali -n istio-system --timeout=90s
View sample app status in Kiali
# forwards local port and opens local browser to Kiali dashboard bin/istioctl dashboard kiali
Notice if we navigate to “Applications” in Kiali, the applications are listed as “Out of mesh”. This is because the applications have not been directed to participate in the Ambient mesh yet.
Enable Ambient mode for sample app
As shown in the previous section, the sample app is not participating in the Ambient mesh. We can enable this by adding an “istio.io/dataplane-mode” label to the default namespace.
# show current namespaces that are ambient mesh enabled (should be none) kubectl get ns default -L istio.io/dataplane-mode=ambient # make sample app part of the mesh with namespace label kubectl label namespace default istio.io/dataplane-mode=ambient
With that single namespace label, the sample “Book Info” sample app services are now participating in the Ambient service mesh. It did not require any restarts of the workloads.
Verify Ambient mode participation for sample app
If we now throw load at the service, we can go back to the Kiali web UI and validate they are participating in the Ambient mesh.
# forward service to local port kubectl port-forward svc/bookinfo-gateway-istio 8080:80 # throw load at app from a different console for i in $(seq 1 100); do curl -sS http://localhost:8080/productpage >/dev/null; done # open Kiali in local browser bin/istioctl dashboard kiali
Now we should be able to see that the “default” namespace is participating in the mesh, per the “Ambient” icon next to its name.
And in “Applications” the deployments no longer have the “out of mesh” message, proving they are indeed mesh enabled.
And from “Traffic Graph” the product page we put load against shows us its backing dependencies going to the details and reviews service.
REFERENCES
istio.io, Introducing Ambient Mesh
istio.io, ambient reaches beta, throw away your sidecar
istio.io, Getting started with Ambient Mesh
istio.io, Ambient Mesh and sample app
Billy Ridgway, Trying Ambient Mesh
github, Troubleshooting Ambient
karampok.me, chained CNI plugins
solo.io, ztunnel configuration
istio.io, network dual-stack IPv4/IPv6
istio.io, getting started (sample app)
Sabuj Jana, touring Ambient mesh
Praynay Singhal, istio proxy sidecar memory consumption problem because large service list
github issues, troubleshooting CNI pod to ztunnel route
NOTES
If CNI=calico, then Daemonset exists
kubectl describe ds calico-node -n kube-system
Validate required route from CNI pod to ztunnel pod, github issue
CNI_IP=$(kubectl get -n istio-system pod $(kubectl get pod -n istio-system -l k8s-app=istio-cni-node -o jsonpath='{.items..metadata.name}') -o jsonpath='{.status.podIP}')
ZTUN_IP=$(kubectl get -n istio-system pod $(kubectl get pod -n istio-system -l app=ztunnel -o jsonpath='{.items..metadata.name}') -o jsonpath='{.status.podIP}')
kubectl debug -it $(kubectl get pod -n istio-system -l k8s-app=istio-cni-node -oname) -n istio-system --image=busybox -- ip route get from $CNI_IP to $ZTUN_IP
Prometheus was also installed as part of this article
you can forward to a local port and open the browser using:
# https://istio.io/latest/docs/tasks/observability/metrics/querying-metrics/ bin/istioctl dashboard prometheus
Below is an example of a query against ‘istio_requests_total’
show kernel modules loaded and required by istio CNI, github issue
minikube ssh # operate as root sudo su # show currently loaded kernel modules lsmod # load modules required by istio CNI modprobe br_netfilter ; modprobe nf_nat ; modprobe xt_REDIRECT ; modprobe xt_owner; modprobe iptable_nat; modprobe iptable_mangle; modprobe iptable_filter
update kubeconfig for minikube
minikube update-context