The power of kustomize lies in its ability to transform yaml, and to that end it has built-in support for the JSON 6902 RFC specification. This RFC specifies the syntax for adding, removing, replacing, and moving elements and values in a yaml file. And since Kubernetes manifests are yaml files, it makes perfect sense to … Kubernetes: kustomize transformations with patchesJson6902
Kubernetes has a rich way of expressing volumes/ volumeMounts for mounting files, emptyDir for ephemeral directories, and env/envFrom for adding environment variables to your container definition running on a Kubernetes cluster. However, if you are actively iterating on the development of an image, it may slow you down to require a deployment to a remote … Kubernetes: volumeMount, emptyDir, and env equivalents during local Docker development
With kustomize built into the kubectl CLI since version 1.14, there is little reason not to take advantage of this overlay system to deploy components to your Kubernetes cluster. Kustomize has the advantage that it is purpose built to understand and validate yaml and Kubernetes CRD, as opposed to bespoke templating solutions using sed/envsubst, Ansible, … Kubernetes: kustomize overlay to enrich a base resource
At some point you may need to schedule a maintenance window for your solution But that doesn’t mean the end-user traffic or client integrations will stop requesting the services from the GCP external HTTPS LB that fronts all client requests. The VM instances and GKE clusters that normally respond to requests may not be able … GCP: Cloud Function to handle requests to HTTPS LB during maintenance
GCP Cloud Functions have taken a step forward with the 2nd generation release. One of the biggest architectural differences is that now multiple request can run concurrently on a single instance, enabling large traffic loads. In this article, I will show you how to deploy a simple Python Flask web server as a 2nd gen … GCP: Deploying a 2nd gen Python Cloud Function and exposing from an HTTPS LB
The Compute Engine default service account is automatically generated for your project with the Editor role, and by default is attached to all VM instances created in the project. You can pull the exact id using gcloud. gcloud iam service-accounts list –filter=”displayName:’Compute Engine default service account'” –format=’value(email)’ The syntax will be ${project_id}-compute@developer.gserviceaccount.com. If you want … GCP: VM instances running as the Compute Engine default service account
If you have unmanaged GCP VM instances running services on insecure ports (e.g. Apache HTTP on port 80), one way to secure the public external traffic is to create an external GCP HTTPS load balancer. Conceptually, we want to expose a secure front to otherwise insecure services. While the preferred method would be to secure … GCP: global external HTTPS LB for securely exposing insecure VM services
If you have unmanaged GCP VM instances running services on insecure ports (e.g. Apache HTTP on port 80), one way to secure the internal communication coming from other internal pods/apps is to create an internal GCP HTTPS load balancer. Conceptually, we want to expose a secure front to otherwise insecure services. While the preferred method … GCP: internal HTTPS LB for securely exposing insecure VM services
A Bash script can often run into the situation where a utility in the pipeline creates a file, but because of an unexpected error, the size of the resulting file is zero bytes. And if that situation is not prepared for, the mere existence of the file might signal success to the following commands in … Bash: test both file existence and size to avoid signalling success
No matter how highly available your services, there may still be significant backend events that require planned maintenance. During this downtime, you should still reply to end users and service integrations with a proper response. In this article, I will show you how to configure your GCP HTTPS Loadbalancer so that a single maintenance service … GCP: serving a maintenance page using an HTTPS LB and container native routing
If you need to delete a GKE instance from a node pool, you cannot simply treat the node as a raw VM instance. You must delete the VM instance from the managed instance group of which it is a member. A cluster will have an instance group for each region. In a zonal cluster, the … Kubernetes: deleting a GKE node from a managed instance node pool
If your intent is to delete all the objects in a namespace, but the command is not completing, emptying the namespace finalizer will often allow the deletion to finish. For example, if you have tried deleting the “my-namespace” like below and it will not complete. kubectl delete ns my-namespace –force –grace-period=0 Then as written by … Kubernetes: emptying the finalizers for a namespace that will not delete
If you are using GCP HTTPS LB to deliver your public services, be sure to apply an explicit SSL Policy that controls how TLS is negotiated with clients. Setting a SSL policy allows you to control minimum version of TLS as well as available cipher families. A basic SSL policy that limits clients to … GCP: enabling SSL policies on HTTPS LB Ingress
It is not necessary to create an independent GCP HTTPS LB or other improvisation to redirect insecure HTTP traffic to your HTTPS load balancer. The existing public Ingress can reference a FrontendConfig object that specifies redirection to HTTPS. Below is a FrontendConfig definition that can redirect the insecure traffic. apiVersion: networking.gke.io/v1beta1 kind: FrontendConfig metadata: name: … GCP: HTTP to HTTPS redirection using HTTPS LB Ingress
GKE Autopilot reduces the operational costs of managing GKE clusters by freeing you from node level maintenance, instead focusing just on pod workloads. Costs are accrued based on pod resource consumption and not on node resource sizes or node count, which are managed by Google. Since you no longer own the node level, there are … GCP: Private GKE cluster in Autopilot mode using Terraform
As opposed to public GKE clusters which have their IP addresses exposed, private GKE clusters use private internal IP addresses. This provides an enhanced security stance, but also means we need a solution such as Anthos Service Mesh to explicitly expose our services. In our previous article, we built a private GKE cluster using Terraform. … GCP: Private GKE Cluster with Anthos Service Mesh exposing services
As opposed to public GKE clusters which have their IP addresses exposed, private GKE clusters use private internal IP addresses that offer a level of security and segmentation that should always be preferred. In this article, I will show you how to create a private GKE cluster with Terraform. The endpoint is private also for … GCP: Private GKE Cluster with private endpoint using Terraform
If you are using Anthos Service Mesh to deliver your public applications from a GFE HTTPS LB, I would strongly suggest enabling Cloud Armor which is a WAF (web application firewall) that can mitigate and defend against a variety of attacks such as cross-site scripting and denial of service. As a summary overview, the first … GCP: enabling Cloud Armor on GCP HTTPS LB for Anthos Service Mesh
For bulk administration of hosts via ssh, there are powerful tools such as Ansible for the job. However, if you need to automate ssh for a one-off task, you may find the remote host systems throwing up interactive authentication prompts for sudo that need to be answered. If you need to run sudo commands that … Bash: automating ssh login and sudo that require interactive login
Killing a defunct zombie process means identifying and killing the parent process. In this article, I will show how you can force a zombie process so that you can practice removing it. A simple way of forcing a zombie process is to start a process that has a short sleep in the background, immediately invoking … Bash: identifying and killing a zombie child processes
Handling signals from within Bash has a simple syntax and can be used to handle user interruption with Ctrl-C (SIGINT), as a cleanup hook at the end of processing (SIGEXIT), or even as a custom messaging mechanism (SIGUSR1). Trapping a signal is as easy as specifying a custom function name and the signal name as … Bash: trapping signals during script processing
When calling processes within a script, there may be situations where you need to put a time limit on the call due to significant blocking times. Perhaps a DNS or network call would block if certain firewall rules do not exist yet, or a long-running API call would need to be considered in error if … Bash: using timeout to put time limit on invoked commands
If the logic in your Bash script needs to force the last exit code variable “$?”, you can accomplish that with an exit inside a subprocess. For example, here is a script snippet. $(exit 99) echo “last process exit code was $?” This would return the output below. last process exit code was 99 This … Bash: forcing the process exit code using a subshell
If you delete the entire nginx namespace and reinstall again via helm chart, your nginx admission controller may throw a “x509 certificate signed by unknown authority” message when you attempt to create an nginx ingress. This will happen regardless if the ingress is using http only or secure https. And also whether or not the … Kubernetes: ingress-nginx-controller-admission error, x509 certificate signed by unknown authority
Instead of deploying a pod or service and periodically checking its status for readiness, or having your automation scripts wait for a certain number of seconds before moving to the next operation, it is much cleaner to use ‘kubectl wait’ to sense completion. Wait for pod Here is how you would wait for READY status … Kubernetes: using kubectl to wait for condition of pods, deployments, services
The ‘export’ flag in kubectl was deprecated in 1.18, but you still need a way to dump the yaml of Kubernetes objects, often for the purpose of reapplying changes. The problem with doing a simple ‘get’ using a yaml output like below is that additional internal accounting attributes like ‘selfLink’, ‘creationTimestamp’, ‘uid’, will be present … Kubernetes: alternative to export for removing internal fields from yaml
The kubectl jsonpath has a built-in ‘range’ directive that allows you to iterate lists, and then extract elements from each list item. For example, if you wanted a list of all the pod names and their namespace in CSV format, you could use the following: # pods and their namespace with just kubectl kubectl get … Kubernetes: jsonpath range to iterate list and extract fields
Update Aug 2023: using newer ‘signed-by’ attribute for apt signing keys. Installing Helm using apt is a straight-forward procedure and documented on the official site. Coming straight from the official helm documentation, here are the commands for Ubuntu 22. curl https://baltocdn.com/helm/signing.asc | gpg –dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null sudo chmod 644 /usr/share/keyrings/helm.gpg sudo … Helm: Installing Helm on Ubuntu
If you are in the development lifecycle and need to quickly test email functionality, you can deploy the codecentric/mailhog image directly within Kubernetes. It will receive all email regardless of address, and from its web interface show you all the email that has been received. In this article, I will show you how to deploy … Kubernetes: running a mail container for testing email during development
If you have an external NFS export and want to share that with a pod/deployment, you can leverage the nfs-subdir-external-provisioner to create a storageclass that can dynamically create the persistent volume. In contrast to manually creating the persistent volume and persistent volume claim, this dynamic method cedes the lifecycle of the persistent volume over to … Kubernetes: NFS mount using dynamic volume and Storage Class
If you have an external NFS server and want to share that volume in RWX mode (ReadWriteMany), the most basic way is to manually create the persistent volume and persistent volume claim. In this article, I will show you how to manually create a pv (persistent volume) representing an external NFS, and persistent volume claim … Kubernetes: ReadWriteMany (RWX) NFS mount using static volume
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. In this article, I will be following the steps required to upgrade from 1.9 to 1.10 on VMware. The instructions provided here are assuming you have used the Ansible scripts and Seed VM described in my previous Anthos 1.9 installation article.
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a platform this extensive requires many steps as officially documented here. However, if you want to practice in a … Kubernetes: Anthos GKE on-prem 1.10 on nested VMware environment
If you need to batch rename a set of files, the ‘rename‘ utility is much easier than alternatives such as pairing find/exec or even putting together a small script. The rename utility gives you all the power of Perl regular expressions for converting the filenames. # make sure utility is installed $ sudo apt install … Bash: batch renaming files using the rename utility
GitHub has a feature that allows other members to follow your account and receive alerts on actions you take in any repository. Unfortunately, there is not a setting that allows you to disable this at a global level. The only workaround is to temporarily block the user, which removes the relationship. This can be done … GitHub: removing followers with the GitHub api
If you have a Bash script that needs to run periodically, you can run it using a crontab entry. But you can also have it invoked by Systemd using systemd.timer. Furthermore, you can run Systemd services as user-level services instead of the typical system-level service for even further isolation. Running via Systemd provides more powerful … Ubuntu: Running a bash script periodically with a user-level Systemd timer
If you have a Bash script that needs to run periodically, you can run it using a crontab entry or file. But you can also have it invoked from Systemd using systemd.timer. Running via Systemd provides more powerful constructs for invocation, configuration, monitoring, and logging. In this article, I will show how to periodically run … Ubuntu: Running a bash script periodically with a system-level Systemd timer
If your VSCode terminal does not contain the directory location you need in its PATH, you can add it locally to the VSCode settings and it will remain persistent every time you open the IDE. From the main menu, go to File > Preferences > Settings, and click on the “Open Settings (JSON)” icon shown … VSCode: Add a directory to the terminal PATH
If the dataset you want to analyze with pandas is coming from a normalized relational database, then you can use ‘pandas.read_sql‘ to pull the data directly. In this article, we will deploy a small MariaDB instance with Docker and show how we can create DataFrame directly from a single table or from a join between … Python: constructing a DataFrame from a relational database with pandas
Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want … Docker: Installing Docker CE on Ubuntu focal 20.04