Handling signals from within Bash has a simple syntax and can be used to handle user interruption with Ctrl-C (SIGINT), as a cleanup hook at the end of processing (SIGEXIT), or even as a custom messaging mechanism (SIGUSR1). Trapping a signal is as easy as specifying a custom function name and the signal name as … Bash: trapping signals during script processing
When calling processes within a script, there may be situations where you need to put a time limit on the call due to significant blocking times. Perhaps a DNS or network call would block if certain firewall rules do not exist yet, or a long-running API call would need to be considered in error if … Bash: using timeout to put time limit on invoked commands
If the logic in your Bash script needs to force the last exit code variable “$?”, you can accomplish that with an exit inside a subprocess. For example, here is a script snippet. $(exit 99) echo “last process exit code was $?” This would return the output below. last process exit code was 99 This … Bash: forcing the process exit code using a subshell
If you delete the entire nginx namespace and reinstall again via helm chart, your nginx admission controller may throw a “x509 certificate signed by unknown authority” message when you attempt to create an nginx ingress. This will happen regardless if the ingress is using http only or secure https. And also whether or not the … Kubernetes: ingress-nginx-controller-admission error, x509 certificate signed by unknown authority
Instead of deploying a pod or service and periodically checking its status for readiness, or having your automation scripts wait for a certain number of seconds before moving to the next operation, it is much cleaner to use ‘kubectl wait’ to sense completion. Wait for pod Here is how you would wait for READY status … Kubernetes: using kubectl to wait for condition of pods, deployments, services
The ‘export’ flag in kubectl was deprecated in 1.18, but you still need a way to dump the yaml of Kubernetes objects, often for the purpose of reapplying changes. The problem with doing a simple ‘get’ using a yaml output like below is that additional internal accounting attributes like ‘selfLink’, ‘creationTimestamp’, ‘uid’, will be present … Kubernetes: alternative to export for removing internal fields from yaml
The kubectl jsonpath has a built-in ‘range’ directive that allows you to iterate lists, and then extract elements from each list item. For example, if you wanted a list of all the pod names and their namespace in CSV format, you could use the following: # pods and their namespace with just kubectl kubectl get … Kubernetes: jsonpath range to iterate list and extract fields
Update Aug 2023: using newer ‘signed-by’ attribute for apt signing keys. Installing Helm using apt is a straight-forward procedure and documented on the official site. Coming straight from the official helm documentation, here are the commands for Ubuntu 22. curl https://baltocdn.com/helm/signing.asc | gpg –dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null sudo chmod 644 /usr/share/keyrings/helm.gpg sudo … Helm: Installing Helm on Ubuntu
If you are in the development lifecycle and need to quickly test email functionality, you can deploy the codecentric/mailhog image directly within Kubernetes. It will receive all email regardless of address, and from its web interface show you all the email that has been received. In this article, I will show you how to deploy … Kubernetes: running a mail container for testing email during development
If you have an external NFS export and want to share that with a pod/deployment, you can leverage the nfs-subdir-external-provisioner to create a storageclass that can dynamically create the persistent volume. In contrast to manually creating the persistent volume and persistent volume claim, this dynamic method cedes the lifecycle of the persistent volume over to … Kubernetes: NFS mount using dynamic volume and Storage Class
If you have an external NFS server and want to share that volume in RWX mode (ReadWriteMany), the most basic way is to manually create the persistent volume and persistent volume claim. In this article, I will show you how to manually create a pv (persistent volume) representing an external NFS, and persistent volume claim … Kubernetes: ReadWriteMany (RWX) NFS mount using static volume
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. In this article, I will be following the steps required to upgrade from 1.9 to 1.10 on VMware. The instructions provided here are assuming you have used the Ansible scripts and Seed VM described in my previous Anthos 1.9 installation article.
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a platform this extensive requires many steps as officially documented here. However, if you want to practice in a … Kubernetes: Anthos GKE on-prem 1.10 on nested VMware environment
If you need to batch rename a set of files, the ‘rename‘ utility is much easier than alternatives such as pairing find/exec or even putting together a small script. The rename utility gives you all the power of Perl regular expressions for converting the filenames. # make sure utility is installed $ sudo apt install … Bash: batch renaming files using the rename utility
GitHub has a feature that allows other members to follow your account and receive alerts on actions you take in any repository. Unfortunately, there is not a setting that allows you to disable this at a global level. The only workaround is to temporarily block the user, which removes the relationship. This can be done … GitHub: removing followers with the GitHub api
If you have a Bash script that needs to run periodically, you can run it using a crontab entry. But you can also have it invoked by Systemd using systemd.timer. Furthermore, you can run Systemd services as user-level services instead of the typical system-level service for even further isolation. Running via Systemd provides more powerful … Ubuntu: Running a bash script periodically with a user-level Systemd timer
If you have a Bash script that needs to run periodically, you can run it using a crontab entry or file. But you can also have it invoked from Systemd using systemd.timer. Running via Systemd provides more powerful constructs for invocation, configuration, monitoring, and logging. In this article, I will show how to periodically run … Ubuntu: Running a bash script periodically with a system-level Systemd timer
If your VSCode terminal does not contain the directory location you need in its PATH, you can add it locally to the VSCode settings and it will remain persistent every time you open the IDE. From the main menu, go to File > Preferences > Settings, and click on the “Open Settings (JSON)” icon shown … VSCode: Add a directory to the terminal PATH
If the dataset you want to analyze with pandas is coming from a normalized relational database, then you can use ‘pandas.read_sql‘ to pull the data directly. In this article, we will deploy a small MariaDB instance with Docker and show how we can create DataFrame directly from a single table or from a join between … Python: constructing a DataFrame from a relational database with pandas
Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want … Docker: Installing Docker CE on Ubuntu focal 20.04
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. In this article, I will be following the steps required to perform a minor-version upgrade from 1.9.1 to 1.9.2 on VMware. I will be using the same environment and config files described in my Anthos 1.9 installation article.
If your data is coming from CSV dumps of relational database tables, then you probably need to flatten the data before starting the training or analysis. It requires an understanding of the relational schema so you can specify the foreign key lookups, but from that pandas can reconstruct the relationships and produce a single flattened … Python: flattening exported csv relational data with pandas
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. In this article, I will be following the steps required to upgrade from 1.8 to 1.9 on VMware. The instructions provided here are assuming you have used the Ansible scripts and Seed VM described in my previous Anthos 1.8 installation article.
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a platform this extensive requires many steps as officially documented here. However, if you want to practice in a … Kubernetes: Anthos GKE on-prem 1.9 on nested VMware environment
Update Dec 2021: I have written an updated version of this article for vCenter 7.0U1 and Anthos 1.9. Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a … Kubernetes: Anthos GKE on-prem 1.8 on nested VMware environment
Although there are Python modules [1,2,3] specially suited for displaying text to the console in color, if you want a quick no-dependency method then you can use ANSI color codes directly. Here is an example of printing a line in green, then red. print(“\033[0;32mOK this is green\033[00m”) print(“\033[0;31mERROR this is red\033[00m”) Additional color codes can … Python: printing in color using ANSI color codes
If you have just virtualized the VMware ESXi server on top of KVM, the next step will be to install vCenter for its centralized control and additional feature set and management capabilities. In my last article we took KVM running on bare metal and deployed an ESXi 7.0 host on top of it. In this … KVM: Deploy the VMware vCenter 7.0 appliance using the CLI installer
If you are running KVM on an Ubuntu server, you already have an excellent Type 1 virtualization engine. Luckily, if you need to test something specific to VMware you can always run ESXi 7.0 nested inside a KVM virtual machine. In this article, I’ll be using a bare metal server running Ubuntu and KVM as … KVM: Deploying a nested version of VMware ESXi 7.0 on KVM
If you have a complex JSON data structure or perhaps an array of rich data structures as the result of module output, you may want to extract this into a workable list or dictionary you can take action on. For this article, I will emulate the resulting output from the ‘stat‘ module when it checks … Ansible: extracting a list or dictionary from a complex data structure
Whether it is the most recent log file, image, or report – sometimes you will need to find the most recently modified file in a directory. The example below finds the latest file in the “/tmp” directory. import os import glob # get list of files that matches pattern pattern=”/tmp/*” files = list(filter(os.path.isfile, glob.glob(pattern))) # … Python: find the most recently modified file matching a pattern
If you have a file with special characters (single quotes, wildcard, etc) in the name, it can be difficult to discover the exact escape sequence to correctly delete. To avoid playing with escape characters, you can simply use the inode number of the file instead. For example, let’s say you accidentally specify tar options incorrectly … Bash: deleting a file with special characters using its inode value
Most of the modern cloud platforms and utilities have us manipulate either JSON or YAML configuration files. And when you start dealing with real world scenarios with hundreds of lines of embedded data structures it is too difficult and error-prone to manually inspect indentation levels to determine the exact dotted or json path to an … Python: converting JSON to dot notation for easier path determination
Managing certificates is one of the most mundane, yet critical chores in the maintenance of environments. However, this manual maintenance can be off-loaded to cert-manager on Kubernetes. In this article, we will use cert-manager to generate TLS certs for a public NGINX ingress using Let’s Encrypt. The primary ingress will have two different hosts using … Kubernetes: LetsEncrypt certificates using HTTP and DNS solvers on DigitalOcean
Updated Aug 2023: tested with Kubernetes 1.25 and ingress-nginx 1.8.1 Creating a Kubernetes cluster on DigitalOcean can be done manually using its web Control Panel, but for automation purposes it is better to use Terraform. In this article, we will use Terraform to create a Kubernetes cluster on DigitalOcean infrastructure. We will then use helm … Terraform: creating a Kubernetes cluster on DigitalOcean with public NGINX ingress
If you are creating a VM resource and must run a Bash script as part of the initialization, that can be done within Terraform using the remote-exec provisioner and its ability to execute scripts via ssh. If you need to send arguments to this script, there is a standard pattern described in the official documentation … Terraform: post-configuration by calling remote-exec script with parameters
If the Terraform resource you are creating supports multiple dependent entities (e.g. a single VM with multiple disks or networks), but only by adding hardcoded duplicate text blocks, then you should consider Terraform dynamic blocks. For example, if you are creating a vsphere_virtual_machine with two additional data disks, then here is a snippet showing how … Terraform: using dynamic blocks to add multiple disks on a vsphere_virtual_machine
Specifying input variables in the “terraform.tfvars” file in HCL syntax is commonly understood. But if the values you need are already coming from a json source, it might make more sense to feed those directly to Terraform. Here is an example where the simple variable “a” is provided via an external json file. # … Terraform: using json files as input variables and local variables
If you are using a Terraform “for_each” and get the error message below, it is most likely because you are sending an ordered list instead of an unordered set (which is not supported at the resource level). The given “for_each” argument value is unsuitable: the “for_each” argument must be a map, or set of strings, … Terraform: converting ordered lists to sets to avoid errors with for_each
By default, ‘qemu-image info’ will throw an error if it cannot get exclusive access to the disk file it is trying to read. $ sudo qemu-img info mydisk.qcow2 qemu-img: Could not open ‘mydisk.qcow2’: Failed to get shared “write” lock Is another process using the image [mydisk.qcow2]? Although it is not listed in the man page, … KVM: running qemu-img info without exclusive access using force-share flag
Istio announced it will support upgrades jumping directly from 1.8 to 1.10, instead of forcing an intermediate upgrade through 1.9. In this article, I will show you how to do a canary upgrade from a 1.8 operator to 1.10 operator without affecting end user traffic. We will incorporate the new 1.10 concept of revision tags … Istio: Canary upgrade of Operator from Istio 1.8 directly to 1.10