Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want … Docker: Installing Docker CE on Ubuntu focal 20.04
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. In this article, I will be following the steps required to perform a minor-version upgrade from 1.9.1 to 1.9.2 on VMware. I will be using the same environment and config files described in my Anthos 1.9 installation article.
If your data is coming from CSV dumps of relational database tables, then you probably need to flatten the data before starting the training or analysis. It requires an understanding of the relational schema so you can specify the foreign key lookups, but from that pandas can reconstruct the relationships and produce a single flattened … Python: flattening exported csv relational data with pandas
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. In this article, I will be following the steps required to upgrade from 1.8 to 1.9 on VMware. The instructions provided here are assuming you have used the Ansible scripts and Seed VM described in my previous Anthos 1.8 installation article.
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a platform this extensive requires many steps as officially documented here. However, if you want to practice in a … Kubernetes: Anthos GKE on-prem 1.9 on nested VMware environment
Update Dec 2021: I have written an updated version of this article for vCenter 7.0U1 and Anthos 1.9. Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a … Kubernetes: Anthos GKE on-prem 1.8 on nested VMware environment
Although there are Python modules [1,2,3] specially suited for displaying text to the console in color, if you want a quick no-dependency method then you can use ANSI color codes directly. Here is an example of printing a line in green, then red. print(“\033[0;32mOK this is green\033[00m”) print(“\033[0;31mERROR this is red\033[00m”) Additional color codes can … Python: printing in color using ANSI color codes
If you have just virtualized the VMware ESXi server on top of KVM, the next step will be to install vCenter for its centralized control and additional feature set and management capabilities. In my last article we took KVM running on bare metal and deployed an ESXi 7.0 host on top of it. In this … KVM: Deploy the VMware vCenter 7.0 appliance using the CLI installer
If you are running KVM on an Ubuntu server, you already have an excellent Type 1 virtualization engine. Luckily, if you need to test something specific to VMware you can always run ESXi 7.0 nested inside a KVM virtual machine. In this article, I’ll be using a bare metal server running Ubuntu and KVM as … KVM: Deploying a nested version of VMware ESXi 7.0 on KVM
If you have a complex JSON data structure or perhaps an array of rich data structures as the result of module output, you may want to extract this into a workable list or dictionary you can take action on. For this article, I will emulate the resulting output from the ‘stat‘ module when it checks … Ansible: extracting a list or dictionary from a complex data structure
Whether it is the most recent log file, image, or report – sometimes you will need to find the most recently modified file in a directory. The example below finds the latest file in the “/tmp” directory. import os import glob # get list of files that matches pattern pattern=”/tmp/*” files = list(filter(os.path.isfile, glob.glob(pattern))) # … Python: find the most recently modified file matching a pattern
If you have a file with special characters (single quotes, wildcard, etc) in the name, it can be difficult to discover the exact escape sequence to correctly delete. To avoid playing with escape characters, you can simply use the inode number of the file instead. For example, let’s say you accidentally specify tar options incorrectly … Bash: deleting a file with special characters using its inode value
Most of the modern cloud platforms and utilities have us manipulate either JSON or YAML configuration files. And when you start dealing with real world scenarios with hundreds of lines of embedded data structures it is too difficult and error-prone to manually inspect indentation levels to determine the exact dotted or json path to an … Python: converting JSON to dot notation for easier path determination
Managing certificates is one of the most mundane, yet critical chores in the maintenance of environments. However, this manual maintenance can be off-loaded to cert-manager on Kubernetes. In this article, we will use cert-manager to generate TLS certs for a public NGINX ingress using Let’s Encrypt. The primary ingress will have two different hosts using … Kubernetes: LetsEncrypt certificates using HTTP and DNS solvers on DigitalOcean
Updated Aug 2023: tested with Kubernetes 1.25 and ingress-nginx 1.8.1 Creating a Kubernetes cluster on DigitalOcean can be done manually using its web Control Panel, but for automation purposes it is better to use Terraform. In this article, we will use Terraform to create a Kubernetes cluster on DigitalOcean infrastructure. We will then use helm … Terraform: creating a Kubernetes cluster on DigitalOcean with public NGINX ingress
If you are creating a VM resource and must run a Bash script as part of the initialization, that can be done within Terraform using the remote-exec provisioner and its ability to execute scripts via ssh. If you need to send arguments to this script, there is a standard pattern described in the official documentation … Terraform: post-configuration by calling remote-exec script with parameters
If the Terraform resource you are creating supports multiple dependent entities (e.g. a single VM with multiple disks or networks), but only by adding hardcoded duplicate text blocks, then you should consider Terraform dynamic blocks. For example, if you are creating a vsphere_virtual_machine with two additional data disks, then here is a snippet showing how … Terraform: using dynamic blocks to add multiple disks on a vsphere_virtual_machine
Specifying input variables in the “terraform.tfvars” file in HCL syntax is commonly understood. But if the values you need are already coming from a json source, it might make more sense to feed those directly to Terraform. Here is an example where the simple variable “a” is provided via an external json file. # … Terraform: using json files as input variables and local variables
If you are using a Terraform “for_each” and get the error message below, it is most likely because you are sending an ordered list instead of an unordered set (which is not supported at the resource level). The given “for_each” argument value is unsuitable: the “for_each” argument must be a map, or set of strings, … Terraform: converting ordered lists to sets to avoid errors with for_each
By default, ‘qemu-image info’ will throw an error if it cannot get exclusive access to the disk file it is trying to read. $ sudo qemu-img info mydisk.qcow2 qemu-img: Could not open ‘mydisk.qcow2’: Failed to get shared “write” lock Is another process using the image [mydisk.qcow2]? Although it is not listed in the man page, … KVM: running qemu-img info without exclusive access using force-share flag
Istio announced it will support upgrades jumping directly from 1.8 to 1.10, instead of forcing an intermediate upgrade through 1.9. In this article, I will show you how to do a canary upgrade from a 1.8 operator to 1.10 operator without affecting end user traffic. We will incorporate the new 1.10 concept of revision tags … Istio: Canary upgrade of Operator from Istio 1.8 directly to 1.10
Istio 1.7 has the ability to do canary upgrades for revisioned control planes and operators. In this article, I will show you how to do a canary upgrade of the Istio control plane between minor versions without affecting end user traffic.
Istio 1.7 has the ability to do canary upgrades for revisioned control planes and operators, but if you did your initial installation without the ‘revision’ flag, then you’ll need to apply these settings. In this article, I will show you how to go from an non-revisioned 1.7.5 Istio operator and control plane to a 1.7.5 … Istio: Upgrading from Istio 1.7 operator without revision to fully revisioned control plane
Istio has the ability to do canary upgrades for revisioned control planes, but it was only in 1.7 that the Operator itself got support for the ‘revision’ flag. In this article, I will show you how to go from an non-revisioned 1.6.6 Istio operator and control plane to a 1.7 revisioned operator and control plane … Istio: Upgrading from Istio 1.6 operator without revision to 1.7 fully revisioned control plane
kubectl will give you a sythesized column showing how many container instances in a pod are READY with the default ‘get pods’ command. But if you are dealing with json output and need this information, then you can extract it using jsonpath or jq. Here is an example output from ‘get pods’ showing the READY … Kubernetes: pulling out the ready status of individual containers using kubectl
Although ‘kubectl annotate‘ will set an annotation on a object directly, it will not set the annotation on the more deeply nested pod template for a Deployment or Daemonset. If you want to quickly set the annotation on a pod template (.spec.template.metadata.annotations) without modifying the full manifest, you can use the ‘patch‘ command. As a … Kubernetes: adding and removing pod template annotations using kubectl
By default, K3s uses the Traefik ingress controller and Klipper service load balancer to expose services. But this can be replaced with a MetalLB load balancer and Istio ingress controller. K3s is perfectly capable of handling Istio operators, gateways, and virtual services if you want the advanced policy, security, and observability offered by Istio. In … Kubernetes: K3s with multiple Istio ingress gateways
Updated March 2023: using K3s 1.26 and MetalLB 0.13.9 By default, K3s uses the Traefik ingress controller and Klipper service load balancer to expose services. But this can be replaced with a MetalLB load balancer and NGINX ingress controller. But a single NGINX ingress controller is sometimes not sufficient. For example, the primary ingress may … Kubernetes: K3s with multiple metalLB endpoints and nginx ingress controllers
Updated Aug 2023: using the latest K3s 1.27, MetalLB 0.13.9, and NGINX v1.8.1 K3s is a lightweight Kubernetes deployment by Rancher that is fully compliant, yet also compact enough to run on development boxes and edge devices. In this article, I will show you how to deploy a three-node K3s cluster on Ubuntu nodes that … Kubernetes: K3s cluster on Ubuntu using Ansible
Updated March 2023: using K3s 1.25 and Ubuntu 22.04 K3s is a lightweight Kubernetes deployment by Rancher that is fully compliant, yet also compact enough to run on development boxes and edge devices. In this article, I will show you how to deploy a three-node K3s cluster on Ubuntu nodes that are created using Terraform … Kubernetes: K3s cluster on Ubuntu using terraform and libvirt
For security reasons, you should be very aware that accepting a remote host fingerprint automatically is a procedure that should be considered high-risk. But if you are working with automated infrastructure or pipelines where human intervention is not possible and the constructed entities are being built in a secure fashion with guaranteed provenance, then ssh-keyscan … Bash: accepting a remote host fingerprint with ssh-keyscan
If you want to create a new datastore backed by an NFSv4 share, this can be done either from the ESXi embedded web client or using esxcli. In this article, I will explain how to test a remote NFSv4 volume from an Ubuntu Linux server as validation. Then move on to mounting it as a … VMware: mounting a new NFSv4 datastore
If you followed my previous article on deploying a nested ESXi on KVM, you may eventually need to increase the size of the default datastore1 as you continue your experimentation and deployments. Do not simply increase the size of the original backing KVM qcow2 disk (qemu-img resize), because then you would need to get this … VMware: Extending datastore1 on a ESXi host nested within KVM
If you have captured the multi-line output of a previous Bash command in a variable and later need to count the number of lines stored in that variable, you may reach for an apparently easy solution first. But there are corner cases you may not have covered. The first syntax gotcha is to make sure … Bash: count number of lines in previously captured stdout
Update Dec 2021: I have written an updated version of this article for vCenter 7.0U1 and Anthos 1.8. Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a … Kubernetes: Anthos GKE on-prem 1.4 on nested VMware environment
In this article I will demonstrate how to create an Ubuntu 20 Focal template in vCenter. Then use Terraform to create a vSphere VM based on this template. The VM template creation is done by manually stepping through an installation using the minimal Ubuntu server ISO followed by a set of preparation steps. Then Terraform … Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter
Needing to find the most recently modified files in a directory is a pretty common need. Luckily the find utility has flags to easily explore a directory recursively and list recently modified files. If you want to find modified files within ‘N’ days ago from the current directory. # files within the last 24 hours … Bash: find most recently modified files
microk8s has convenient out-of-the-box support for MetalLB and an NGINX ingress controller. But microk8s is also perfectly capable of handling Istio operators, gateways, and virtual services if you want the advanced policy, security, and observability offered by Istio. In this article, we will install the Istio Operator, and allow it to create the Istio Ingress … Kubernetes: microk8s with multiple Istio ingress gateways
Out-of-the-box, microk8s has add-ons that make it easy to enable MetalLB as a network load balancer as well as an NGINX ingress controller. But a single ingress controller is often not sufficient. For example, the primary ingress may be serving up all public traffic to your customers. But a secondary ingress might be necessary to … Kubernetes: microk8s with multiple metalLB endpoints and nginx ingress controllers
If you are using “–extra-vars” at runtime to override a boolean variable, then you should use the JSON syle (instead of key-value pairs) to avoid type conversion issues. For example, if there is a boolean variable named “myflag” that is default to true in your playbook and you attempt to override it like below, it … Ansible: overriding boolean values using extra-vars at runtime