Handling signals from within Bash has a simple syntax and can be used to handle user interruption with Ctrl-C (SIGINT), as a cleanup hook at the end of processing (SIGEXIT), or even as a custom messaging mechanism (SIGUSR1). Trapping a signal is as easy as specifying a custom function name and the signal name as … Bash: trapping signals during script processing
The ‘export’ flag in kubectl was deprecated in 1.18, but you still need a way to dump the yaml of Kubernetes objects, often for the purpose of reapplying changes. The problem with doing a simple ‘get’ using a yaml output like below is that additional internal accounting attributes like ‘selfLink’, ‘creationTimestamp’, ‘uid’, will be present … Kubernetes: alternative to export for removing internal fields from yaml
If you need to batch rename a set of files, the ‘rename‘ utility is much easier than alternatives such as pairing find/exec or even putting together a small script. The rename utility gives you all the power of Perl regular expressions for converting the filenames. # make sure utility is installed $ sudo apt install … Bash: batch renaming files using the rename utility
If you have a Bash script that needs to run periodically, you can run it using a crontab entry. But you can also have it invoked by Systemd using systemd.timer. Furthermore, you can run Systemd services as user-level services instead of the typical system-level service for even further isolation. Running via Systemd provides more powerful … Ubuntu: Running a bash script periodically with a user-level Systemd timer
If you have a Bash script that needs to run periodically, you can run it using a crontab entry or file. But you can also have it invoked from Systemd using systemd.timer. Running via Systemd provides more powerful constructs for invocation, configuration, monitoring, and logging. In this article, I will show how to periodically run … Ubuntu: Running a bash script periodically with a system-level Systemd timer
Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want … Docker: Installing Docker CE on Ubuntu focal 20.04
If you have just virtualized the VMware ESXi server on top of KVM, the next step will be to install vCenter for its centralized control and additional feature set and management capabilities. In my last article we took KVM running on bare metal and deployed an ESXi 7.0 host on top of it. In this … KVM: Deploy the VMware vCenter 7.0 appliance using the CLI installer
If you are running KVM on an Ubuntu server, you already have an excellent Type 1 virtualization engine. Luckily, if you need to test something specific to VMware you can always run ESXi 7.0 nested inside a KVM virtual machine. In this article, I’ll be using a bare metal server running Ubuntu and KVM as … KVM: Deploying a nested version of VMware ESXi 7.0 on KVM
If you have a file with special characters (single quotes, wildcard, etc) in the name, it can be difficult to discover the exact escape sequence to correctly delete. To avoid playing with escape characters, you can simply use the inode number of the file instead. For example, let’s say you accidentally specify tar options incorrectly … Bash: deleting a file with special characters using its inode value
For security reasons, you should be very aware that accepting a remote host fingerprint automatically is a procedure that should be considered high-risk. But if you are working with automated infrastructure or pipelines where human intervention is not possible and the constructed entities are being built in a secure fashion with guaranteed provenance, then ssh-keyscan … Bash: accepting a remote host fingerprint with ssh-keyscan
If you want to create a new datastore backed by an NFSv4 share, this can be done either from the ESXi embedded web client or using esxcli. In this article, I will explain how to test a remote NFSv4 volume from an Ubuntu Linux server as validation. Then move on to mounting it as a … VMware: mounting a new NFSv4 datastore
In this article I will demonstrate how to create an Ubuntu 20 Focal template in vCenter. Then use Terraform to create a vSphere VM based on this template. The VM template creation is done by manually stepping through an installation using the minimal Ubuntu server ISO followed by a set of preparation steps. Then Terraform … Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter
Needing to find the most recently modified files in a directory is a pretty common need. Luckily the find utility has flags to easily explore a directory recursively and list recently modified files. If you want to find modified files within ‘N’ days ago from the current directory. # files within the last 24 hours … Bash: find most recently modified files
If you need to create a file that has the exact same ownership and permission bits as an existing file, the ‘reference’ flag provides a convenient shortcut. For example, if you had a file named ‘myoriginal’ that had the exact ownership and permissions required for a new file ‘mynewfile’, you could use the commands below … Bash: cloning the ownership and permissions of another file using reference
Terraform is a popular tool for provisioning infrastructure on cloud providers such as EC2 and Azure, but there is also a provider written for local KVM libvirt resources. Using the libvirt provider, we can use standard Terraform constructs to create local VMs, networks, and disks. And unlike older versions of this provider, the plugin binary … KVM: installing Terraform and the libvirt provider for local KVM resources
One way to implement character padding in Bash is to use printf and substring extraction. This can be especially useful in reports or menu display. Given a $padding variable that contains the maximum length of characters, you can subtract out the length of a display string like below. # length of maximum padding padding=”………………………………..” printf … Bash: using printf to display fixed-width padded string
WireGuard aims to be the successor to IPsec and more performant than OpenVPN. It is a general purpose VPN that is secure enough for servers, yet light enough to run on embedded devices. In a previous article I described the installation of a WireGuard server with a Windows VPN client. I will extend those instructions … Ubuntu: WireGuard VPN for Ubuntu server, with an iPhone client
WireGuard aims to be the successor to IPsec and more performant than OpenVPN. It is a general purpose VPN that is secure enough for servers, yet light enough to run on embedded devices. In this article, I will show how to install WireGuard on two Ubuntu servers in completely different hyperscalers that are linked by … Ubuntu: site-to-site VPN with WireGuard
It is relatively straightforward to create a GCP public subnet where the compute instances have access to the public internet via the default internet gateway. But once you start building private subnets behind it, you must start considering firewall, routing, and the NAT gateways required to reach public services. In this article, I will use … Terraform: provisioning GCP servers in both public and private subnets
WireGuard aims to be the successor to IPsec and more performant than OpenVPN. It is a general purpose VPN that is secure enough for servers, yet light enough to run on embedded devices. In this article, I will show how to install WireGuard on an Ubuntu server and then access it using a Windows client. … Ubuntu: WireGuard VPN for Ubuntu servers, with a Windows client
If you are using vim on a constrained system (container or production system), you may not have the ability to setup vim plugin managers and plugins to mimic a full development environment. However, with a few basic settings in ‘~/.vimrc’, you can get a reasonable rendering that will assist in troubleshooting. ” find list using: … Bash: minimal .vimrc settings for python and yaml editing
By default, if an SSH key file is dropped into your personal ‘~/.ssh’ directory that matches a set of standard names, then it will automatically be used as an identity when logging into a remote site (id_rsa, id_dsa, id_ecsda, id_ed25519, or identity). For example, this makes it simple to comply with Github’s requirement to use … Ubuntu: loading a key into ssh-agent at login with a user-level systemd service
Using ldapsearch to query against the insecure port of a Windows Domain Controller is straightforward. However, it can be challenging to get all the pieces in place for a production environment where the secure port must be used and the root CA certificate is typically not from a public CA. Assuming the standard insecure port … Ubuntu: using ldapsearch to query against a secure Windows Domain Controller
It is common for a virtualized Guest OS base image to have a generic storage capacity. This capacity can easily be exceeded by production scenarios, performance testing, logging, or even the general cruft of running a machine 24×7. If your VM is using Logical Volume Management (LVM), adding space can be done by following a … Ubuntu: Extending capacity of an LVM volume group using an existing or new disk
Update July 2021: I have seen errors with external snapshots of volumes on versions of QEMU/KVM/libvirt from Ubuntu 20 Focal. Adding note on using internal snapshot on volume backed by qcow2. Internal snapshots created on QEMU copy-on-write (qcow2) disks are the most commonly used snapshot when using libvirt. It is easy to see why; … KVM: creating and reverting libvirt external snapshots
A centralized authentication/authorization mechanism is a key part of a security stance as well as a convenience to end users. There are multiple packages and systems to achieve this, and in this article I will focus on integrating back into Windows Active Directory using SSSD for login and group membership. When complete, you will have … Ansible: Login to Ubuntu with Windows Active Directory using SSSD
Even where there is an existing git pull request (PR) addressing a bug fix or feature you might find critical for your use case, it is not uncommon for it to be awaiting a merge by the project owner. As an example, the hellofresh company has done a great service to the community offering up … Git: Incorporating multiple pull requests from the main project into your fork
Because of the inherited nature of .gitignore, it can be tricky to determine if you have any files in your repository folder that are “hiding” from source control without your knowledge. This may be in the form of files that should be under source control but got caught in a broad inherited .gitignore rule, or … Git: Identifying files that .gitignore is purposely skipping
File encoding issues are difficult to diagnose and troubleshoot. Most files in the operations world are expected to be text-only ASCII 7 bit, so if a file goes into UTF-8 encoding and has embedded Unicode character inserted, it can often throw off the tool chain or systems using the file. Here are two example files, … Bash: Fixing an ASCII text file changed with Unicode character sequences
In a production environment where a proxy such as squid is necessary to reach the public internet, you may need use apt-add-repository to install a package from a ppa. Using the Ansible ppa as an example, the command from the documentation is: sudo apt-add-repository –yes –update ppa:ansible/ansible You might be tempted to believe that if … Ubuntu: Using add-apt-repository with a proxy
Desktop screen sharing is ubiquitous, but there is still a lot of value in sharing a live terminal screen with another set of users that can not only view, but also have the ability to copy-paste and even provide input. The tmux utility can create a session using a shared socket. To start a tmux … Bash: Sharing a terminal screen among users with tmux
Bash has a rich set of control structures but it is still common to see logic expressions used a shorthand for if-then-else/tertiary statements. This can provide a curt and single-line ingestable way of controlling the flow of your Bash script. For example: $ [ 1 -eq 1 ] && echo “correct, 1 does indeed equal … Bash: Using logic expressions as a shorthand for if-then-else control
If you need to output only lines before (or after) a unique line identified by a regular expression, then here are examples of using sed. All examples in this article can be found in github. Consider the following lines of content in the variable “$lines” $ echo “$lines” ape swings frog jumps logs dog barks … Bash: output all lines before/after line identified by regex
Whether looking at differences in filenames, installed packages, etc. it can be useful to calculate the difference between two Bash arrays. SiegeX on stackoverflow.com offered the following function using awk, and I have built a full example available on github. function arraydiff() { awk ‘BEGIN{RS=ORS=” “} {NR==FNR?a[$0]++:a[$0]–} END{for(k in a)if(a[k])print k}’ <(echo -n “${!1}”) <(echo … Bash: Difference between two arrays
If your backend components or application servers use a custom CA (Certificate Authority), then you may need to add it to the system trusted root certificate store so that the standard tools and other utilities trust the TLS communication.
YAML is a popular syntax for configuration, and it is common to have certificate definitions embedded in these files. But since the cert is typically Base64 PEM encoded, it means you can’t easily view its attributes (subject, expiration date, etc) and so you are left with the manual task of copy-pasting it out, saving as … Bash: Examining each certificate in a yaml file using sed and openssl
The ability to quickly stand up a guest OS with cloud-init is most often associated with deployment of virtual machines in an IaaS like EC2 or Azure. But cloud-init is not just for remote cloud providers, and using cloud-init for local images that can be quickly deployed in KVM works great for local development and … KVM: Testing cloud-init locally using KVM for a RHEL cloud image
Within current distributions of Linux, there is a kernel component called netem that can be used to test and simulate the type of issues one would see over a Wide Area Network. This component is managed with a tool called traffic controller. This can be helpful during testing/troubleshooting to emulate the network latency or packet … Linux: Introducing latency and packet loss into network for testing
The ability to quickly stand up a guest OS with cloud-init is most often associated with deployment of virtual machines in an IaaS like EC2 or Azure. But cloud-init is not just for remote cloud providers, and using cloud-init for local images that can be quickly deployed in KVM works great for local development and … KVM: Testing cloud-init locally using KVM for a CentOS cloud image
The ability to quickly stand up a guest OS with cloud-init is most often associated with deployment of virtual machines in an IaaS like EC2 or Azure. But cloud-init is not just for remote cloud providers, and using cloud-init for local images that can be quickly deployed in KVM works great for local development and … KVM: Testing cloud-init locally using KVM for an Ubuntu cloud image