Linux

Ubuntu: Running a bash script periodically with a user-level Systemd timer

If you have a Bash script that needs to run periodically, you can run it using a crontab entry.  But you can also have it invoked by Systemd using systemd.timer. Furthermore, you can run Systemd services as  user-level services instead of the typical system-level service for even further isolation. Running via Systemd provides more powerful Ubuntu: Running a bash script periodically with a user-level Systemd timer

Ubuntu: Running a bash script periodically with a system-level Systemd timer

If you have a Bash script that needs to run periodically, you can run it using a crontab entry or file.  But you can also have it invoked from Systemd using systemd.timer. Running via Systemd provides more powerful constructs for invocation, configuration, monitoring, and logging.  In this article, I will show how to periodically run Ubuntu: Running a bash script periodically with a system-level Systemd timer

Docker: Installing Docker CE on Ubuntu focal 20.04

Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those tasks essential for installing the Docker Community Edition on Ubuntu focal 20.04. If you want Docker: Installing Docker CE on Ubuntu focal 20.04

Bash: deleting a file with special characters using its inode value

If you have a file with special characters (single quotes, wildcard, etc) in the name, it can be difficult to discover the exact escape sequence to correctly delete.  To avoid playing with escape characters, you can simply use the inode number of the file instead. For example, let’s say you accidentally specify tar options incorrectly Bash: deleting a file with special characters using its inode value

Bash: accepting a remote host fingerprint with ssh-keyscan

For security reasons, you should be very aware that accepting a remote host fingerprint automatically is a procedure that should be considered high-risk. But if you are working with automated infrastructure or pipelines where human intervention is not possible and the constructed entities are being built in a secure fashion with guaranteed provenance, then ssh-keyscan Bash: accepting a remote host fingerprint with ssh-keyscan

Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter

In this article I will demonstrate how to create an Ubuntu 20 Focal template in vCenter.  Then use Terraform to create a vSphere VM based on this template. The VM template creation is done by manually stepping through an installation using the minimal Ubuntu server ISO followed by a set of preparation steps. Then Terraform Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter

Bash: find most recently modified files

Needing to find the most recently modified files in a directory is a pretty common need.  Luckily the find utility has flags to easily explore a directory recursively and list recently modified files. If you want to find modified files within ‘N’ days ago from the current directory. # files within the last 24 hours Bash: find most recently modified files

Bash: cloning the ownership and permissions of another file using reference

If you need to create a file that has the exact same ownership and permission bits as an existing file, the ‘reference’ flag provides a convenient shortcut. For example, if you had a file named ‘myoriginal’ that had the exact ownership and permissions required for a new file ‘mynewfile’, you could use the commands below Bash: cloning the ownership and permissions of another file using reference

KVM: installing Terraform and the libvirt provider for local KVM resources

Terraform is a popular tool for provisioning infrastructure on cloud providers such as EC2 and Azure, but there is also a provider written for local KVM libvirt resources. Using the libvirt provider, we can use standard Terraform constructs to create local VMs, networks, and disks.  And unlike older versions of this provider, the plugin binary KVM: installing Terraform and the libvirt provider for local KVM resources

Bash: using printf to display fixed-width padded string

One way to implement character padding in Bash is to use printf and substring extraction.  This can be especially useful in reports or menu display. Given a $padding variable that contains the maximum length of characters, you can subtract out the length of a display string like below. # length of maximum padding padding=”………………………………..” printf Bash: using printf to display fixed-width padded string

Terraform: provisioning GCP servers in both public and private subnets

It is relatively straightforward to create a GCP public subnet where the compute instances have access to the public internet via the default internet gateway. But once you start building private subnets behind it, you must start considering firewall, routing, and the NAT gateways required to reach public services. In this article, I will use Terraform: provisioning GCP servers in both public and private subnets

Ubuntu: loading a key into ssh-agent at login with a user-level systemd service

By default, if an SSH key file is dropped into your personal ‘~/.ssh’ directory that matches a set of standard names, then it will automatically be used as an identity when logging into a remote site (id_rsa, id_dsa, id_ecsda, id_ed25519, or identity). For example, this makes it simple to comply with Github’s requirement to use Ubuntu: loading a key into ssh-agent at login with a user-level systemd service

Ubuntu: using ldapsearch to query against a secure Windows Domain Controller

Using ldapsearch to query against the insecure port of a Windows Domain Controller is straightforward.  However, it can be challenging to get all the pieces in place for a production environment where the secure port must be used and the root CA certificate is typically not from a public CA. Assuming the standard insecure port Ubuntu: using ldapsearch to query against a secure Windows Domain Controller

Ubuntu: Extending capacity of an LVM volume group using an existing or new disk

It is common for a virtualized Guest OS base image to have a generic storage capacity.  This capacity can easily be exceeded by production scenarios, performance testing, logging, or even the general cruft of running a machine 24×7. If your VM is using Logical Volume Management (LVM), adding space can be done by following a Ubuntu: Extending capacity of an LVM volume group using an existing or new disk

KVM: creating and reverting libvirt external snapshots

Update July 2021: I have seen errors with external snapshots of volumes on versions of QEMU/KVM/libvirt from Ubuntu 20 Focal.  Adding note on using internal snapshot on volume backed by qcow2.   Internal snapshots created on QEMU copy-on-write (qcow2) disks are the most commonly used snapshot when using libvirt.  It is easy to see why; KVM: creating and reverting libvirt external snapshots

Bash: Fixing an ASCII text file changed with Unicode character sequences

File encoding issues are difficult to diagnose and troubleshoot.  Most files in the operations world are expected to be text-only ASCII 7 bit, so if a file goes into UTF-8 encoding and has embedded Unicode character inserted, it can often throw off the tool chain or systems using the file. Here are two example files, Bash: Fixing an ASCII text file changed with Unicode character sequences

Bash: Difference between two arrays

Whether looking at differences in filenames, installed packages, etc. it can be useful to calculate the difference between two Bash arrays. SiegeX on stackoverflow.com offered the following function using awk, and I have built a full example available on github. function arraydiff() { awk ‘BEGIN{RS=ORS=” “} {NR==FNR?a[$0]++:a[$0]–} END{for(k in a)if(a[k])print k}’ <(echo -n “${!1}”) <(echo Bash: Difference between two arrays