Linux

Ubuntu: Adding a root certificate authority

If your backend components or application servers use a custom CA (Certificate Authority), then you may need to add it to the system trusted root certificate store so that the standard tools and other utilities trust the TLS communication.

Bash: Examining each certificate in a yaml file using sed and openssl

YAML is a popular syntax for configuration, and it is common to have certificate definitions embedded in these files. But since the cert is typically Base64 PEM encoded, it means you can’t easily view its attributes (subject, expiration date, etc) and so you are left with the manual task of copy-pasting it out, saving as Bash: Examining each certificate in a yaml file using sed and openssl

Linux: Introducing latency and packet loss into network for testing

Within current distributions of Linux, there is a kernel component called netem that can be used to test and simulate the type of issues one would see over a Wide Area Network.  This component is managed with a tool called traffic controller. This can be helpful during testing/troubleshooting to emulate the network latency or packet Linux: Introducing latency and packet loss into network for testing

KVM: Testing cloud-init locally using KVM for a CentOS cloud image

The ability to quickly stand up a guest OS with cloud-init is most often associated with deployment of virtual machines in an IaaS like EC2 or Azure. But cloud-init is not just for remote cloud providers, and using cloud-init for local images that can be quickly deployed in KVM works great for local development and KVM: Testing cloud-init locally using KVM for a CentOS cloud image

KVM: Testing cloud-init locally using KVM for an Ubuntu cloud image

The ability to quickly stand up a guest OS with cloud-init is most often associated with deployment of virtual machines in an IaaS like EC2 or Azure. But cloud-init is not just for remote cloud providers, and using cloud-init for local images that can be quickly deployed in KVM works great for local development and KVM: Testing cloud-init locally using KVM for an Ubuntu cloud image

KVM: Terraform and cloud-init to create local KVM resources

Terraform is a popular tool for provisioning infrastructure on cloud provider such as EC2 and Azure, but there is also a provider written for local KVM libvirt resources. Using the libvirt provider, we can use standard Terraform constructs to create local VMs, networks, and disks.

Bash: Using shell or environment variables in awk output

If you are in the middle of a text processing pipeline, and need to insert a shell or environment variable into the output of awk, you can use the “-v” flag. Here are two files containing animal classifications: $ echo -e “shark=fish\ndolphin=mammal” > ocean.txt $ echo -e “dog=mammal\neagle=bird” > land.txt By passing the loop variable Bash: Using shell or environment variables in awk output

Docker: Placing limits on cpu usage in containers

Containers themselves are light, but by default a container has access to all the CPU resources the Docker host kernel scheduler will allow. Internally Docker uses cgroups to limit CPU resources, and this is exposed as the flag “–cpus” when bringing up a docker container: sudo docker run -it –cpus=1.0 alpine:latest /bin/sh This will limit Docker: Placing limits on cpu usage in containers

Docker: Placing limits on container memory using cgroups

Containers themselves are light, but by default a container has access to all the memory resources of the Docker host. Internally Docker uses cgroups to limit memory resources, and in its simplest form is exposed as the  flags “-m” and “–memory-swap” when bringing up a docker container. sudo docker run -it -m 8m –memory-swap 8m Docker: Placing limits on container memory using cgroups

Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size

If you are using the overlay2 storage driver, you can place limits on the rootfs within a container but only if using an xfs backing filesystem (not ext4). As a quick test of your Docker install, check your Docker storage driver and backing filesystem, then attempt to spin up a small alpine image with a Docker: Use overlay2 with an xfs backing filesystem to limit rootfs size

Linux: Mounting a loopback ext4/xfs filesystem to isolate or enforce storage limits

The physical partitions and filesystem formats on your host are configured for your main workload, but if you want an application to use a specific filesystem (xfs, ext4, zfs) and size capacity without reconfiguration at the physical level then you can consider a loopback image. For example, if we create a 100Mb disk file named Linux: Mounting a loopback ext4/xfs filesystem to isolate or enforce storage limits

Linux: Using xfs project quotas to limit capacity within a subdirectory

XFS is a journaled filesystem that has excellent parallel performance, and is licensed under the GPL which means it has been included in many Linux distributions. One of the features of XFS is the ability to enforce quotas based on user, group, and project.  In this article, I will show how to assign filesize quotas Linux: Using xfs project quotas to limit capacity within a subdirectory

Docker: Installing Docker CE on Ubuntu bionic 18.04

Update Dec 2021: I have written an updated article for installing Docker on focal 20.04 Docker is a container platform that streamlines software delivery and provides isolation, scalability, and efficiency with less overhead than OS level virtualization. These instructions are taken directly from the official Docker for Ubuntu page, but I wanted to reiterate those Docker: Installing Docker CE on Ubuntu bionic 18.04

KVM: Creating a bridged network with NetPlan on Ubuntu 18.04 bionic

UPDATE September 2022: New article for bridged networks written for Ubuntu 22.04 In order to expose KVM virtual machines on the same network as your Host, you need to enable bridged networking. In this article, I’ll show how to implement KVM bridged networking on Ubuntu 18.04 bionic using Netplan.  This bridged network will expose the KVM: Creating a bridged network with NetPlan on Ubuntu 18.04 bionic

VMware: Using the govc CLI to automate vCenter commands

The vSphere web GUI is a nice visual tool, but if you need to retrieve vCenter information in bulk or perform mass operations across VMs, then a command line tool such as govc in invaluable. govc is written in Go, which means it has support on Linux as well as most other platforms.

Ubuntu: X2Go on Ubuntu bionic for remote desktop access

Updated April 2023: Tested on Ubuntu 22.04 LTS with X2GO sever 4.1.0 X2Go provides remote desktop access for Linux, similar to VNC or xRDP.  It tunnels over ssh which can provide SSH public key authentication for security and is easily understood when opening firewall rules. Additionally, it is optimized for narrow bandwidth requirements, making it Ubuntu: X2Go on Ubuntu bionic for remote desktop access

Git: client error, server certificate verification failed

Especially with private git repositories that may be self-signed or have private CA, you may get the following error from the git client after a certificate has been updated: fatal: unable to access ‘https://git.mycompany.com/myuser/myrepo.git/’: server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none This means that the git client cannot verify the integrity of the certificate Git: client error, server certificate verification failed

Ubuntu: Auditing sudo commands and forwarding audit logs using syslog

sudo provides users with temporary elevated privileges to perform operations.  No matter what your security philosophy, sudo is more than likely enabled on your system if even for a limited number of users. And if it is enabled, creating an audit log of exactly what was run (and who ran it) is essential to reporting.  Ubuntu: Auditing sudo commands and forwarding audit logs using syslog

KVM: Deploy the VMware vCenter 6.7 appliance using the CLI installer

Update Nov 2021: I have written a newer article that deploys vCenter 7.0. If you have just virtualized the VMware ESXi 6.7 server on top of KVM, the next step will be to install vCenter 6.7 for its centralized control and additional feature set and management capabilities. In my last article we took KVM running KVM: Deploy the VMware vCenter 6.7 appliance using the CLI installer

Linux: Using GPG encrypted credentials for enhanced security

If you currently store sensitive credentials in plaintext to automate scripting or integration to other systems, you should consider an extra layer of security by storing them encrypted using GPG. There is no fullproof way to hide sensitive information for a service that also needs to decrypt them as part of normal operations (think DVD Linux: Using GPG encrypted credentials for enhanced security