Automation

GKE: terraform lifecycle ‘ignore_changes’ to manage external changes to GKE cluster


Categories: Automation, DevOps, Hyperscaler

As much as Terraform pushes to be the absolute system of record for resources it creates, often valid external processes are assisting in managing those same resources. Here are some examples of legitimate external changes: Other company-approved Terraform scripts applying labeling to resources in order to track ownership and costs Security teams modifying IAM roles … GKE: terraform lifecycle ‘ignore_changes’ to manage external changes to GKE cluster

Ansible: adding custom apt repository with ‘signed-by’ gpg key


Categories: Automation

The centralized system keyring for apt was deprecated starting in Ubuntu 21, and is being replaced with an explicit path to the local gpg key in the ‘signed-by’ attribute. I have written more extensive articles on this subject [here,here], but from an Ansible perspective, this means ensuring the gpg key is downloaded to ‘/usr/share/keyrings’ with … Ansible: adding custom apt repository with ‘signed-by’ gpg key

Ansible: generating templates with deep directory structure using with_filetree


Categories: Automation

If you have a simple directory containing multiple template files that should be generated on a target host, the ‘with_fileglob‘ lookup plugin provides an easy way to render them.  Below is an example rendering all the files from the ‘templates’ directory of a role. – name: create file out of every file in template directory … Ansible: generating templates with deep directory structure using with_filetree

Kubernetes: LetsEncrypt certificates using HTTP and DNS solvers on DigitalOcean


Categories: Automation, Containers

Managing certificates is one of the most mundane, yet critical chores in the maintenance of environments.   However, this manual maintenance can be off-loaded to cert-manager on Kubernetes. In this article, we will use cert-manager to generate TLS certs for a public NGINX ingress using Let’s Encrypt.   The primary ingress will have two different hosts using … Kubernetes: LetsEncrypt certificates using HTTP and DNS solvers on DigitalOcean

Terraform: creating a Kubernetes cluster on DigitalOcean with public NGINX ingress


Categories: Automation, Containers

Updated Aug 2023: tested with Kubernetes 1.25 and ingress-nginx 1.8.1 Creating a Kubernetes cluster on DigitalOcean can be done manually using its web Control Panel, but for automation purposes it is better to use Terraform. In this article, we will use Terraform to create a Kubernetes cluster on DigitalOcean infrastructure. We will then use helm … Terraform: creating a Kubernetes cluster on DigitalOcean with public NGINX ingress

Terraform: post-configuration by calling remote-exec script with parameters


Categories: Automation

If you are creating a VM resource and must run a Bash script as part of the initialization, that can be done within Terraform using the remote-exec provisioner and its ability to execute scripts via ssh. If you need to send arguments to this script, there is a standard pattern described in the official documentation … Terraform: post-configuration by calling remote-exec script with parameters

Terraform: using dynamic blocks to add multiple disks on a vsphere_virtual_machine


Categories: Automation

If the Terraform resource you are creating supports multiple dependent entities (e.g. a single VM with multiple disks or networks), but only by adding hardcoded duplicate text blocks, then you should consider Terraform dynamic blocks. For example, if you are creating a vsphere_virtual_machine with two additional data disks, then here is a snippet showing how … Terraform: using dynamic blocks to add multiple disks on a vsphere_virtual_machine

Terraform: using json files as input variables and local variables


Categories: Automation

Specifying input variables in the “terraform.tfvars” file in HCL syntax is commonly understood.   But if the values you need are already coming from a json source, it might make more sense to feed those directly to Terraform. Here is an example where the simple variable “a” is provided via an external json file. # … Terraform: using json files as input variables and local variables

Bash: accepting a remote host fingerprint with ssh-keyscan


Categories: Automation, Linux

For security reasons, you should be very aware that accepting a remote host fingerprint automatically is a procedure that should be considered high-risk. But if you are working with automated infrastructure or pipelines where human intervention is not possible and the constructed entities are being built in a secure fashion with guaranteed provenance, then ssh-keyscan … Bash: accepting a remote host fingerprint with ssh-keyscan

Ansible: Ubuntu alternatives using the community.general collection


Categories: Automation

In a previous article, I showed how to manually setup Alternatives so that different versions of a binary could co-exist on a target machine. In that step-by-step example, we used the Terraform binary as an example, and placed two independent versions in /usr/local/bin, and then set the priority so that terraform14 was preferred. To do … Ansible: Ubuntu alternatives using the community.general collection

Ansible: implementing a looping block using include_tasks


Categories: Automation

Ansible blocks provide a convenient way to logically group tasks.  So it is unfortunate that native Ansible syntax does not allow looping to be combined with a block.  Consider the simple conditional block below controlled by a variable ‘do_block_logic’: – name: simple block with conditional block: – name: simple block task1 debug: msg=”hello” – name: … Ansible: implementing a looping block using include_tasks

Ansible: unzipping an encrypted file using the unarchive module


Categories: Automation

If you need to expand an encrypted zip file using the Ansible unarchive module, then you will need to provide the password using the ‘extra_opts’ parameter. Per below, make sure you place the “-P” flag as an independent argument to the password. – name: unzip encrypted zip unarchive: src: mysource.zip dest: /remote/path extra_opts: – “-P” … Ansible: unzipping an encrypted file using the unarchive module

Ansible: installing the latest Ansible on Ubuntu


Categories: Automation

Update Sep 2023: Installing ansible-core at user level (not system) with pip Ansible is an agentless configuration management tool that helps operations teams manage installation, patching, and command execution across a set of servers. In this article I’ll describe how to install the latest release of Ansible.

Ansible: orchestrating ssh access through a bastion host


Categories: Automation

Ansible uses ssh to configure its target host inventory, but for on-premise datacenters as well as hyperscalers like EC2/GCP/Azure, the target hosts are often purposely located in deeper private subnets that cannot be reached from the Ansible orchestrator host. One solution is to enable a bastion/jumpbox host that serves as the forwarding host.  It sits … Ansible: orchestrating ssh access through a bastion host

Ansible: installing linux-headers matching kernel for Ubuntu


Categories: Automation

For Ubuntu, there are a couple of ways you can install the linux-headers package matching the kernel version.  You can either explicitly specify the version, or use the meta package as shown below. # specify kernel version using subshell sudo apt-get install -y linux-headers-$(uname -r) # OR meta package that auto-matches kernel sudo apt-get install … Ansible: installing linux-headers matching kernel for Ubuntu

Ansible: pulling values from nested dictionaries when path might not exist


Categories: Automation

It is typically straightforward to resolve Ansible errors where a simple variable used within an action is not defined.  However, when dealing with nested dictionary variables in your Ansible tasks, it can become more complex because not only can the leaf be missing, but also any of the parent container paths. For example, the nested … Ansible: pulling values from nested dictionaries when path might not exist

Ansible: action only executed if tag set, avoiding ‘all’ behavior


Categories: Automation

Even if the actions in your playbook/role are tagged to separate their logic, this ability to selectively execute will not operate properly when called without any tags because then you will fallback to the special ‘all‘ behavior. Consider a playbook with the following actions. tasks: – debug: msg=”when tag ‘run'” tags: run – debug: msg=”when … Ansible: action only executed if tag set, avoiding ‘all’ behavior

Ansible: creating SAN certificates with a custom root CA


Categories: Automation

Ansible has support for generating self-signed certificates as well as certificates using a custom root CA (certificate authority).  This is possible using the community.crypto collection. I’ve put this into a role named ansible-role-cert-with-ca available on github, and it can be used from a playbook like below: vars: # custom CA, leaving undefined will create self-signed … Ansible: creating SAN certificates with a custom root CA

Ansible: generating content for all template files using with_fileglob


Categories: Automation

There are scenarios where instead of being explicit about each file name when generating templates from a role, you may want to take all the suffixed files in the ‘templates’ directory and dump their generated content into a destination directory. As a quick example, if you are in a role and want every script in … Ansible: generating content for all template files using with_fileglob

AWS: Installing the AWS CLI on Ubuntu


Categories: Automation

Amazon EC2 provides a web interface for managing IaaS, but for repeatable infrastructure deployment what you really want is the ability to deploy and manage this infrastructure using an API or command line tool. In this article we want to focus on the CLI (command line interface), which shields us from the API innards and … AWS: Installing the AWS CLI on Ubuntu

SaltStack: salt-ssh for agentless automation on Ubuntu


Categories: Automation, DevOps

Configuration Management tools like SaltStack are invaluable for managing infrastructure at scale.  Even in the growing world of containerization, there is the need for bulk automation. This article will detail installation of  Salt SSH which leverages the power of SaltStack without the requirements for an agent install.

SaltStack: Installing a Salt Master on Ubuntu Xenial


Categories: Automation, DevOps

Configuration Management tools like SaltStack are invaluable for managing infrastructure at scale.  Even in the growing world of containerization where immutable image deployment is the norm, those images need to be built in a repeatable and auditable fashion. This article will detail installation of the SaltStack master on Ubuntu Xenial 16.04, with validation using a single Minion. Note that … SaltStack: Installing a Salt Master on Ubuntu Xenial

Ubuntu: Standing up a Windows 2012 instance on Ubuntu using Sysprep


Categories: Automation, Linux

In the world of Linux containers where deployment takes on the order of seconds, even the best-case scenario for spinning up a new Windows host can seem like an eternity. Clearly, you don’t want to wait for the entire Windows install process each time you bring up a Windows guest OS.  Even automated, this would … Ubuntu: Standing up a Windows 2012 instance on Ubuntu using Sysprep