Github Actions provide the ability to define a build workflow, including the packaging and publishing of a Helm chart. This allows tools like Helm to refer to the URL of the public source project, add it as a remote Helm repository, and then use the packaged chart to deploy a workload to a Kubernetes cluster. … GitHub: automated publish of Helm chart using GitHub Actions
As much as Terraform pushes to be the absolute system of record for resources it creates, often valid external processes are assisting in managing those same resources. Here are some examples of legitimate external changes: Other company-approved Terraform scripts applying labeling to resources in order to track ownership and costs Security teams modifying IAM roles … GKE: terraform lifecycle ‘ignore_changes’ to manage external changes to GKE cluster
The centralized system keyring for apt was deprecated starting in Ubuntu 21, and is being replaced with an explicit path to the local gpg key in the ‘signed-by’ attribute. I have written more extensive articles on this subject [here,here], but from an Ansible perspective, this means ensuring the gpg key is downloaded to ‘/usr/share/keyrings’ with … Ansible: adding custom apt repository with ‘signed-by’ gpg key
If you have a simple directory containing multiple template files that should be generated on a target host, the ‘with_fileglob‘ lookup plugin provides an easy way to render them. Below is an example rendering all the files from the ‘templates’ directory of a role. – name: create file out of every file in template directory … Ansible: generating templates with deep directory structure using with_filetree
When orchestrating a playbook against multiple hosts, you may need to pull facts from one host in order to feed those values to another host. For example, a database master may need to provide its IP address to its many replicas, or a CA certificate from a controller may need to be distributed among its … Ansible: accessing a fact from a different host using cached facts
If you are creating a log file or perhaps an archive directory, it can be a convenience to end users to use a sortable timestamp in the file name itself, similar to “YYYYMMDD-HHMMSS”. This allows them to easily find the file relevant to their needs. Here are a couple of ways to implement this logic … Ansible: embedding a timestamp in a file name
If you have a complex JSON data structure or perhaps an array of rich data structures as the result of module output, you may want to extract this into a workable list or dictionary you can take action on. For this article, I will emulate the resulting output from the ‘stat‘ module when it checks … Ansible: extracting a list or dictionary from a complex data structure
Managing certificates is one of the most mundane, yet critical chores in the maintenance of environments. However, this manual maintenance can be off-loaded to cert-manager on Kubernetes. In this article, we will use cert-manager to generate TLS certs for a public NGINX ingress using Let’s Encrypt. The primary ingress will have two different hosts using … Kubernetes: LetsEncrypt certificates using HTTP and DNS solvers on DigitalOcean
Updated Aug 2023: tested with Kubernetes 1.25 and ingress-nginx 1.8.1 Creating a Kubernetes cluster on DigitalOcean can be done manually using its web Control Panel, but for automation purposes it is better to use Terraform. In this article, we will use Terraform to create a Kubernetes cluster on DigitalOcean infrastructure. We will then use helm … Terraform: creating a Kubernetes cluster on DigitalOcean with public NGINX ingress
If you are creating a VM resource and must run a Bash script as part of the initialization, that can be done within Terraform using the remote-exec provisioner and its ability to execute scripts via ssh. If you need to send arguments to this script, there is a standard pattern described in the official documentation … Terraform: post-configuration by calling remote-exec script with parameters
If the Terraform resource you are creating supports multiple dependent entities (e.g. a single VM with multiple disks or networks), but only by adding hardcoded duplicate text blocks, then you should consider Terraform dynamic blocks. For example, if you are creating a vsphere_virtual_machine with two additional data disks, then here is a snippet showing how … Terraform: using dynamic blocks to add multiple disks on a vsphere_virtual_machine
Specifying input variables in the “terraform.tfvars” file in HCL syntax is commonly understood. But if the values you need are already coming from a json source, it might make more sense to feed those directly to Terraform. Here is an example where the simple variable “a” is provided via an external json file. # … Terraform: using json files as input variables and local variables
For security reasons, you should be very aware that accepting a remote host fingerprint automatically is a procedure that should be considered high-risk. But if you are working with automated infrastructure or pipelines where human intervention is not possible and the constructed entities are being built in a secure fashion with guaranteed provenance, then ssh-keyscan … Bash: accepting a remote host fingerprint with ssh-keyscan
If you are using “–extra-vars” at runtime to override a boolean variable, then you should use the JSON syle (instead of key-value pairs) to avoid type conversion issues. For example, if there is a boolean variable named “myflag” that is default to true in your playbook and you attempt to override it like below, it … Ansible: overriding boolean values using extra-vars at runtime
Unfortunately, with_fileglob only works on files local to the Ansible orchestrator host. It will not build a list of files from the remote directory, even if you use something like the copy module that has ‘remote_source: true’. In these situation, you can use the find module to generate a list of files on the remote … Ansible: find module to create glob of remote files
In a previous article, I showed how to manually setup Alternatives so that different versions of a binary could co-exist on a target machine. In that step-by-step example, we used the Terraform binary as an example, and placed two independent versions in /usr/local/bin, and then set the priority so that terraform14 was preferred. To do … Ansible: Ubuntu alternatives using the community.general collection
Ansible blocks provide a convenient way to logically group tasks. So it is unfortunate that native Ansible syntax does not allow looping to be combined with a block. Consider the simple conditional block below controlled by a variable ‘do_block_logic’: – name: simple block with conditional block: – name: simple block task1 debug: msg=”hello” – name: … Ansible: implementing a looping block using include_tasks
The legacy method of running crontab jobs from files located at /var/spool/cron (edited using ‘crontab -e’) is slowly giving way to files located in /etc/cron.d. This way of creating a file entry for each job is easily handled by Ansible using the cron module as shown below. – name: Creates a cron file under /etc/cron.d … Ansible: creating a cron.d file for periodic system jobs
If you need to expand an encrypted zip file using the Ansible unarchive module, then you will need to provide the password using the ‘extra_opts’ parameter. Per below, make sure you place the “-P” flag as an independent argument to the password. – name: unzip encrypted zip unarchive: src: mysource.zip dest: /remote/path extra_opts: – “-P” … Ansible: unzipping an encrypted file using the unarchive module
Update Sep 2023: Installing ansible-core at user level (not system) with pip Ansible is an agentless configuration management tool that helps operations teams manage installation, patching, and command execution across a set of servers. In this article I’ll describe how to install the latest release of Ansible.
Ansible uses ssh to configure its target host inventory, but for on-premise datacenters as well as hyperscalers like EC2/GCP/Azure, the target hosts are often purposely located in deeper private subnets that cannot be reached from the Ansible orchestrator host. One solution is to enable a bastion/jumpbox host that serves as the forwarding host. It sits … Ansible: orchestrating ssh access through a bastion host
With Ansible, it is possible to have a variable populated with either the content of a remote or local file. For a file on a remote host, you can use the ‘slurp‘ module to put the content into a register, but then you must base64 decode the content. – name: get content of remote file … Ansible: creating a variable from a remote or local file content
In a single Ansible playbook, you may wish to apply some roles to all hosts, while limiting other roles to only certain groups. While it is certainly possible to apply a role to all hosts and then use a ‘when’ to filter down to the desired group like below: – hosts: all gather_facts: yes roles: … Ansible: applying roles to certain groups in a single playbook
For Ubuntu, there are a couple of ways you can install the linux-headers package matching the kernel version. You can either explicitly specify the version, or use the meta package as shown below. # specify kernel version using subshell sudo apt-get install -y linux-headers-$(uname -r) # OR meta package that auto-matches kernel sudo apt-get install … Ansible: installing linux-headers matching kernel for Ubuntu
Pulling the most recent script/file/asset from a remote host may be preferable, but you would like to provide a fallback to a local file just in case the remote pull fails. This is relatively easy to do with Ansible by allowing the remote pull to ‘ignore_errors’ and then doing a local file copy if an … Ansible: preferring a pull from a URL with fallback to a local file
It is typically straightforward to resolve Ansible errors where a simple variable used within an action is not defined. However, when dealing with nested dictionary variables in your Ansible tasks, it can become more complex because not only can the leaf be missing, but also any of the parent container paths. For example, the nested … Ansible: pulling values from nested dictionaries when path might not exist
Even if the actions in your playbook/role are tagged to separate their logic, this ability to selectively execute will not operate properly when called without any tags because then you will fallback to the special ‘all‘ behavior. Consider a playbook with the following actions. tasks: – debug: msg=”when tag ‘run'” tags: run – debug: msg=”when … Ansible: action only executed if tag set, avoiding ‘all’ behavior
Ansible has support for generating self-signed certificates as well as certificates using a custom root CA (certificate authority). This is possible using the community.crypto collection. I’ve put this into a role named ansible-role-cert-with-ca available on github, and it can be used from a playbook like below: vars: # custom CA, leaving undefined will create self-signed … Ansible: creating SAN certificates with a custom root CA
There are scenarios where instead of being explicit about each file name when generating templates from a role, you may want to take all the suffixed files in the ‘templates’ directory and dump their generated content into a destination directory. As a quick example, if you are in a role and want every script in … Ansible: generating content for all template files using with_fileglob
The Ansible file module will allow you to use ‘state: absent’ to remove a file path, but if you need to perform this action if and only if the path is a symbolic link then you will need to register a test for this condition. By first using the stat module to test if the … Ansible: deleting a file path, but only if a symbolic link
If a git repository requires credentials to clone, and you are still using a username/password (instead of ssh key), it is still possible to have the repository cloned in your automation scripts without be prompted. You just have to ensure that the username and password are properly URL encoded. From the command line, the syntax … Ansible: cloning a git repository that requires credentials
A centralized authentication/authorization mechanism is a key part of a security stance as well as a convenience to end users. There are multiple packages and systems to achieve this, and in this article I will focus on integrating back into Windows Active Directory using SSSD for login and group membership. When complete, you will have … Ansible: Login to Ubuntu with Windows Active Directory using SSSD
Amazon EC2 provides a web interface for managing IaaS, but for repeatable infrastructure deployment what you really want is the ability to deploy and manage this infrastructure using an API or command line tool. The AWS SDK for Python (Boto3) provides a lower level as well as resource level API for managing and creating infrastructure. This … AWS: Installing the AWS SDK for Python on Ubuntu
Amazon EC2 provides a web interface for managing IaaS, but for repeatable infrastructure deployment what you really want is the ability to deploy and manage this infrastructure using an API or command line tool. In this article we want to focus on the CLI (command line interface), which shields us from the API innards and … AWS: Installing the AWS CLI on Ubuntu
There are many Linux based ftp clients, but if you run into connectivity issues when using the HTTP CONNECT method through a proxy such as Squid, you may need to leverage more control by writing your interactions using a library such as ftp4j. In this article, I’ll provide a Java project with sample code that … Java: FTP with an HTTP proxy using the CONNECT method
The cmd.run state module is a handy way to run commands on a remote host. However, if your parameter values contain dollar signs ($) they will be interpolated in the target shell, which will lead to unexpected results. The trick is to escape properly by adding slashes before the dollar sign.
Configuration Management tools like SaltStack are invaluable for managing infrastructure at scale. Even in the growing world of containerization, there is the need for bulk automation. This article will detail installation of Salt SSH which leverages the power of SaltStack without the requirements for an agent install.
Configuration Management tools like SaltStack are invaluable for managing infrastructure at scale. Even in the growing world of containerization where immutable image deployment is the norm, those images need to be built in a repeatable and auditable fashion. This article will detail installation of the SaltStack master on Ubuntu Xenial 16.04, with validation using a single Minion. Note that … SaltStack: Installing a Salt Master on Ubuntu Xenial
WinSCP is a Windows application for transferring files via fto or scp to remote host. A feature not many take advantage of is the ability to create automation scripts that can execute these transfers as silent batch jobs. In this article I will show how to use scripted FTP commands and SCP commands to automate … WinSCP: Automated file transfer from a Windows host
If your Salt Minion version is too far removed from the Salt Master version, you may find yourself with unexplained errors. This problem can be faced when the OS template you are deploying was packaged years earlier with an older Salt minion while the Salt Master has been kept up to date. But it can … SaltStack: Installing an older Salt Master or Minion for compatibility