AWS: using Certificate Manager to generate a TLS certificate for a delegated subdomain


Categories: aws

AWS Certificate Manager can generate custom TLS certificates that can be used by AWS resources to enable secure TLS communication. In this article, I will delegate authority of a custom subdomain to AWS Route 53, then allow AWS ACM to generate certificates for this subdomain.  This subdomain and certificate can then be used within the … AWS: using Certificate Manager to generate a TLS certificate for a delegated subdomain

Porkbun DNS: create DNS records via API using curl


Categories: Scripting

Porkbun is an ICANN accredited domain name registrar with value pricing and an API that allows domain record creation and updates, which makes it especially useful for scripting. This article assumes you already have an account, and have Porkbun managing at least one domain name.

GitLab: query the project and group level GitLab CI/CD variables using curl


Categories: Automation, CI/CD

GitLab Pipelines provide the ability to define a workflow that has externalized variables that can be set at the direct project level, or inherited from the group or instance level. These can be viewed in the GitLab web UI, but for scripting and automation can also be queried from the GitLab REST API or GraphQL … GitLab: query the project and group level GitLab CI/CD variables using curl

AWS: retrieving standalone database instances by filtering out those with cluster membership


Categories: aws, Scripting

The AWS CLI provides a way to query for DB instances using “aws rds describe-db-instances“, but this will return all DB instances including those that are members of a cluster. If we want to query for just the standalone DB instances (not part of a cluster), here are a couple of options. You can use … AWS: retrieving standalone database instances by filtering out those with cluster membership

jq: Resolving “Cannot index array with string” errors from jq


Categories: Scripting

jq has a very powerful expression language for querying and transforming json.  But when dealing with complex data structures and responses in the real-world, well-tested happy path queries can fail. One common error is: jq: error (at <stdin>:1): Cannot index array with string “…” The root cause is that your query is attempting to pull … jq: Resolving “Cannot index array with string” errors from jq

Flux: installing Flux on a Kubernetes cluster with bootstrap command


Categories: DevOps, Kubernetes

Flux is a GitOps continuous delivery tool that provides a framework for keeping a Kubernetes cluster in-sync with source git repositories, OCI registries, and published Helm charts [1]. The recommended way to install Flux on a Kubernetes cluster is to bootstrap using the Flux CLI, so I will go through those details in this article. … Flux: installing Flux on a Kubernetes cluster with bootstrap command

GitLab: pipeline to publish Helm chart to GitLab Package Registry


Categories: DevOps, Kubernetes

GitLab Pipelines provide the ability to define a build workflow, including the packaging and publishing of a Helm chart to the GitLab Package Registry. This allows tools like Helm to refer to the public URL of the Gitlab Package Registry, add it as a remote Helm repository, and then use the packaged chart. Pipeline job … GitLab: pipeline to publish Helm chart to GitLab Package Registry

Mac: tensorflow-metal pip module on M1 chip for GPU support


Categories: Development, Python

Enabling the use of the GPU on your Mac M1 with the tensorflow-metal plugin can be challenging because there is a lot of conflicting documentation and older forum questions and replies. I’ve written this article for a Mac M1 running on macOS Sequoia 15.1.1.  As of December 2024, you should pair Python 3.11 with TensorFlow … Mac: tensorflow-metal pip module on M1 chip for GPU support

Terraform: converting hex and decimal representation of random_id back to id


Categories: DevOps

The random_id Terraform resource generates a value that can be used to create remote infrastructure that requires a unique identifier. The primary attribute it exposes is ‘.id’ which contains upper+lower+number characters, but it also has ‘.dec’ and ‘.hex’ equivalent representations that can be used to support infrastructure requiring a limited character set. As an example, … Terraform: converting hex and decimal representation of random_id back to id

Kubernetes: deploying Kyverno for cluster policy control


Categories: Kubernetes

Kyverno is an open-source project that manages and enforces policies within a Kubernetes cluster.  The policy definitions are defined as yaml and deployed as Kubernetes objects. Kyverno has become popular for its Kubernetes-specific policy engine and declarative rule definitions (as opposed to a general policy engine like OPA/Gatekeeper that use a domain specific language).  It … Kubernetes: deploying Kyverno for cluster policy control

Kubernetes: targeting workloads to a node pool/group using taints and tolerations


Categories: Hyperscaler, Kubernetes

If you have specific intentions for a Kubernetes node pool/group (workload isolation, cpu type, etc.), then you can assign labels to attract workloads in conjunction with taints to repel workloads that do not have explicit tolerations applied. And although the generalized kubectl utility can assign labels and taints to specific nodes, the assignment of labels … Kubernetes: targeting workloads to a node pool/group using taints and tolerations

Terraform: module for conditional include of related resources


Categories: DevOps

If you have a set of resources in Terraform that are conditionally included based on the same criteria, instead of appending a “count/for_each” on every resource definition, consider refactoring them into a module. The conditional can then be placed on the module definition instead of polluting each resource definition. For example, if you had several … Terraform: module for conditional include of related resources

Bash: falling back to file autocompletion if errors introduced by program autocompletion


Categories: Linux

At the Bash command line interface, there is the concept of programmable completion and regular file/directory completion. This means that when you press the <TAB>, the alternatives can be provided by a custom program or the filesystem hierarchy. There is always the chance that a program may introduce undesirable behavior to your auto-completion, and if … Bash: falling back to file autocompletion if errors introduced by program autocompletion

Github: security scanning built into GitHub Actions image build


Categories: Containers, Development, DevOps

Github Actions provide the ability to define a build workflow, and for projects that are building an OCI (Docker) image, there are custom actions available for running the Trivy container security scanner. In this article, I will show you how to modify your GitHub Action to run the Trivy security scanner against your image, and … Github: security scanning built into GitHub Actions image build

GitLab: security scanning built into GitLab Pipelines image build


Categories: Containers, Development, DevOps

GitLab Pipelines provide the ability to define a build workflow, and for projects that are building an OCI (Docker) image, there is a convenient method for doing container security scanning as part of the build process. Include Container Scanning As described in the official documentation, add the following include to your .gitlab-ci.yml pipeline definition. include: … GitLab: security scanning built into GitLab Pipelines image build

GCP: publishing and reading from Google PubSub Topic using Python client libraries


Categories: Hyperscaler, Python

Google Pub/Sub is a managed messaging platform providing a scalable, asynchronous, loosely-coupled solution for communication between application entities. It centers around the concept of a Topic (queue).  A Publisher can put messages on the Topic, and a Subscriber can read messages from the Subscription on a Topic. In this article, I will first use the … GCP: publishing and reading from Google PubSub Topic using Python client libraries

GCP: Installing KEDA on a GKE cluster with workload identity and testing Scalers


Categories: Hyperscaler, Kubernetes

KEDA is an open-source event-driven autoscaler that greatly enhances the abilities of the standard HorizontalPodAutoscaler.  It can scale based on internal metrics as well as external Scaler sources. In this article, I will illustrate how to install KEDA on a GKE cluster that has Workload Identity enabled, and then how to configure KEDA scaling events … GCP: Installing KEDA on a GKE cluster with workload identity and testing Scalers

GCP: historical log of GKE cluster and nodepool upgrades and scaling


Categories: Hyperscaler

Although the simple ‘gcloud container operations list‘ command is the easiest way to find recent upgrade events on your GKE cluster or nodepool, it returns only the recent events and does not provide a historical record. If you need to look at historical events, you can use Logs Explorer web UI or use the ‘gcloud … GCP: historical log of GKE cluster and nodepool upgrades and scaling

Bash: calculating number of days till certificate expiration using openssl


Categories: Scripting

The openssl utility can be used to show the details of a certificate, including its ‘Not After’ expiration date in string format.  This can be transformed into “how many days till expiration” with a bit of Bash date math. Create test certificate and key Using a line provided by Diego Woitasen for non-interactive self-signed certification … Bash: calculating number of days till certificate expiration using openssl

GitLab: URL shortcut to override pipeline variable values


Categories: DevOps

GitLab pipelines are a convenient way to expose deployment/delivery tasks.  But with their rudimentary web UI for variable input, it can be challenging for users to populate the required list of variables. One way of making it more convenient for end-users is to provide them a URL pre-populated with the specific branch and pipeline variable … GitLab: URL shortcut to override pipeline variable values

OpenTofu: installing OpenTofu on Debian/Ubuntu


Categories: DevOps

Terraform has now been open-source and forked with the OpenTofu project.  The ‘tofu’ binary is a drop-in replacement for terraform, and this article will show you how to install on Debian/Ubuntu. After installation, we will then use the Debian/Ubuntu Alternatives concept to supersede existing calls to ‘terraform’ to instead invoke ‘tofu’. Setup OpenTofu apt repository … OpenTofu: installing OpenTofu on Debian/Ubuntu

Terraform: external yaml file as a contribution model for outside teams


Categories: DevOps

If you are using Terraform as a way to provide infrastructure/services to outside teams, your Terraform definitions and variables may initially be owned by your team alone, with all “tf apply” operations done by trusted internal group members at the CLI. But once there is a certain amount of maturity in the solution, the Terraform … Terraform: external yaml file as a contribution model for outside teams

yq: updating deeply nested elements


Categories: Linux

Mike Farah’s yq yaml processor has a a rich set of operators and functions for advanced usage.  In this article, I will illustrate how to update deeply nested elements in yaml.  This can be done for both known paths as well as arbitrarily deep paths. Sample yaml We will use the following yaml files to … yq: updating deeply nested elements

yq: validate yaml syntax


Categories: Linux

Mike Farah’s yq yaml processor has a a full-featured validation command that is very detailed in its reporting, but the yaml specification itself is very lenient, which means yq may accept scenarios you did not expect (e.g. an empty file). yq -v file.yaml >/dev/null ; echo “final result = $?” Luckily, the yq tips-and-tricks section … yq: validate yaml syntax

Ubuntu: pyenv for managing multiple Python versions and environments


Categories: Python

Keeping the Ubuntu system-level Python version and modules independent from those desired at each project level is a difficult task best managed by a purpose-built tool. There are many solutions in the Python ecosystem, but one that stands out for simplicity is pyenv and pyenv-virtualenv. pyenv allows you to install and switch between different versions … Ubuntu: pyenv for managing multiple Python versions and environments

Ubuntu: LLama2 model on Ubuntu using llama.cpp


Categories: Linux

It is relatively easy to experiment with a base LLama2 model on Ubuntu, thanks to llama.cpp written by Georgi Gerganov. The llama.cpp project provides a C++ implementation for running LLama2 models, and works even on systems with only a CPU (although performance would be significantly enhanced if using a CUDA-capable GPU).

Mac: LLama2 model on Apple Silicon and GPU using llama.cpp


Categories: Mac

It is relatively easy to experiment with a base LLama2 model on M family Apple Silicon, thanks to llama.cpp written by Georgi Gerganov. The llama.cpp project provides a C++ implementation for running LLama2 models, and takes advantage of the Apple integrated GPU to offer a performant experience (see M family performance specs).