Development

Mac: tensorflow-metal pip module on M1 chip for GPU support

Enabling the use of the GPU on your Mac M1 with the tensorflow-metal plugin can be challenging because there is a lot of conflicting documentation and older forum questions and replies. I’ve written this article for a Mac M1 running on macOS Sequoia 15.1.1.  As of December 2024, you should pair Python 3.11 with TensorFlow Mac: tensorflow-metal pip module on M1 chip for GPU support

Github: security scanning built into GitHub Actions image build

Github Actions provide the ability to define a build workflow, and for projects that are building an OCI (Docker) image, there are custom actions available for running the Trivy container security scanner. In this article, I will show you how to modify your GitHub Action to run the Trivy security scanner against your image, and Github: security scanning built into GitHub Actions image build

GitLab: security scanning built into GitLab Pipelines image build

GitLab Pipelines provide the ability to define a build workflow, and for projects that are building an OCI (Docker) image, there is a convenient method for doing container security scanning as part of the build process. Include Container Scanning As described in the official documentation, add the following include to your .gitlab-ci.yml pipeline definition. include: GitLab: security scanning built into GitLab Pipelines image build

Docker: building multi-platform images that use fat manifest list/index

Docker can build multi-platform images that use a manifest index (fat manifest list) by using the Docker buildx command with backing containerd runtime and QEMU for cross-platform emulation. Using a manifest index for multi-platform images simplifies application level orchestration by using the same name and version for all architectures.  For example: # same image name Docker: building multi-platform images that use fat manifest list/index

GitLab: generating URL that can be used for Merge Request from fork to upstream

The forked workflow is popularized by the Open Source community where your personal contributions are made by having your own personal fork of a repository and pushing a GitLab Merge Request to a central repository. A GitLab Merge Request can be submitted from the web UI by clicking on “Merge requests” and manually selecting the GitLab: generating URL that can be used for Merge Request from fork to upstream

GCP: Cloud Run with build trigger coming from remote GitHub repository

GCP build triggers can easily handle Continuous Deployment (CD) when the source code is homed in a Google Cloud Source repository.  But even if the system of record for your source is a remote GitHub repository, these same type of push and tag events can be consumed if you configure a connection and repository link. GCP: Cloud Run with build trigger coming from remote GitHub repository

Github: automated build and publish of containerized GoLang app with Github Actions

Github Actions provide the ability to define a build workflow based on Github repository events.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a statically-linked GoLang binary that when Github: automated build and publish of containerized GoLang app with Github Actions

Gradle: running more than one command in an Exec task

A Gradle Exec task will only run the last ‘commandLine’ defined inside its block.  Putting multiple entries inside its block will not run multiple commands. As an example, if you run the following Gradle task. task willOnlyRunLast(type: Exec) { commandLine “echo”, “first” commandLine “echo”, “second” commandLine “echo”, “last” } The task above will only echo Gradle: running more than one command in an Exec task

Github: automated Github release for Spring Boot jar using Github Actions

Github Actions provide the ability to define a build workflow directly in Github.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a simple Spring Boot web application that when Github: automated Github release for Spring Boot jar using Github Actions

Github: automated build and publish of containerized Spring Boot app using GitHub Actions

Github Actions provide the ability to define a build workflow directly in Github.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a simple Spring Boot web application that when Github: automated build and publish of containerized Spring Boot app using GitHub Actions

Github: locally invoked release process for a Gradle built Java Spring Boot project

The GitHub “Release” page for a repository can provide your consumers a convenient way to download a binary version of your software as well as track the latest changes and enhancements. In this article, I will show how to invoke a local release process for a Java Spring Boot jar built with Gradle.  A new Github: locally invoked release process for a Gradle built Java Spring Boot project

GoLang: Running a Go binary as a systemd service on Ubuntu 22.04

The Go language with its simplicity, concurrency support,  rich package ecosystem, and ability to compile down to a single binary is an attractive solution for writing services on Ubuntu. However, the Go language does not natively provide a reliable way to daemonize itself.  In this article I will describe how to take a couple of simple Go language programs GoLang: Running a Go binary as a systemd service on Ubuntu 22.04

GoLang: Installing the Go Programming language on Ubuntu 22.04

Update Sept 2024: changed the page parsed for latest version, validated installation of go1.23.1 on Ubuntu22.04 The Go programming language consistently ranks as one of the most popular languages in developer surveys.  In fact, Kubernetes as well as most of the CNF projects are written in Go.  And it compiles down to machine code, which GoLang: Installing the Go Programming language on Ubuntu 22.04

Java: Spring Security OAuth2/OIDC protecting Client App and Resource Server

The Spring Security framework provides a robust and customizable framework for authentication and authorization for Spring based applications. Using Spring Security, a Spring developer can add OIDC authentication and OAuth2 protection of resources by including the libraries in the build, configuring the Spring application.yml, and enabling various component configurations and annotations. In this article, I Java: Spring Security OAuth2/OIDC protecting Client App and Resource Server

GoLang: Installing the Go Programming language on Ubuntu 20.04

See the newer version of this article, “Installing the Go Programming language on Ubuntu 22.04“. The Go programming language consistently ranks as one of the most popular languages in developer surveys.  In fact, Kubernetes as well as most of the CNF projects are written in Go.  And it compiles down to machine code, which has GoLang: Installing the Go Programming language on Ubuntu 20.04

Ubuntu: Installing .NET SDK 6 on Ubuntu 20.04

The Microsoft .NET SDK is an open-source development platform for developing applications across multiple architectures and operating systems. In this article, I will show you how to install the .NET SDK on Ubuntu 20.04 and then create/compile/run a simple web application. Ubuntu 22 will have the dotnet-sdk available in the default Ubuntu apt repositories, but Ubuntu: Installing .NET SDK 6 on Ubuntu 20.04

Java: Creating Docker image for Spring Boot web app using gradle

While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator Java: Creating Docker image for Spring Boot web app using gradle

Java: adding custom health indicator to Spring Boot Actuator

If you have enabled Actuator in your Spring Boot application, you can add custom status metrics to the standard health check at ‘/actuator/health’. Additionally, your custom health indicator can signal an UP/DOWN status that propagates to the main level status and can then be used by an external monitoring/alerting solutions or as an indicator to Java: adding custom health indicator to Spring Boot Actuator

Java: Adding custom metrics to Spring Boot Micrometer Prometheus endpoint

If you have enabled Actuator and the ‘micrometer-registry-prometheus’ dependency in your Spring Boot application, then you will have a new ‘/actuator/prometheus’ web endpoint that returns general information about threads, garbage collection, disk, and memory. This information is delivered in standard prometheus formatting as plaintext, with one metric per line. This is exactly the type of Java: Adding custom metrics to Spring Boot Micrometer Prometheus endpoint

Ubuntu: install latest git client from PPA to fix ‘unsafe repository’ errors

Since the announcement of CVE-2022-24765, newer git clients from the Ubuntu security and archive package repositories may throw errors about “unsafe repository … is owned by someone else” if directories are not owned by your personal user id. First, try to resolve the issue by running the command suggested in the error message. # attempt Ubuntu: install latest git client from PPA to fix ‘unsafe repository’ errors

Git: cloning a git repository from one location to another

Most Git providers-as-a-service have administrative functions for renaming, moving, and even importing repositories from other provider URLs. However, it is also valid to perform these operations manually by repointing the origin and then pushing all commits and tags to a new repository URL. # make sure all changes are pushed first git push # check Git: cloning a git repository from one location to another

GoLang: Go modules for package management during a multi-stage Docker build

My previous article on multi-stage builds to create  Docker images for Go laid the foundation for using an intermediate image as the builder your Go binary.  However, this example was intentionally simplistic and did not address package and dependency management. Since the release of Go 1.11, the standard tooling has natively supported the concept of GoLang: Go modules for package management during a multi-stage Docker build

GoLang: Using multi-stage builds to create clean Docker images

The Go programming language is a natural fit for containers because it can compile down to a single statically-linked binary.  And if you place that single executable on top of scratch, a distroless image, or a small image like alpine, your final image has a minimal footprint which is great for consumption and reuse. But the GoLang: Using multi-stage builds to create clean Docker images

GoLang: Installing the Go Programming language on Ubuntu

The Go programming language has gotten considerable momentum, and the fact that it compiles down to a single statically linked binary has made it popular in containers, where a single executable binary fits the execution model perfectly. This article will detail installation on Ubuntu with the standard hello world validation.

Git: BFG for removing secrets from entire git history

If you accidentally pushed a secret or password into a git repository, the BFG Repo-Clean utility is a convenient option for removing all traces of the secret from the entire git commit history. It is also possible to use ‘git-filter-branch‘, but I find BFG more convenient and faster.

Git: client error, server certificate verification failed

Especially with private git repositories that may be self-signed or have private CA, you may get the following error from the git client after a certificate has been updated: fatal: unable to access ‘https://git.mycompany.com/myuser/myrepo.git/’: server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none This means that the git client cannot verify the integrity of the certificate Git: client error, server certificate verification failed

Git: Contributing to a git project using a pull request

With so many critical projects available in public git repositories like github, it is important to be able to contribute back additional functionality, tests, and documentation to the original project.  While most projects do not allow a direct commit, contributions can be made by submitting a pull request to original repository. Often times, repositories are Git: Contributing to a git project using a pull request

Git: Sharing a single git controlled folder among a group under Linux

With the modern mantra of “everything is code”, operations and network teams must come to terms with how they want to work with source control in a team environment. Imagine a repository that contains configuration templates and scripts for maintaining an application or appliance.  For a multi-member operations team who shares the responsibility for this Git: Sharing a single git controlled folder among a group under Linux

Python: Using pip with a squid proxy

In a production datacenter it would not be uncommon for internet access to be limited to domains whitelisted on a web proxy such as Squid.  If this is the case, and you are using pip to install packages, then you will need to: Have your Squid administrators whitelist pypi.python.org Add the “–proxy” switch when invoking Python: Using pip with a squid proxy

MongoDB: Installing a MongoDB client on Ubuntu

In order to communicate with MongoDB using its default TCP protocol on port 27017, you will need a MongoDB client.  There are many language bindings available, but in this article we’ll focus on the client available from the “mongodb-org-shell” Debian package.