pipeline

GitLab: security scanning built into GitLab Pipelines image build


Categories: Containers, Development, DevOps

GitLab Pipelines provide the ability to define a build workflow, and for projects that are building an OCI (Docker) image, there is a convenient method for doing container security scanning as part of the build process. Include Container Scanning As described in the official documentation, add the following include to your .gitlab-ci.yml pipeline definition. include: … GitLab: security scanning built into GitLab Pipelines image build

GitLab: URL shortcut to override pipeline variable values


Categories: DevOps

GitLab pipelines are a convenient way to expose deployment/delivery tasks.  But with their rudimentary web UI for variable input, it can be challenging for users to populate the required list of variables. One way of making it more convenient for end-users is to provide them a URL pre-populated with the specific branch and pipeline variable … GitLab: URL shortcut to override pipeline variable values

GitLab: Continuous Deployment with Agent for Kubernetes and GitLab pipeline


Categories: DevOps

GitLab pipelines are frequently used for the building of binaries and publishing of images to container registries, but do not always follow through with Continuous Deployment to a live environment. One reason is that pipelines do not usually have access to the internal systems where these applications are meant to be deployed. In this article, … GitLab: Continuous Deployment with Agent for Kubernetes and GitLab pipeline

GitLab: self-managed runner for CI/CD jobs on GCP VM instances


Categories: DevOps

The globally shared set of GitLab runners for CI/CD jobs works well for building binaries, publishing images, and reaching out to publicly available endpoints for services and infrastructure building. But the ability to run a private, self-managed runner can grant pipelines entirely new levels of functionality on several fronts: Can communicate openly to private, internal … GitLab: self-managed runner for CI/CD jobs on GCP VM instances

GitLab: automated build and publish of multi-platform container image with GitLab pipeline


Categories: Containers, DevOps

GitLab CI/CD pipelines can be used to automatically build and push Docker images to the GitLab Container Registry. Beyond building a simple image, in this article I will show how to define a workflow that builds and pushes a multi-platform image (amd64,arm64,arm32) with manifest index to the GitLab Container Registry.  This is enabled by using … GitLab: automated build and publish of multi-platform container image with GitLab pipeline

Github: automated build and publish of multi-platform container image with Github Actions


Categories: Containers, DevOps

Github Actions provide the ability to define a build workflow based on Github repository events.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article, I will show how to define workflow steps that build and push a … Github: automated build and publish of multi-platform container image with Github Actions

Github: automated build and publish of containerized Spring Boot app using GitHub Actions


Categories: Containers, Development, Java

Github Actions provide the ability to define a build workflow directly in Github.  The workflow steps are defined as yaml and can be triggered by various events, including a code push, branch, or tagging in the repository. In this article I will detail the steps of creating a simple Spring Boot web application that when … Github: automated build and publish of containerized Spring Boot app using GitHub Actions

Bash: accepting a remote host fingerprint with ssh-keyscan


Categories: Automation, Linux

For security reasons, you should be very aware that accepting a remote host fingerprint automatically is a procedure that should be considered high-risk. But if you are working with automated infrastructure or pipelines where human intervention is not possible and the constructed entities are being built in a secure fashion with guaranteed provenance, then ssh-keyscan … Bash: accepting a remote host fingerprint with ssh-keyscan

Bash: using multiple values from an input pipeline to construct and execute a command


Categories: Scripting

If you have multiple values coming through the Bash input pipeline, it can be difficult to process these into a complex, formatted set of arguments unless you use intermediate temporary files. But one way to neatly put together a complex set of arguments from an input pipeline with multiple values is to use awk printf … Bash: using multiple values from an input pipeline to construct and execute a command

ELK: metadata fields in Logstash for grok and conditional processing


Categories: Linux, Logging

When building complex, real-world Logstash filters, there can be a fair bit of processing logic.  There are typically multiple grok patterns as well as fields used as flags for conditional processing. The problem is, these intermediate extracted fields and processing flags are often ephemeral and unnecessary in your ultimate persistent store (e.g. ElasticSearch), but they will be inserted … ELK: metadata fields in Logstash for grok and conditional processing

Jenkins: Setting up a continuous integration server on Ubuntu


Categories: DevOps, Linux

Jenkins is the open-source automation server that is critical in building a continuous integration and delivery pipeline.  It is extensible and has a wealth of plugins that  integrate with numerous enterprise systems. Here are the detailed steps for installing a Jenkins server on Ubuntu.

ELK: Architectural points of extension and scalability for the ELK stack


Categories: DevOps

The ELK stack (ElasticSearch-Logstash-Kibana), is a horizontally scalable solution with multiple tiers and points of extension and scalability. Because so many companies have adopted the platform and tuned it for their specific use cases, it would be impossible to enumerate all the novel ways in which scalability and availability had been enhanced by load balancers, … ELK: Architectural points of extension and scalability for the ELK stack

ELK: Feeding the logging pipeline


Categories: DevOps

The most varied point in an ELK (Elasticsearch-Logstash-Kibana) stack is the mechanism by which custom events and logs will get sent to Logstash for processing. Companies running Java applications with logging sent to log4j or SLF4J/Logback will have local log files that need to be tailed.  Applications running in containers may send everything to stdout/stderr, … ELK: Feeding the logging pipeline