If you need to enable JDWP debugging of your Spring Boot web application in an IDE such as Eclipse, you can append the ‘debug-jvm’ flag to the gradle task, and then from your IDE enable a remote debugging connection to port 5005. I will walk you through a full example, but very quickly, here is … Gradle: interactive JDWP debugging of bootRun gradle task in Eclipse IDE
The Spring Security framework provides a robust and customizable framework for authentication and authorization for Spring based applications. Using Spring Security, a Spring developer can add OIDC authentication and OAuth2 protection of resources by including the libraries in the build, configuring the Spring application.yml, and enabling various component configurations and annotations. In this article, I … Java: Spring Security OAuth2/OIDC protecting Client App and Resource Server
Windows AD FS provides enterprise Identity and Authentication services, which includes support for OAuth2 and OIDC authentication flows. In this article, we will create and configure an ADFS Application group that supports the Authorization Code flow. This flow allows an application to access a 3rd party API on behalf of the end user as illustrated … Microsoft: configuring an Application Group for OAuth2/OIDC on ADFS 2019
The Go programming language consistently ranks as one of the most popular languages in developer surveys. In fact, Kubernetes as well as most of the CNF projects are written in Go. And it compiles down to machine code, which has made it popular in containers like Docker where a single executable binary fits the execution model … GoLang: Installing the Go Programming language on Ubuntu 20.04
The Microsoft .NET SDK is an open-source development platform for developing applications across multiple architectures and operating systems. In this article, I will show you how to install the .NET SDK on Ubuntu 20.04 and then create/compile/run a simple web application. Ubuntu 22 will have the dotnet-sdk available in the default Ubuntu apt repositories, but … Ubuntu: Installing .NET SDK 6 on Ubuntu 20.04
If the gradle wrapper for your project is broken, you can reinstall it with the “gradle wrapper” command. For example, below is a common error if you have cloned a git repository where the owner has not pushed all the necessary gradle components into the repo. $ ./gradlew –info Error: Could not find or load … Gradle: fixing the gradle wrapper for a Java project
Based on my previous article on creating a Windows2019 Windows Domain Controller from a Sysprep template, we now want to move on to creating an Windows 2019 ADFS server in a similar fashion. Once this new Windows 2019 guest OS is provisioned, we will prepare it and then run Powershell scripts that will enable the … KVM: Creating a Windows2019 ADFS server using Powershell
Based on my previous article on creating a Windows2019 base template using Sysprep, we can now move on to creating a Windows Domain Controller using this template as a backing file. Once this new Windows 2019 guest OS is provisioned, we will prepare it and then run Powershell scripts that will enable the Active Directory … KVM: creating a Windows2019 Domain Controller using Powershell
In the world of Linux containers where deployment takes on the order of seconds, even the best-case scenario for spinning up a new Windows host can seem like an eternity. Clearly, you don’t want to wait for the entire Windows install process each time you bring up a Windows guest OS. Even automated, this would … KVM: configuring a base Window2019 instance with Sysprep
The Kubernetes Dashboard provides a convenient web interface for viewing cluster resources. However, if you are logged using a token tied to the ‘cluster-admin’ role, you will have privileges beyond what are typically necessary. In this article, I will show you how to create a ServiceAccount and ClusterRole with limited privileges that can be used … Kubernetes: accessing the Kubernetes Dashboard with least privilege
In a previous article, I provided a full step-by-step example of creating and running a Spring Boot web application that was packaged as a Docker image and run using the local Docker daemon. In this article, I want to focus on the same Spring Boot REST web application, but without using Docker. Instead we will … Java: creating OCI-compatible image for Spring Boot web using buildah
buildah is a tool that can create OCI compatible (Docker) images without the Docker daemon. podman can run these images without the need for a Docker daemon. The buildah and podman packages are available on the Debian Bullseye testing branch, but are not available from an Ubuntu 20.04 LTS release because these test packages could … Buildah: Installing buildah and podman on Ubuntu 20.04
If you have a domain that you want CoreDNS to forward to a specific upstream DNS, you can add a block to the CoreDNS configmap. Adding this domain block to the CoreDNS configmap and restarting the CoreDNS pods is sufficient will make the change take affect. You may need to restart client pods if DNS … Kubernetes: custom upstream for domain with CoreDNS
The default setting for pod DNS resolution has CoreDNS use the settings from the underlying OS of the worker node. If your Kubernetes VMs are joined to multiple networks or search domains, this can cause unexpected results as well as performance issues. If you are using K3s, you can provide an independent resolv.conf file for … Kubernetes: independent resolv.conf for CoreDNS with K3s
The default setting for pod DNS resolution has CoreDNS use the settings from the underlying OS of the worker node. If your Kubernetes VMs are joined to multiple networks or search domains, this can cause unexpected results as well as performance issues. If you are using kubeadm, you can provide an independent resolv.conf file for … Kubernetes: independent resolv.conf for CoreDNS with kubeadm
The kube-prometheus-stack bundles the Prometheus Operator, monitors/rules, Grafana dashboards, and AlertManager needed to monitor a Kubernetes cluster. But there are customizations necessary to tailor the Helm installation for a Kubernetes cluster built using kubeadm. In this article, I will detail the necessary modifications to deploy a healthy monitoring stack on a kubeadm cluster.
If you are running the Prometheus Operator (e.g. with kube-prometheus-stack) then you can specify additional scrape config jobs to monitor your custom services. An additional scrape config uses regex evaluation to find matching services en masse, and targets a set of services based on label, annotation, namespace, or name. Note that adding an additional scrape … Prometheus: monitoring services using additional scrape config for Prometheus Operator
If you are running the Prometheus Operator as part of your monitoring stack (e.g. kube-prometheus-stack) then you can have your custom Service monitored by defining a ServiceMonitor CRD. The ServiceMonitor is an object that defines the service endpoints that should be scraped by Prometheus and at what interval. In this article, we will deploy a … Prometheus: monitoring a custom Service using ServiceMonitor and PrometheusRule
If your Grafana deployment is using a sidecar to watch for new dashboards defined as a ConfigMap, then adding a dashboard is a dynamic operation that can be done without even restarting the pod. If you have deployed the Prometheus/Grafana stack with kube-prometheus-stack, then you can check for the existence of the ‘grafana-sc-dashboard’ sidecar using: … Prometheus: adding a Grafana dashboard using a ConfigMap
If you need to validate your AlertManager routing configuration by sending a test alert through AlertManager, you can port-forward the AlertManager pod and send it from curl on your development host. Real alerts typically have scrape delays and then durations that must be met, so this is a way of getting almost immediate feedback on … Prometheus: sending a test alert through AlertManager
While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator … Java: build OCI compatible image for Spring Boot web app using jib
The kube-prometheus-stack bundles AlertManager for taking action on Prometheus alerts. And if you are customizing the Heml custom values file to configure email alerting, there are multiple options available. The simplest is to allow the system to fallback to using the default subject and html templates. But if you need to tailor the email content … Prometheus: external template for AlertManager html email with kube-prometheus-stack
The kube-prometheus-stack bundles Prometheus, Grafana, and AlertManager for monitoring a Kubernetes cluster. By default, the Ingress of these services is disabled. In this article I will show you how to expose these services with NGINX Ingress either via subdomain (e.g. prometheus.my.domain) or web context (e.g. my.domain/prometheus). You would not want to expose these monitoring applications … Prometheus: exposing Prometheus/Grafana as Ingress for kube-prometheus-stack
The kube-prometheus-stack bundles the Prometheus Operator, monitors/rules, Grafana dashboards, and AlertManager needed to monitor a Kubernetes cluster. But there are customizations necessary to tailor the Helm installation for K3s, a lightweight Kubernetes installation. In this article, I will detail the necessary modifications to deploy a healthy monitoring stack on a K3s cluster.
If you have a Kubernetes yaml manifest that contains multiple documents, targeting a single document for modification while still outputting the other documents untouched can be a challenge. As an example, consider the simple example below were you have a single yaml file that contains: a Namespace, Deployment, and DaemonSet. And we want to add … Kubernetes: targeting the addition of array items to a multi-document yaml manifest
Spring Boot excels as a framework that makes it convenient to expose a set of REST based services. But once an application is developed, it is so trivial to create a new resource or modify a method signature that it becomes difficult to keep the documentation up-to-date so that clients can properly consume the REST … Java: Spring Boot REST service with OpenAPI/Swagger documentation
A Kubernetes liveness and readiness probe is how the kubelet determines health of a pod. This is often times as simple as checking the ability to reach the main service port over TCP or HTTP. But if you are using Spring Boot and have enabled the Actuator dependency, you have the ability to create even … Kubernetes: liveness probe for Spring Boot with custom Actuator health check
While working on your Spring Boot web application locally, gradle provides the ‘bootRun’ for a quick development lifecycle and ‘bootJar’ for packaging all the dependencies as a single jar deliverable. But for most applications these days, you will need this packaged into an OCI compatible (i.e. Docker) image for its ultimate deployment to an orchestrator … Java: Creating Docker image for Spring Boot web app using gradle
If you have enabled Actuator in your Spring Boot application, you can add custom status metrics to the standard health check at ‘/actuator/health’. Additionally, your custom health indicator can signal an UP/DOWN status that propagates to the main level status and can then be used by an external monitoring/alerting solutions or as an indicator to … Java: adding custom health indicator to Spring Boot Actuator
If you have enabled Actuator and the ‘micrometer-registry-prometheus’ dependency in your Spring Boot application, then you will have a new ‘/actuator/prometheus’ web endpoint that returns general information about threads, garbage collection, disk, and memory. This information is delivered in standard prometheus formatting as plaintext, with one metric per line. This is exactly the type of … Java: Adding custom metrics to Spring Boot Micrometer Prometheus endpoint
If you have enabled Actuator in your Spring Boot application, then in addition to the standard endpoints for health and metrics, you can also add your own endpoint to expose custom metrics. In its simplest form, this is done by adding a @Component and @Endpoint annotation to a custom class that returns the metrics, and … Java: exposing a custom Actuator endpoint with Spring Boot
Although kubectl has the native ability to query on labels, it does not provide the same support for annotations. This is unfortunate, because the need to target deployments and services by annotation is commonplace. You can, however, use the power of jsonpath to find these objects. Below is an example of finding all Services that … Kubernetes: query by annotation with kubectl
It is easy to export the full manifest of an object from a Kubernetes cluster, but it will have extraneous accounting fields that not only make it hard to visually assess and compare to its original form, but can also cause a re-apply to fail. Using a combination of the jq and yq utilities, we … Kubernetes: export a clean yaml manifest that can be re-imported
If you are configuring Istio/ASM ingress gateways with a BackendConfig for specifying health checks, timeouts, or Cloud Armor policies, then you need to ensure that your GKE cluster has the HttpLoadBalancing feature enabled. If this feature is not enabled, you will see an error message like below when attempting to apply the BackendConfig manifest: unable … GCP: Enable HttpLoadBalancing feature on Cluster to avoid errors when applying BackEndConfig
Update Sep 2023: tested on Kubeadm 1.28, MetalLB v0.13.9, ingress-nginx v1.8.1 Kubeadm is a tool for quickly bootstrapping a minimal Kubernetes cluster. In this article, I will show you how to deploy a three-node Kubeadm cluster on Ubuntu 22 nodes that are created using Terraform and a local KVM libvirt provider. Ansible is used for … KVM: kubeadm cluster on KVM using Ansible
With Workload Identity enabled on a GKE cluster, your container can access Google Cloud API services (Compute Engine, Storage, etc.) using a Kubernetes Service Account (KSA). This is done by having the container run as the KSA, where the KSA has been bound to the Google Service Account (GSA). This is the recommended way of … GCP: running a container on a GKE cluster using Workload Identity
A Kubernetes Service Account (KSA) can be used to provide least-privileged access to a pod for a cluster that has Role-based access control (RBAC) enabled. This is done by making the KSA the subject in an RBAC role. But it can be challenging to discover and test whether the KSA has the correct set of … Kubernetes: testing RBAC authorization of a Kubernetes Service Account
When configuring networks and loadbalancers, sometimes you need the network CIDR block used by Services of a Kubernetes cluster. There are various ways to pull this information from different Kubernetes implementations, but one trick that works across implementations is looking at the error message from kubectl if you attempt to create a service at an … Kubernetes: retrieving services and pods network CIDR block from cluster
GKE cluster upgrades do not need to be a manual process. GKE clusters can be auto upgraded by subscribing the cluster to an appropriate release channel and assigning a sensible maintenance window. As long as adequate pod disruption budgets, replicas, and ingress are configured, these upgrades can happen without interrupting availability. To check the current … GCP: Enabling autoUpgrade for node-pools to reduce manual maintenance
Anthos GKE on-prem is a managed platform that brings GKE clusters to on-premise datacenters. This product offering brings best practice security measures, tested paths for upgrades, basic monitoring, platform logging, and full enterprise support. Setting up a platform this extensive requires many steps as officially documented here. However, if you want to practice in a … Kubernetes: Anthos GKE on-prem 1.11 on nested VMware environment