Fabian Lee : Software Engineer

Mac: bare-metal virtualization on Apple Silicon with virtualbuddy

January 5, 2024
Categories: Mac, Virtualization

The Apple Virtualization Framework (AVF) provides the ability to run completely independent virtual machines on top of M family Apple Silicon. For example, you can run multiple versions of MacOS virtualized for validating an application or its dependencies against different environments. Additionally, cloning an existing VM (with little cost thanks to APFS copy-on-write) allows you … Mac: bare-metal virtualization on Apple Silicon with virtualbuddy

Docker: QEMU emulation to run arm64 images from native amd64 host

September 15, 2023
Categories: Containers, Virtualization

If you want to use Docker to build cross-platform images, the first step is to enable QEMU to run images targeted at others architectures via emulation. I assume you have installed Docker CE and its containerd runtime as described here, and are running on a x86_64 host.

Terraform: creating an Ubuntu 22 template and then guest VM in vCenter

October 12, 2022
Categories: Linux, Virtualization

In this article I will demonstrate how to create an Ubuntu 22 template in vCenter. Then use Terraform to create a vSphere VM based on this template. The VM template creation is done by manually stepping through the Ubuntu server ISO installation wizard, followed by a set of preparation steps. Then Terraform is used to … Terraform: creating an Ubuntu 22 template and then guest VM in vCenter

KVM: Creating a bridged network with NetPlan on Ubuntu 22.04

September 20, 2022
Categories: Linux, Virtualization

In order to expose KVM virtual machines on the same network as your bare-metal Host, you need to enable bridged networking. In this article, I’ll show how to implement KVM bridged networking on Ubuntu 22.04 using Netplan. This bridged network will expose the KVM Guest OS as a peer on the upstream network, with no … KVM: Creating a bridged network with NetPlan on Ubuntu 22.04

KVM: Creating a Windows2019 ADFS server using Powershell

August 8, 2022
Categories: Virtualization

Based on my previous article on creating a Windows2019 Windows Domain Controller from a Sysprep template, we now want to move on to creating an Windows 2019 ADFS server in a similar fashion. Once this new Windows 2019 guest OS is provisioned, we will prepare it and then run Powershell scripts that will enable the … KVM: Creating a Windows2019 ADFS server using Powershell

KVM: creating a Windows2019 Domain Controller using Powershell

August 6, 2022
Categories: Virtualization

Based on my previous article on creating a Windows2019 base template using Sysprep, we can now move on to creating a Windows Domain Controller using this template as a backing file. Once this new Windows 2019 guest OS is provisioned, we will prepare it and then run Powershell scripts that will enable the Active Directory … KVM: creating a Windows2019 Domain Controller using Powershell

KVM: configuring a base Window2019 instance with Sysprep

August 6, 2022
Categories: Virtualization

In the world of Linux containers where deployment takes on the order of seconds, even the best-case scenario for spinning up a new Windows host can seem like an eternity. Clearly, you don’t want to wait for the entire Windows install process each time you bring up a Windows guest OS. Even automated, this would … KVM: configuring a base Window2019 instance with Sysprep

KVM: Deploy the VMware vCenter 7.0 appliance using the CLI installer

November 18, 2021
Categories: Linux, Virtualization

If you have just virtualized the VMware ESXi server on top of KVM, the next step will be to install vCenter for its centralized control and additional feature set and management capabilities. In my last article we took KVM running on bare metal and deployed an ESXi 7.0 host on top of it. In this … KVM: Deploy the VMware vCenter 7.0 appliance using the CLI installer

KVM: Deploying a nested version of VMware ESXi 7.0 on KVM

November 18, 2021
Categories: Linux, Virtualization

If you are running KVM on an Ubuntu server, you already have an excellent Type 1 virtualization engine. Luckily, if you need to test something specific to VMware you can always run ESXi 7.0 nested inside a KVM virtual machine. In this article, I’ll be using a bare metal server running Ubuntu and KVM as … KVM: Deploying a nested version of VMware ESXi 7.0 on KVM

KVM: running qemu-img info without exclusive access using force-share flag

September 23, 2021
Categories: Virtualization

By default, ‘qemu-image info’ will throw an error if it cannot get exclusive access to the disk file it is trying to read. $ sudo qemu-img info mydisk.qcow2 qemu-img: Could not open ‘mydisk.qcow2’: Failed to get shared “write” lock Is another process using the image [mydisk.qcow2]? Although it is not listed in the man page, … KVM: running qemu-img info without exclusive access using force-share flag

VMware: Extending datastore1 on a ESXi host nested within KVM

August 22, 2021
Categories: Virtualization

If you followed my previous article on deploying a nested ESXi on KVM, you may eventually need to increase the size of the default datastore1 as you continue your experimentation and deployments. Do not simply increase the size of the original backing KVM qcow2 disk (qemu-img resize), because then you would need to get this … VMware: Extending datastore1 on a ESXi host nested within KVM

Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter

August 16, 2021
Categories: Linux, Virtualization

In this article I will demonstrate how to create an Ubuntu 20 Focal template in vCenter. Then use Terraform to create a vSphere VM based on this template. The VM template creation is done by manually stepping through an installation using the minimal Ubuntu server ISO followed by a set of preparation steps. Then Terraform … Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter

Ubuntu: site-to-site VPN with WireGuard

May 31, 2021
Categories: Linux, Virtualization

WireGuard aims to be the successor to IPsec and more performant than OpenVPN. It is a general purpose VPN that is secure enough for servers, yet light enough to run on embedded devices. In this article, I will show how to install WireGuard on two Ubuntu servers in completely different hyperscalers that are linked by … Ubuntu: site-to-site VPN with WireGuard

Terraform: provisioning AWS servers in both public and private subnets

May 30, 2021
Categories: Virtualization

It is relatively straightforward to create an AWS public subnet where the compute instances have access to the public internet via the default internet gateway. But once you start building private subnets behind it, you must start considering security groups, routing, and the NAT gateways required to reach public services. In this article, I will … Terraform: provisioning AWS servers in both public and private subnets

Terraform: provisioning an RDP enabled Windows server in Azure

May 30, 2021
Categories: Virtualization

The ‘azurerm‘ Terraform provider allows you to build a Windows server in Microsoft’s Azure hyperscaler. However, in order to use this provisioner, you must first install the Azure CLI. And in line with automation best practices we will use a Service Account (Principal) to create the networks, security rules, and compute instances. When complete, you’ll … Terraform: provisioning an RDP enabled Windows server in Azure

Azure: installing the Azure CLI on Ubuntu

May 29, 2021
Categories: Virtualization

Managing resources in Azure from the command line can be done natively from Ubuntu using the Azure CLI. First, add the prerequisite packages. sudo apt-get update sudo apt-get install ca-certificates curl apt-transport-https lsb-release gnupg -y Then install the Microsoft signing key and add the custom repository. curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg –dearmor | sudo tee … Azure: installing the Azure CLI on Ubuntu

Terraform: invoking a startup script for a GCE google_compute_instance

May 28, 2021
Categories: Virtualization

You can bake a startup script directly into the creation of your GCE compute instance when using Terraform. Although complex post-configuration should be left to tools such as Ansible, essential bootstrap type commands or custom routes for instances in private subnets are reasons why you might need to use this hook. Below is an example … Terraform: invoking a startup script for a GCE google_compute_instance

Terraform: invoking a startup script for an EC2 aws_instance

May 28, 2021
Categories: Virtualization

You can bake a startup script directly into the creation of your EC2 instance when using Terraform. Although complex post-configuration should be left to tools such as Ansible, essential bootstrap type commands or custom routes for instances in private subnets are reasons why you might need to use this hook. Below is an example of … Terraform: invoking a startup script for an EC2 aws_instance

Docker: determining container responsible for largest overlay directories

April 8, 2021
Categories: Virtualization

Whether you are running a docker daemon on a development host or a GKE worker node using Docker as the container engine, it is important to understand the amount of disk storage being utilized by the containers. If you navigate into the ‘/var/lib/docker/overlay2’ directory, you will see cryptic hashed ids representing the containers layers instead … Docker: determining container responsible for largest overlay directories

Terraform: Using non-authoritative resources to avoid IAM membership dependency web

February 5, 2021
Categories: Virtualization

One of the most challenging aspects of using Terraform is dealing with external changes and sprawl of dependent objects that may originate outside your control. Terraform wants to be a system of record and have everything documented in its state as resource/data, however keeping your state in sync when other groups may be doing automation … Terraform: Using non-authoritative resources to avoid IAM membership dependency web

GCP: Using gcloud to create and configure a service account

January 30, 2021
Categories: Virtualization

If you need to bootstrap a GCP project’s infrastructure, one of the first things you will want is a service account. The creation of the service account, creating its key, and then assigning binding roles can all be done from the GCP console but for scripting purposes can also be done using the gcloud utility. … GCP: Using gcloud to create and configure a service account

KVM: creating and reverting libvirt external snapshots

January 10, 2021
Categories: Linux, Virtualization

Update July 2021: I have seen errors with external snapshots of volumes on versions of QEMU/KVM/libvirt from Ubuntu 20 Focal. Adding note on using internal snapshot on volume backed by qcow2. Internal snapshots created on QEMU copy-on-write (qcow2) disks are the most commonly used snapshot when using libvirt. It is easy to see why; … KVM: creating and reverting libvirt external snapshots

KVM: Testing cloud-init locally using KVM for a CentOS cloud image

March 14, 2020
Categories: Linux, Virtualization

The ability to quickly stand up a guest OS with cloud-init is most often associated with deployment of virtual machines in an IaaS like EC2 or Azure. But cloud-init is not just for remote cloud providers, and using cloud-init for local images that can be quickly deployed in KVM works great for local development and … KVM: Testing cloud-init locally using KVM for a CentOS cloud image

KVM: Testing cloud-init locally using KVM for an Ubuntu cloud image

February 23, 2020
Categories: Linux, Virtualization

KVM: Terraform and cloud-init to create local KVM resources

February 22, 2020
Categories: Linux, Virtualization

Terraform is a popular tool for provisioning infrastructure on cloud provider such as EC2 and Azure, but there is also a provider written for local KVM libvirt resources. Using the libvirt provider, we can use standard Terraform constructs to create local VMs, networks, and disks.

KVM: Creating a guest VM on a network in routed mode

June 5, 2019
Categories: Linux, Virtualization

In a previous article I showed how to use a bridged network to give a VM access to the same network as the host, and then followed that up with creating a guest in a NAT network for network segmentation. In this article I will show how to create a network in Routed mode. This … KVM: Creating a guest VM on a network in routed mode

KVM: Creating a guest VM on a NAT network

May 26, 2019
Categories: Linux, Virtualization

In a previous article I showed how to use a bridged network to give a KVM Guest access to the same network as the Host. Now we will go over NAT networks which allow the KVM Guest to reach outside the network, but hosts from the outside cannot reach the guest directly. This opens up … KVM: Creating a guest VM on a NAT network

KVM: Creating a bridged network with NetPlan on Ubuntu 18.04 bionic

April 1, 2019
Categories: Linux, Virtualization

UPDATE September 2022: New article for bridged networks written for Ubuntu 22.04 In order to expose KVM virtual machines on the same network as your Host, you need to enable bridged networking. In this article, I’ll show how to implement KVM bridged networking on Ubuntu 18.04 bionic using Netplan. This bridged network will expose the … KVM: Creating a bridged network with NetPlan on Ubuntu 18.04 bionic

Python: JSONPath to extract vCenter information using govc

March 10, 2019
Categories: Python, Virtualization

In a previous post, I described how to install and use the ‘govc‘ tool – which is a command line tool for working with VMware vCenter. When you do a listing on a virtual machine “govc ls -l -json <vmpath>”, you are returned back a very detailed JSON data structure that contains all the information … Python: JSONPath to extract vCenter information using govc

VMware: Using the govc CLI to automate vCenter commands

March 9, 2019
Categories: Linux, Virtualization

The vSphere web GUI is a nice visual tool, but if you need to retrieve vCenter information in bulk or perform mass operations across VMs, then a command line tool such as govc in invaluable. govc is written in Go, which means it has support on Linux as well as most other platforms.

KVM: virt-manager to connect to a remote console using qemu+ssh

February 16, 2019
Categories: Linux, Virtualization

If you are running KVM on a console-only server, you still have the option to use the graphical virt-manager. You just need to specify the method of communication (ssh, tls, tcp, etc). In this article I will show how to use virt-manager from an Ubuntu client desktop to a server running KVM and libvirtd, with … KVM: virt-manager to connect to a remote console using qemu+ssh

CloudFoundry: The lifecycle of a simple BOSH release

January 12, 2019
Categories: Virtualization

BOSH is a project that unifies release, deployment, and lifecycle management of cloud based software. Software to be deployed via BOSH is called a release, and in this article I will use a very simple release to illustrate how to create, deploy, version, and revert these releases.

AWS: Bash helper functions for common AWS CLI calls

January 12, 2019
Categories: Scripting, Virtualization

The AWS CLI provides a comprehensive command set, but if you want advanced functionality such as checking if an object already exists before you create it, or polling an instance until it is a certain state then you will need a slightly higher level of abstraction. My aws-common-funcs.sh script provides a set of Bash functions … AWS: Bash helper functions for common AWS CLI calls

CloudFoundry: Installing a BOSH Director on AWS

January 12, 2019
Categories: Containers, Virtualization

BOSH is a project that unifies release, deployment, and lifecycle management of cloud based software. It also serves as the underlying infrastructure for deploying the CloudFoundry PaaS. In this article, I will lead you through deploying the BOSH Director to Amazon EC2 which is the first step in deploying both CloudFoundry CFAR as well as CFCR … CloudFoundry: Installing a BOSH Director on AWS

KVM: Deploy the VMware vCenter 6.7 appliance using the CLI installer

November 6, 2018
Categories: Linux, Virtualization

Update Nov 2021: I have written a newer article that deploys vCenter 7.0. If you have just virtualized the VMware ESXi 6.7 server on top of KVM, the next step will be to install vCenter 6.7 for its centralized control and additional feature set and management capabilities. In my last article we took KVM running … KVM: Deploy the VMware vCenter 6.7 appliance using the CLI installer

KVM: Using dnsmasq for libvirt DNS resolution

October 22, 2018
Categories: Linux, Virtualization

On Linux host servers, libvirt uses a separate instance of dnsmasq for each virtual network. So if you want full name and reverse lookup for KVM guests on the default 192.168.122.0 network, you can configure this particular dnsmasq instance using virsh. The ability to customize the libvirt instances are limited, however, so if you need … KVM: Using dnsmasq for libvirt DNS resolution

KVM: Deploying a nested version of VMware ESXi 6.7 inside KVM

September 19, 2018
Categories: Linux, Virtualization

Update Nov 2021: I have written a newer article that deploys ESXi 7.0U1. If you are running KVM on an Ubuntu server, you already have an excellent Type 1 virtualization engine. Luckily, if you need to test something specific to VMware you can always run ESXi 6.7 nested inside a KVM virtual machine. In this … KVM: Deploying a nested version of VMware ESXi 6.7 inside KVM

KVM: Cloning and fixing Ubuntu host using KVM

September 17, 2018
Categories: Linux, Virtualization

Usually the purpose of cloning a KVM guest OS is to have it use the same binaries and configuration as the parent, but allow it to run as an independent entity. When first cloned, the whole guest OS is cloned including host name, network settings, etc. and for those reasons you cannot run the parent … KVM: Cloning and fixing Ubuntu host using KVM