Terraform: creating an Ubuntu 20 Focal template and then guest VM in vCenter


Categories: Linux, Virtualization

In this article I will demonstrate how to create an Ubuntu 20 Focal template in vCenter.  Then use Terraform to create a vSphere VM based on this template.

The VM template creation is done by manually stepping through an installation using the minimal Ubuntu server ISO followed by a set of preparation steps.

Then Terraform is used to provision a VM with the customization parameters that tailor it to the environment.

Upload Ubuntu minimal iso to vCenter

Download the Ubuntu minimal mini.iso to your local disk.  I suggest you rename it to “mini-focal.iso” to make it clear which version you have.

Then login to vCenter, and select your main datastore, then the “Files” tab.  Press “Upload Files” and upload your “mini-focal.iso”.  A progress bar will display until complete.

Create VM based on Ubuntu minimal iso

Create a folder for templates in vCenter:

  • From vCenter, go to “Hosts and Clusters” and right-click on your vCenter cluster or esxi host
  • Go to the “VMs and Templates” tab on the left
  • Right-click your datacenter or cluster, right-click and select “New Folder” > New VM and Template folder, name=”templates”

Create a vCenter VM that will become a template:

  • From vCenter, go to the “VMs and Templates” tab on the left
  • Right-click your “templates” folder created above, select “New Virtual Machine”
  • Create a new virtual Machine, next
  • Virtual machine name=”ubuntu-focal”, the “templates” folder will already be selected, next
  • Select a compute resource, next
  • Select a storage location, next
  • Compatible with your version (e.g. ESXi 6.7 or later), next
  • Guest OS=Linux, version=Ubuntu 64 bit, next
  • Configure VM, cpu=1, mem=1Gb, Hard disk=60Gb, network=VM Network, DVD=datatstore ISO file, then select the ISO you uploaded earlier, next
  • Finish

Follow Ubuntu server installation process

Go to “VMs and Templates” tab on the left, select your VM “ubuntu-focal”, then “Edit Settings”.  The CD/DVD drive will not have the “Connect” checkbox checked, enable that checkbox and press OK.

Then power on the VM and “Launch Web Console” and you should see that you are entered into the standard wizard steps of installing Ubuntu 20 Focal.

This series of installation wizard screens is very standard so is documented in several locations throughout the web with screenshots [1,2,3].  Here are the descriptions of each screen and how to populate:

  • language=English
  • territory=US
  • detect keyboard=no
  • keyboard=english
  • keyboard layout=english
  • network
    • if DHCP, allow it to detect
    • If autoconfig fails then setup manually according to vCenter network selected
  • hostname=ubuntu-focal
  • domain name=home.lab
  • mirror=US
  • proxy=<none>
  • user account name=ubuntu
  • account name=ubuntu
  • password=ExamplePass@456
  • timezone correct=yes
  • Disk, Guided – use entire disk and setup LVM
    • select only partition
    • write changes and LVM, Yes
    • amount of volume group to use = allow full disk default, continue
    • write changes to disk, yes
  • No automatic updates
  • Software select – check three: Ubuntu Cloud Image (instance), OpenSSH server, and Basic Ubuntu server
  • Install Grub, yes
  • System clock set to UTC, yes
  • Installation complete, press continue

Then from vCenter:

  • Select the VM in vCenter and Power > Power Off
  • Select “Edit Settings” and uncheck the CD/DVD checkbox so it disconnects
  • Power > Power On
  • Launch Web Console

Post config steps to prepare for templating

Now we need to run some housekeeping and preparation steps to get this ready to be a VM template.

  • Login as ‘ubuntu’ with password ‘ExamplePass@456’ like we provided in the installation wizard
  • sudo apt update
  • sudo apt dist-upgrade -y
  • apt-install dnsutils traceroute -y
  • echo vm.swappiness=10 | sudo tee -a /etc/sysctl.conf
  • echo net.ipv6.conf.all.disable_ipv6=1 | sudo tee -a /etc/sysctl.conf
  • echo net.ipv6.conf.default.disable_ipv6=1 | sudo tee -a /etc/sysctl.conf
  • echo net.ipv6.conf.lo.disable_ipv6=1 | sudo tee -a /etc/sysctl.conf
  • sudo sysctl -p
  • sudo sed -i ‘s/preserve_hostname: false/preserve_hostname: true/g’ /etc/cloud/cloud.cfg
  • ls -l /etc/netplan
  • sudo rm /etc/netplan/*.yaml
  • history -c
  • sudo shutdown -h now

Instead of running all the commands above manually, I’ve also provided a script.  Do not run this from your host VM, run it only on the vSphere VM being prepared as a template!

wget https://raw.githubusercontent.com/fabianlee/tf-vsphere-singlevm-from-template/main/on_template_only/prepare_os_as_template.sh

/bin/bash ./prepare_os_as_template.sh

Convert to template

From vCenter, select the “ubuntu-focal” VM.  Right-click and select Template > Convert to Template.

At this point, you could manually create VMs from this template in the vCenter web GUI.  But you would want to create a VM customization specification (Menu > Policies and Profiles) so that you could provide the network values.

But since we want to use this from Terraform, we can have it do all the customizations of hostname, DNS, and networking.  So let’s move on to the next section.

Install Terraform

If you have not installed Terraform, follow my instructions here.

Pull Terraform project from github

Pull the necessary Terraform files from my github project

# get OS packages required
sudo apt install git make -y

# clone project
git clone https://github.com/fabianlee/tf-vsphere-singlevm-from-template.git

cd tf-vsphere-singlevm-from-template

# edit files per your environment
# customize vcenter connection, networks, jumphost name and network
vi terraform.20.tfvars

Create VM from template using Terraform

Now run terraform and create the VM within vSphere using the template created earlier.

# get plugins
terraform init

# create
terraform apply -var-file=terraform.20.tfvars

 

REFERENCES

Myles Gray, cloud-init for templating on vsphere

vrd83, creating an Ubuntu 20 cloud-init template and then terraform to create guest vm

kublr.com, prepare ubuntu template with cloud-init for vsphere

microsoft.com, building ubuntu 18 template from ISO

Navneet Verma, Ubuntu template with cloud-init and fixes to make it work with vSphere 7

sh0rez, deploying ubuntu cloud images to vsphere

govc command reference

vmware, inserting metadata and userdata into already uploaded ova

A different approach is using package to create a template image [1,2,3]

tadamhicks, ubuntu ova spec and vsphere

adamtheautomator, installing vmware tools using apt

linoproject, terraform the vsphere cloud-init

fabianlee.org, fixing and cloning an Ubuntu host

terraform, vsphere_virtual_machine resource

github terraform issue, good explanation by tmashos of chain for how metadata is passed from terraform-vsphere-cloud-init

blahcloud.com, advice at bottom on trouble spots and troubleshooting cloud-init

notch.org, using govc to set userdata(cloud_config) and metadata on a vm

NOTES

to export vm as ova

$ vm=ubuntu-focal

$ govc find -type p
/mydc1/host/esxi1.home.lab/Resources

$ govc vm.markasvm --pool /mydc1/host/esxi1.home.lab/Resources $vm

# export into directory
$ govc export.ovf -vm $vm ${vm}-ovf

# turn back into template
$ govc vm.markastemplate $vm

to attach userdata(cloud-config) and metadata to vm for cloud-init [1]

# userdata (cloud-config)
export CLOUD_CONFIG=$(gzip -c9 <cloud-config.yaml | base64)

# network metadata
export METADATA=$(sed 's~NETWORK_CONFIG~'"$(gzip -c9 <network.config.yaml | \
base64)"'~' <metadata.json | gzip -9 | base64)

# set on vm
govc vm.change -vm $vm -e guestinfo.metadata="${METADATA}"
govc vm.change -vm $vm -e guestinfo.metadata.encoding=gzip+base64
govc vm.change -vm $vm -e guestinfo.userdata="${CLOUD_CONFIG}"
govc vm.change -vm $vm" -e guestinfo.userdata.encoding=gzip+base64

installing the vmware remote console bundle

If you have issues with vmware web console and keyboard input, try the “VMware Remote Console” instead. The download link is available from https://esxi1.home.lab/ui.

sudo ./VMware-Remote-Console-12.0.1-18113358.x86_64.bin

for ova template import (could not get to work)

cd vsphere-ubuntu-focal-cloudimg

# get ovf spec from ova
govc import.spec ~/Downloads/focal-server-cloudimg-amd64.ova > focal-ova.json

# customize with variables
# make 'hostname' empty
# put ${password} in 'password' value
# put ${user-data} in 'user-data' value
vi focal-ova.json

# do replacement of values in focal-ova.json
# then base64 encode and put into cloud_init.cfg
make

# create folder for ova template
govc folder.create /mydc1/vm/templates

# import into vcenter
govc import.ova -folder=/mydc1/vm/templates -options=/tmp/focal-ova.json -name=focal-server-cloudimg ~/Downloads/focal-server-cloudimg-amd64.ova

# increase disk size
govc vm.disk.change -vm focal-server-cloudimg -disk.label "Hard disk 1" -size 60G
# mark as template
govc vm.markastemplate focal-server-cloudimg

# do update of metadat or userdata to template
export METADATA=$(gzip -c9 <metadata.yaml | { base64 -w0 2>/dev/null || base64; })
export USERDATA=$(gzip -c9 <userdata.yaml | { base64 -w0 2>/dev/null || base64; })

govc vm.change -vm test140 -e guestinfo.metadata="${METADATA}" -e guestinfo.metadata.encoding="gzip+base64"

govc vm.change -vm test140  -e guestinfo.userdata="${USERDATA}" -e guestinfo.userdata.encoding="gzip+base64"