Terraform: provisioning AWS servers in both public and private subnets

It is relatively straightforward to create an AWS public subnet where the compute instances have access to the public internet via the default internet gateway.

But once you start building private subnets behind it, you must start considering security groups, routing, and the NAT gateways required to reach public services.

In this article, I will use Terraform to provision a public subnet 172.16.1.0/24, and a private subnet 172.16.1.0/24.  The Linux compute instance placed at 172.16.1.10 will be publicly accessible via an external IP address.  The Linux instance placed at 172.16.2.129 will not be accessible publicly, and must use a NAT gateway to reach the public internet.

Prerequisite: Install Terraform

See my previous article on installing Terraform.

Prerequisite: AWS Service Account credentials

In order to perform operations with Terraform against AWS, you will need a set of credentials at ~/.aws/credentials that look like below.

[default]
aws_access_key_id = xxxxxxxxxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxx

In a previous article I describe how to login to the AWS console and manually create a user named ‘awsuser’.   You can follow these instructions and manually create this file OR you can follow the entire article and install the aws CLI.  Running ‘aws configure’ will ask for the key and secret and create this file for you.

But it’s important to note that the Terraform AWS provider does not require the aws CLI tool.

Provision AWS infrastructure

This Terraform code is part of a larger github project, but we are just using the AWS infrastructure building.

# packages required
sudo apt-get install make git -y

# pull project
git clone https://github.com/fabianlee/wireguard-test-gcp-aws-roaming.git
cd wireguard-test-gcp-aws-roaming

# create ssh keypair for login to compute instances
make create-keypair
ls -l ansible_rsa*

WARNING: AWS will bill you for this provisioning! 

# build infrastructure in AWS using terraform
cd aws-infra
make

Terraform will display its progress, and at the end you should see something similar to below.

...
Apply complete! Resources: 19 added, 0 changed, 0 destroyed.

Outputs:

aws-ubuntu-priv-web-private_ip = "172.16.2.129"
aws-ubuntu-pub-wg_private_ip = "172.16.1.10"
aws-ubuntu-pub-wg_public_ip = "18.116.34.220"

This shows the public and private IP address of:

  • ‘aws-ubuntu-pub-wg_*’ – host in public subnet
  • ‘aws-ubuntu-priv-web_*’ – host in private subnet

Test access

First test access to the public host.  In my example output, the public IP address is ‘18.116.34.220’.  Replace with your relevant value.

# test netcat to ssh port 22
nc -vz 18.116.34.220 22

# test remote ssh to public instance, then exit back
ssh -i ../ansible_rsa ubuntu@18.116.34.220
hostname
exit

Now we want to test access to the host (172.16.2.129) in the private subnet, but the only way to reach it is through the host on the public subnet.  So, we first jump into the public host, and from there jump to the private host.

# copy ssh private auth key to public jumphost
scp -i ../ansible_rsa ../ansible_rsa ubuntu@18.116.34.220:/home/ubuntu/.

# ssh to public host first
ssh -i ../ansible_rsa ubuntu@18.116.34.220

# test netcat to private host
nc -vz 172.16.2.129 22

# make sure private key has limited permissions
cd /home/ubuntu
chmod 600 ./ansible_rsa

# ssh to private host
ssh -i ./ansible_rsa 172.16.2.129
hostname
exit

# remove private key from public host and exit one last time
rm /home/ubuntu/ansible_rsa
exit

There are other ways to reach this private host, for example the ssh commands takes a ‘-J’ flag that allows you to specify an intermediary.

ssh -J <user>@<bastion>:22 <targetuser>@<targethost> -vvv

But this requires creating an ~/.ssh/config so ssh knows which target host matches which bastion key as described in a previous article.  But this also saves you from copying the private key to the bastion, which is highly desirable from a security perspective.

Destroy

When done, remove all infrastructure:

make destroy

 

REFERENCES

fabianlee github, diagram

shellhacks.com, StrictHostKeyChecking=no