Ansible uses ssh to configure its target host inventory, but for on-premise datacenters as well as hyperscalers like EC2/GCP/Azure, the target hosts are often purposely located in deeper private subnets that cannot be reached from the Ansible orchestrator host.
One solution is to enable a bastion/jumpbox host that serves as the forwarding host. It sits as an intermediary that can forward the traffic coming from the Ansible orchestrator and pass it along to the target inventory host.
In this article I will show how to define the ‘ansible_ssh_common_args’ variable so Ansible can reach these target hosts.
Define Bastion for host
Each ansible target host you want to reach via a bastion host will need an ‘ansible_ssh_common_args’ defined as a variable. It usually makes sense to define this at the group level so multiple private hosts can leverage the same bastion.
As a concrete example, imagine we have a a public Bastion host (my-public-bastion=34.74.180.98) that can pass traffic back to instances in a private VPC subnet (my-host-behind-bastion=10.0.100.0/24). Ansible uses the private ssh key ‘./ansible_rsa’ and ‘./ansible_bastion_rsa’ as credentials for the ‘ubuntu’ user on their respective hosts.
If you were using an ansible_inventory.ini file, you would want to have the ‘ansible_ssh_common_args’ key defined as a variable applied to the group containing the private hosts.
[jumpboxes_public] my-public-bastion ansible_host=34.74.180.98 [jumpboxes_private] my-host-behind-bastion ansible_host=10.0.100.3 [jumpboxes_private:vars] ansible_ssh_common_args="-o ProxyCommand=\"ssh -q ubuntu@34.74.180.98 -o IdentityFile=./ansible_bastion_rsa -o Port=22 -W %h:%p\"" [all:vars] ansible_ssh_private_key_file=./ansible_rsa ansible_user=ubuntu
If you were instead using Ansible group variables defined in a file, then you would want to create ‘group_vars/jumpboxes_private’ and set the value like below:
ansible_ssh_common_args: "-o ProxyCommand=\"ssh -q ubuntu@34.74.180.98 -o IdentityFile=./ansible_bastion_rsa -o Port=22 -W %h:%p\""
You can then test access to the target host with a simple Ansible ping.
ansible -m ping my-host-behind-bastion
Troubleshooting ssh via bastion
If the Ansible ping above fails, then you will need to troubleshoot. The first step would be to ssh manually to the Bastion host. And from there try a manual ssh to the target host.
ssh ubuntu@34.74.180.98 -i ./ansible_bastion_rsa # from inside the bastion, make sure you can ssh to the private host # you will need to temporarily copy the key for troubleshooting ssh ubuntu@10.0.100.3 -i ./ansible_rsa
If that works, then be sure to check whether IPv4 forwarding is enabled on the Bastion host.
# check value sudo sysctl -a | grep 'net.ipv4.ip_forward ' # enable forwarding sudo sysctl -w net.ipv4.ip_forward=1
If that is enabled, then drop back to the Ansible host and try using ssh’s jumphost flag (-J) to go through the Bastion to the target host.
# ssh -J <user>@<bastion>:22 <targetuser>@<targethost> -vvv ssh -J ubuntu@34.74.180.98:22 ubuntu@10.0.100.3 -vvv
However, you first need to create a “~/.ssh/config” file that has a listing of the hosts and their matching private keys.
# bastion host entry Host 34.74.180.98 IdentityFile /home/fabian/ansible_bastion_rsa StrictHostKeyChecking no # private host entry with wildcard Host 10.0.100.* IdentityFile /home/fabian/ansible_rsa StrictHostKeyChecking no
ssh will automatically use private keys found at ~/.ssh/id_rsa, but you will usually have multiple keys in use for different bastion and so making this explicit is more desirable.
Note: this ~/.ssh/config file is not used by Ansible, this is only used by ssh -J as a way of troubleshooting bastion/jumpbox at the ssh level. If it does not work via ssh, then it won’t work at the Ansible level either.
REFERENCES