In a previous article I showed how to use a bridged network to give a VM access to the same network as the host, and then followed that up with creating a guest in a NAT network for network segmentation.
In this article I will show how to create a network in Routed mode. This provides a segmented network much like a NAT with its own address space, but it does not masquerade packets going outside the network. Instead, it relies on the operator making routing changes in the network infrastructure to send/receive traffic. This typically involves adding a static route to the upstream router.
Routed mode networking means that KVM does less of the filtering/routing setup, and instead relies more on the existing network infrastructure which is more typical of an enterprise deployment where commercial routing/filtering/load-balancing solutions are employed.
Ensure IP forwarding is enabled
Before going forward, make sure IP forwarding is enabled on the host.
# this needs to be "1" cat /proc/sys/net/ipv4/ip_forward
If it is not enabled, then set it on the fly using:
sudo sysctl -w net.ipv4.ip_forward=1
And then enable it permanently by editing the “/etc/sysctl.conf” file with the following key:
net.ipv4.ip_forward=1
Create libvirt NAT network
First check the currently defined networks recognized by libvirt. Use the following command:
virsh net-list --all
To create a new Routed network, create an xml file “routed225.xml”. Use either the physical network or bridge name connected to the physical network for the “dev” attribute of the <forward> element.
cat <<EOF > routed225.xml
<network>
<name>routed225</name>
<forward mode='route' dev='br0'/>
<bridge name='virbr225' stp='on' delay='2'/>
<ip address='192.168.225.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.225.141' end='192.168.225.254'/>
<host name='myclone3' ip='192.168.225.143'/>
</dhcp>
</ip>
</network>
EOF
If you do not set the “dev” attribute to either the physical network device or bridge, then you will get inconsistent behavior because it may take shortcuts and forward traffic directly from one KVM network to another using internal interfaces. By using the physical interface or bridge connected to the physical interface, traffic will be redirected by the outermost network level on the host and ensure the iptables rules are applied correctly
Then create the network using these commands:
# create routed network (net-create is for transient) virsh net-define routed225.xml virsh net-start routed225 virsh net-autostart routed225 # NAT state should be active, autostart, and persistent virsh net-list --all
As described in the libvirt firwall documentation, rules are created in iptables to support this new network.
# virtual bridge virbr225 created ip a | grep 225 # iptables FORWARD rules for routed225 sudo iptables -L FORWARD -nv --line-number | grep virbr225 # There are no POSTROUTING rules needed, expect this to be empty sudo iptables -t nat -L -n -v --line-number | grep 225
Configure upstream router
The distinctive property of KVM routed networks is that you need to manually configure your network infrastructure to direct traffic to them.
This is very router specific so I won’t be able to provide step-by-step instructions for you, but assuming your KVM host is using the IP address 192.168.1.9 and your upstream router/gateway is at 192.168.1.1, then you’ll need to login as an administrator to your upstream router and create a static route that directs all traffic which looks like 192.168.225.0 (netmask 255.255.255.0) to your KVM host address at 192.168.1.9.
Once the traffic reaches your KVM host, it has iptables and routes that allow it to forward traffic to the correct virtual bridge, virbr225.
# rules on KVM host for forwarding traffic to virbr225 $ sudo iptables -L FORWARD -nv --line-number | grep virbr225 # routes on KVM host for traffic to virbr225 $ route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default _gateway 0.0.0.0 UG 0 0 0 br0 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0 192.168.225.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr225
Create VM using routed network
If you are creating guest OS from the console using virt-install, then simply replace the default network specifier with the name of the “routed225” network like below.
--network network=routed225
If using the virt-manager GUI, the network can be selected from the network source pulldown.
Once the guest VM is up and configured, you can run the command “ip a” and see that the new host now has an IP address randomly assigned from the DHCP pool defined in routed225.xml
This new guest VM should be able to reach out the public network, and also allow inbound connections. If you need restrictions on access, then you must do it on the upstream router, or insert a firewall appliance, or create net filters in the guest XML definition.
If you have multiple routed KVM networks, they will be isolated from each other and cannot intra-communicate. This is due to the fact that the KVM host will immediately route the traffic from one internal virtual bridge to another (instead of routing back to the upstream router) and the iptables rules for routed networks is not configured to allow this.
If you must have two KVM routed networks communicate with each other, then you would need to insert rules into the iptables FORWARD chain. For example, to allow the routed network on 192.168.225.0 (virbr225) and 192.168.226.0 (virbr226) to communicate, you can add the following rules.
# insert rules into top of chain sudo iptables -I FORWARD 1 -j ACCEPT -i virbr225 -o virbr226 -s 192.168.225.0/24 -d 192.168.226.0/24 sudo iptables -I FORWARD 2 -j ACCEPT -i virbr226 -o virbr225 -s 192.168.226.0/24 -d 192.168.225.0/24 sudo iptables -L -v -n --line-number | grep FORWARD -A20 # remove intra-communication sudo iptables -D FORWARD 2 sudo iptables -D FORWARD 1
Create VM, assigned specific IP from DHCP Pool
Notice the routed225.xml has a snippet that looks like:
<dhcp> <range start='192.168.225.141' end='192.168.225.254'/> <host name='myclone3' ip='192.168.225.143'/> </dhcp>
If you create a guest VM with the name “myclone3”, then instead of getting a random IP address from the DHCP pool, it will instead receive the one specified “192.168.225.143”. Using virt-install, the parameter used would be:
--name=myclone3
You can also specify by MAC address using the attribute “mac” in the <host> element, but then you must start up the guest first so the MAC can be randomly generated. So it is easier to just use the name.
DHCP Leases
To view the current DHCP leases on a network, use this command:
virsh net-dhcp-leases routed225
REFERENCES