Changes between Version 9 and Version 10 of Internal/VMHostSetup

Timestamp:

May 4, 2012, 10:16:12 PM (12 years ago)

Author:

ssugrim

Comment:

—

Internal/VMHostSetup

@@ -98 +98 @@
     }}}
     Install the package kvm-pxe as referenced [http://www.moe.co.uk/2011/01/07/pci_add_option_rom-failed-to-find-romfile-pxe-rtl8139-bin/ here].
+
+----
+=== Adjustments for multiple !Bridges/Nics ===
+
+Our current setup on external3 requires a separate bridge(vswitch) for each interfaces, because some VM's need to be in 10.50(DMZ) and other need 172.16(network), while others still need both. ovs-vsctl will happily build multiple bridges however, a few tweaks needed to be make to the host inorder of the system to work properly. For some reason, the original /etc/network/interfaces config breaks if you use more than one bridge, even if you only dhcp over just 1. After some expirmentation this is the working network interfaces files:
+{{{
+# This file describes the network interfaces available on your system
+# and how to activate them. For more information, see interfaces(5).
+
+# The loopback network interface
+auto lo eth1 br1
+iface lo inet loopback
+iface eth1 inet manual
+        up ifconfig eth1 up
+        down ifconfig eth1 down
+iface br1 inet dhcp
+        up ifconfig eth0 up
+        up ifconfig br0 up
+        down ifconfig eth0 down
+        down ifconfig br0 down
+}}}
+There are other working examples, but this one seems functional enough. It brings all the interfaces up and then attempts dhcp out of br1. The running ovs-vsctl should look like:
+{{{
+d03e1847-34f4-4129-8821-63fff3403553
+    Bridge "br1"
+        Port "eth1"
+            Interface "eth1"
+        Port "tap3"
+            Interface "tap3"
+        Port "tap1"
+            Interface "tap1"
+        Port "br1"
+            Interface "br1"
+                type: internal
+    Bridge "br0"
+        Port "eth0"
+            Interface "eth0"
+        Port "tap2"
+            Interface "tap2"
+        Port "br0"
+            Interface "br0"
+                type: internal
+        Port "tap0"
+            Interface "tap0"
+    ovs_version: "1.2.2.10448"
+}}}
+Note that the taps are distributed according to which vlan/subnet they are supposed to belong to. I also switch from e1000 emulation to virtio, because its supposed to preform better. The big discovery with this setup was in how to invoke the kvm. Originally I replicated the -net... flag and started the vm. This works but it bonds the two interfaces together. Packets from one virtual interface show up on both bridges and go out of both cards. The reason this happens is a missing vlan keyword in the -net flag. This vlan keyword has nothing to do with vlan tagging for packets egress from the virtual interfaces. It's purely about the internal representation of the interfaces and the internal switching that qemu/kvm does. Specifying different vlan flags for the different interfaces and their respective taps, fixed the bonding problem. Packets were not only present on the proper bridge. I also had to modify all the scripting infrastructure to reflect this change. Instead of a single ovs-ifup/down script, there are now two(ovs-ifup-br0 and ovs-ifup-br1), one for each bridge. There are also now two scripts for starting vms depending on whether you want 1 or two interfaces.
+{{{
+Single interface command string
+kvm -daemonize -vnc :$2 -m 2048 -smp 2 -net nic,model=virtio,macaddr=00:11:22:EE:EE:E$2 -net tap,script=/etc/ovs-ifup-br$3,downscript=/etc/ovs-ifdown-br$3 -drive file=$1
+}}}
+note the 3rd argument specifies the brdige to join
+{{{
+2 interface command string
+kvm -daemonize -vnc :$2 -m 2048 -smp 2 -net nic,model=virtio,vlan=0,macaddr=00:11:22:EE:EE:E$2 -net tap,vlan=0,script=/etc/ovs-ifup-br0,downscript=/etc/ovs-ifdown-br0 -net nic,model=virtio,vlan=1,macaddr=00:11:22:FF:FF:F$2 -net tap,vlan=1,script=/etc/ovs-ifup-br1,downscript=/etc/ovs-ifdown-br1 -drive file=$1
+}}}
+Note the added vlan tags. The ovs-ifup-brX scripts are the same as the original, except for the swith=... keyword. 
 
 ----