wiki:Old/VirtualizationImplementationNotes

System Virtualization

vconsole runs Xen. Instances of Xen are configured in /etc/xen/*.cfg and their raw images in /var/img. The Xen host (dom0) is configured in /etc/xen/*.sxp and by our custom bridge creation scripts in /etc/xen/scripts/*-orbit. We have plenty of disk, so we don't bother using LVM2 to share images yet — each domU has its own full image. Most of the instructions we needed are at https://help.ubuntu.com/community/XenOnUbuntuBinaryInstall. Don't use the Xen packages Edgy/universe, run 6.06 like a sensible Ubuntu admin, and install the xensource binary tarball over it using the given script. Seriously.

We use specific MAC addresses, and manage them with DHCP as if they were otherwise physical devices. See Internal/VMAC.

Network Virtualization

Here are the VLANs for our virtualized grid.

vlan name usual IP range notes
31 vgrid1-co 10.71 command console
32 vgrid1-da 10.81 data collection
33 vgrid2-co 10.72 command console
34 vgrid2-da 10.82 data collection
35 vgrid3-co 10.73 command console
36 vgrid3-da 10.83 data collection
37 vgrid4-co 10.74 command console
38 vgrid4-da 10.84 data collection

Here are the IP addresses of the virtual consoles as they interface to their vgrid VLAN.

vgrid side name dmz side
10.71.0.10 vconsole1 10.50.71.10
10.72.0.10 vconsole2 10.50.72.10
10.73.0.10 vconsole3 10.50.73.10
10.74.0.10 vconsole4 10.50.74.10

Each virtual console runs on a single system, which has 10.10.0.11 on the regular grid and 10.50.10.11 on the dmz.

10.7n.0.40 is repository2 (frisbee, pxe, and home). 10.7n.0.1 is the PIX interface. The current PIX can only be homed on 25 VLANS, so among these it is only homed on VLAN 31.

Prototype Topology

Absent a sophisticated "tetris-capable" Resource Manager (RM), we simply divide the physical grid into four equal quadrants, each becoming a virtual grid. This "prototype topology" is specifically as follows.

Physical Nodes vgrid vx = vy =
(1,1) to (10,10) vgrid1 x y
(11,1) to (20,10) vgrid2 x - 10 y
(1,11) to (10,20) vgrid3 x y - 10
(11,11) to (20,20) vgrid4 x - 10 y - 10

Each virtual grid addresses nodes from (1,1) to (10,10). It is important to note that all the mechanisms for implementing four vgrids can be extended to support larger quantities of simultaneous experiments without additional hardware. The simple 'vx = x - 10' style of mapping virtual to physical nodes can be modified to support arbitrary non-overlapping mappings without significant redesign of the system.

Last modified 12 years ago Last modified on Sep 15, 2008, 6:18:22 PM
Note: See TracWiki for help on using the wiki.