| 3 | | = Setting up the host for the NetFPGA = |
| 4 | | Here we provide the links and directions followed to set up a working NetFPGA host. The directions can be found on the following site: http://netfpga.org/netfpgawiki/index.php/Guide |
| 5 | | |
| 6 | | == Installing CentOS == |
| 7 | | CentOS is the recommended OS for use with NetFPGA. The most current version is 5.3, which is not listed as a version of CentOS that works with NetFPGA, but was used anyways. |
| 8 | | |
| 9 | | === CentOS 5.3 === |
| 10 | | The latest version is CentOS 5.3. |
| 11 | | |
| 12 | | For the installation, the net install iso was chosen because of its small size. This can be found on any mirror site. |
| 13 | | * CentOS mirrors : http://isoredirect.centos.org/centos/5/os/i386/ |
| 14 | | * Network installation of CentOS : http://www.tonybhimani.com/2007/12/20/centos-51-network-install-instructions/ |
| 15 | | |
| 16 | | === CD install of CentOS 5.2 === |
| 17 | | The .iso images can be found here: http://vault.centos.org/5.2/isos/i386/ |
| 18 | | You only need CD's 1 through 4. k3b on Ubuntu was used to burn the images since the images burned with Windows seemed to become corrupt. It seems that you need to manually configure the network interface for eth0. |
| 19 | | |
| 20 | | * Network interface configurations: http://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-networkscripts-interfaces.html |
| 21 | | * VLANs on CentOS: http://www.tummy.com/journals/entries/jafo_20061130_063931 |
| 22 | | |
| 23 | | |
| 24 | | Before installing the netFPGA base package, make sure to update the kernel, since the dependencies for NetFPGA software require a newer version of the kernel than the one for 5.3. To install the basic drivers, install teh yum repository and GPG key, followed by the netFPGA base package. |
| 25 | | {{{ |
| 26 | | rpm -Uhv http://netfpga.org/yum/el5/RPMS/noarch/netfpga-repo-1-1_CentOS5.noarch.rpm |
| 27 | | yum install netfpga-base |
| 28 | | /usr/local/NF2/lib/scripts/user_account_setup/user_account_setup.pl |
| 29 | | }}} |
| 30 | | The directions then go over installing CAD tools, but since we are just trying to make the FPGA into an !OPenFlow switch, we skip over that section of the guide. cd into the NetFPGA directory, then make, make install, and reboot. |
| 31 | | {{{ |
| 32 | | make |
| 33 | | #a bunch of output showing status |
| 34 | | make install |
| 35 | | #more outputs |
| 36 | | reboot |
| 37 | | }}} |
| 38 | | after the machine has rebooted, check that drivers are loaded. The PID should show up: |
| 39 | | {{{ |
| 40 | | [root@localhost ~]# /sbin/lsmod | grep nf2 |
| 41 | | nf2 22540 0 |
| 42 | | }}} |
| 43 | | Finally, verify the interfaces with `/sbin/ifconfig -a` at the terminal. This is the output for our setup: |
| 44 | | {{{ |
| 45 | | eth0 Link encap:Ethernet HWaddr 00:1C:C0:B4:EC:D4 |
| 46 | | inet addr:192.168.202.102 Bcast:192.168.203.255 Mask:255.255.252.0 |
| 47 | | inet6 addr: fe80::21c:c0ff:feb4:ecd4/64 Scope:Link |
| 48 | | UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 |
| 49 | | RX packets:825 errors:0 dropped:0 overruns:0 frame:0 |
| 50 | | TX packets:212 errors:0 dropped:0 overruns:0 carrier:0 |
| 51 | | collisions:0 txqueuelen:1000 |
| 52 | | RX bytes:76080 (74.2 KiB) TX bytes:21489 (20.9 KiB) |
| 53 | | Interrupt:50 Base address:0xa000 |
| 54 | | |
| 55 | | lo Link encap:Local Loopback |
| 56 | | inet addr:127.0.0.1 Mask:255.0.0.0 |
| 57 | | inet6 addr: ::1/128 Scope:Host |
| 58 | | UP LOOPBACK RUNNING MTU:16436 Metric:1 |
| 59 | | RX packets:1162 errors:0 dropped:0 overruns:0 frame:0 |
| 60 | | TX packets:1162 errors:0 dropped:0 overruns:0 carrier:0 |
| 61 | | collisions:0 txqueuelen:0 |
| 62 | | RX bytes:2004680 (1.9 MiB) TX bytes:2004680 (1.9 MiB) |
| 63 | | |
| 64 | | nf2c0 Link encap:Ethernet HWaddr 00:4E:46:32:43:00 |
| 65 | | BROADCAST MULTICAST MTU:1986 Metric:1 |
| 66 | | RX packets:0 errors:0 dropped:0 overruns:0 frame:0 |
| 67 | | TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 |
| 68 | | collisions:0 txqueuelen:1000 |
| 69 | | RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) |
| 70 | | Interrupt:58 |
| 71 | | |
| 72 | | nf2c1 Link encap:Ethernet HWaddr 00:4E:46:32:43:01 |
| 73 | | BROADCAST MULTICAST MTU:1986 Metric:1 |
| 74 | | RX packets:0 errors:0 dropped:0 overruns:0 frame:0 |
| 75 | | TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 |
| 76 | | collisions:0 txqueuelen:1000 |
| 77 | | RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) |
| 78 | | Interrupt:58 |
| 79 | | |
| 80 | | nf2c2 Link encap:Ethernet HWaddr 00:4E:46:32:43:02 |
| 81 | | BROADCAST MULTICAST MTU:1986 Metric:1 |
| 82 | | RX packets:0 errors:0 dropped:0 overruns:0 frame:0 |
| 83 | | TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 |
| 84 | | collisions:0 txqueuelen:1000 |
| 85 | | RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) |
| 86 | | Interrupt:58 |
| 87 | | |
| 88 | | nf2c3 Link encap:Ethernet HWaddr 00:4E:46:32:43:03 |
| 89 | | BROADCAST MULTICAST MTU:1986 Metric:1 |
| 90 | | RX packets:0 errors:0 dropped:0 overruns:0 frame:0 |
| 91 | | TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 |
| 92 | | collisions:0 txqueuelen:1000 |
| 93 | | RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) |
| 94 | | Interrupt:58 |
| 95 | | |
| 96 | | sit0 Link encap:IPv6-in-IPv4 |
| 97 | | NOARP MTU:1480 Metric:1 |
| 98 | | RX packets:0 errors:0 dropped:0 overruns:0 frame:0 |
| 99 | | TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 |
| 100 | | collisions:0 txqueuelen:0 |
| 101 | | RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) |
| 102 | | }}} |
| 103 | | |
| 104 | | == NetFPGA on Ubuntu (11/7) == |
| 105 | | |
| 106 | | Ultimately, the second NetFPGA host was built using Ubuntu 8.04 (kernel 2.6.24-16). To make things neat, user 'netfpga' was created for this process. Make sure the user is added to the sudoers list. |
| 107 | | |
| 108 | | === installing NetFPGA drivers === |
| 109 | | |
| 110 | | First off, you need an account at netfpga.org to download the .tar files needed to install the netfpga drivers. The files needed are netfpga_base_beta_x_x.tar.gz and netfpga_lib.tar.gz. With an account, they can be downloaded from here: http://netfpga.org/beta/distributions/ |
| 111 | | |
| 112 | | Untar and the files with `tar -xvf <filename>`. They will untar into one directory called ''NF2''. Do not make and install yet - At this point, follow the directions on the [http://www.netfpga.org/netfpgawiki/index.php/Ubuntu_Compatibility Ubuntu compatibility] page. They ask you to make a few changes on the files located in NF2. A few things to note regarding the directions: |
| 113 | | |
| 114 | | * to edit /boot/grub/menu.lst as per the instructions, use the directions for modifying grub.conf for CentOS, found here: http://www.netfpga.org/netfpgawiki/index.php/Install_Software_1.0#CentOS_Installation_Instructions |
| 115 | | |
| 116 | | * Environmental variables can be added by the command |
| 117 | | {{{ |
| 118 | | /usr/local/NF2/lib/scripts/user_account_setup/user_account_setup.pl |
| 119 | | }}} |
| 120 | | as per the [http://netfpga.org/netfpgawiki/index.php/Guide#Verify_the_software_and_hardware Guide], however you might need to edit /usr/local/NF2 in user_account_setup.pl to specify the exact path to your NF2 directory: |
| 121 | | {{{ |
| 122 | | else |
| 123 | | { |
| 124 | | print "Copying the NF2 directory to your user account\n"; |
| 125 | | `cp -r /home/netfpga/test/NF2 ~`; |
| 126 | | } |
| 127 | | |
| 128 | | print "Adding the NetFPGA Enviornment Variables to your .bashrc\n"; |
| 129 | | `cat /home/netfpga/test/NF2/bashrc_addon >> ~/.bashrc\n`; |
| 130 | | }}} |
| 131 | | |
| 132 | | Then you can make and install the drivers. Note, the selftest can't be run in Ubuntu. |
| 133 | | |
| 134 | | === installing !OpenFlow on NetFPGA === |
| 135 | | |
| 136 | | General directions are found here: http://netfpga.org/netfpgawiki/index.php/OpenFlow-NetFPGA-090 |
| 137 | | |
| 138 | | In order to install !OpenFlow on the FPGA you need both the netFPGA !OpenFlow switch driver and the actual !OpenFlow software. |
| 139 | | * netFPGA-OpenFLow-switch package : http://netfpga.org/beta/distributions/netfpga_openflow_switch.0_9_0-1.tar.gz (you need an account to access it) |
| 140 | | * !OpenFlow : use git (directions below) |
| 141 | | |
| 142 | | Don't bother with the ./install.sh stuff under the '' Usage '' section of the directions found at netfpga.org, since they are custom-fitted for CentOS. Instead, the directions found at [http://openflowswitch.org/wk/index.php?title=CentOS_Install openflowswitch.org] should be referenced. The directions need to be modified slightly for Ubuntu: |
| 143 | | |
| 144 | | 1. ''' Install Dependencies ''' |
| 145 | | |
| 146 | | The following are needed to install everything properly (you should be able to `apt-get install` these): |
| 147 | | * git-core (recommended) |
| 148 | | * curl - dependency for Git that does not get installed properly due to a known bug in some versions of Ubuntu |
| 149 | | * autoconf |
| 150 | | * automake |
| 151 | | * libtool |
| 152 | | * gcc |
| 153 | | * pkg-config |
| 154 | | |
| 155 | | The stuff needed to build kernel modules: |
| 156 | | * linux-kernel-devel |
| 157 | | * kernel-package |
| 158 | | * linux-libc-dev |
| 159 | | |
| 160 | | |
| 161 | | 2. ''' Install ''' [[BR]] |
| 162 | | |
| 163 | | The commands, as per the CentOS install page at openflow.org: |
| 164 | | {{{ |
| 165 | | git clone git://openflowswitch.org/openflow |
| 166 | | cd openflow |
| 167 | | ./boot.sh |
| 168 | | }}} |
| 169 | | Reboot the machine, then build the kernel modules: |
| 170 | | {{{ |
| 171 | | ./configure --with-l26=/lib/modules/`uname -r`/build --enable-hw-tables=nf2 |
| 172 | | make |
| 173 | | sudo make install |
| 174 | | }}} |
| 175 | | |
| 176 | | 3. ''' Check that all is well/ the kernel modules exist: ''' |
| 177 | | |
| 178 | | the command |
| 179 | | {{{ |
| 180 | | ls <path-to-your-openflow-directory>/datapath/linux-2.6 | grep ofdatapath |
| 181 | | }}} |
| 182 | | should bring up the following 2 things: |
| 183 | | * ofdatapath.ko |
| 184 | | * ofdatapath_netfpga.ko |
| 185 | | |
| 186 | | |
| 187 | | == Regression testing == |
| 188 | | Connecting the SATA feedback loop seems to prevent the test from loading correctly, throwing the following error: |
| 189 | | {{{ |
| 190 | | [root@localhost ~]# /root/NF2/lib/C/download/nf2_download ~/NF2/bitfiles/selftest.bit |
| 191 | | Found net device: nf2c0 |
| 192 | | Bit file built from: nf2_top_par.ncd |
| 193 | | Part: 2vp50ff1152 |
| 194 | | Date: 2008/ 7/21 |
| 195 | | Time: 17: 0:39 |
| 196 | | After resetting Programming interface, expected status to be 1 (FIFO empty). |
| 197 | | However status & 0x102 is 0x102 |
| 198 | | |
| 199 | | Fatal Error, exiting... |
| 200 | | }}} |
| 201 | | |
| 202 | | In addition, the Ethernet cables used for the cross-connection should be crossover, not straight through. Once the SATA cable was removed, and the test run with the command |
| 203 | | {{{ |
| 204 | | ~/NF2/projects/selftest/sw/selftest -n |
| 205 | | }}} |
| 206 | | the proper output was returned. |
| 207 | | |
| 208 | | == Installing the !OpenFlow implementation (!OpenFlow-NetFPGA-090) == |
| 209 | | After successfully installing the drivers, you can install the !OpenFlow switch with the following command: |
| 210 | | {{{ |
| 211 | | [root@localhost ~]# yum install netfpga-openflow_switch |
| 212 | | }}} |
| 213 | | |
| 214 | | = Setting up the NetFPGA host = |
| 215 | | The goal of this project is to set up two NetFPGA hosts as !OpenFlow switches capable of tunneling. The hosts we are going to be using are the NetFPGA pre-built quad-core rack PCs ([http://www.accenttechnologyinc.com/product_info.php?products_id=5 found here]). Although the PC's come with CentOS pre-installed with the proper drivers needed for the NetFPGAs, we want to use Ubuntu. This section will document the steps needed to build a host running Ubuntu 9.10 Server Edition. |
| 216 | | |
| 217 | | Note: We saved the HDD w/ CentOS that came with the PC. |
| 218 | | |
| 219 | | === References === |
| 220 | | Following pages will be used to set the hosts up. |
| 221 | | |
| 222 | | * NetFPGA Installation Guide : http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/Guide |
| 223 | | * NetFPGA Ubuntu compatibility page : http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/UbuntuCompatibility |
| 224 | | * !OpenFlow tunneling : http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/TunnelingOpenFlowNetFPGA090 |
| 225 | | |
| 226 | | === Host info === |
| 227 | | |
| 228 | | * Distro used for PC: Ubuntu 9.10 Server Edition 32bit |
| 229 | | * IPaddr: 192.168.203.187 (this is dynamic so it might change) |
| 230 | | * user: netfpga |
| 231 | | |
| 232 | | == Setup Instructions == |
| 233 | | === 1. Installing Ubuntu === |
| 234 | | 1. Change boot priority of CD drive in BIOS - default will not let you boot into the install CD |
| 235 | | |
| 236 | | 2. Follow most of the defaults, except for the following: |
| 237 | | * Disk partitioning - choose regular automatic partitioning |
| 238 | | * When prompted with the option to install additional packages, install OpenSSH-server |
| 239 | | |
| 240 | | 3. Update packages |
| 241 | | |
| 242 | | 4. Install dependencies for Ubuntu compatibility, as per the [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/UbuntuCompatibility compatibility page]. The packages are: |
| 243 | | |
| 244 | | == 2. Installing NetFPGA Drivers == |
| 245 | | Steps described by the NetFPGA tunneling page: |
| 246 | | 1. Install base package |
| 247 | | 2. Install !OpenFlow components |
| 248 | | |
| 249 | | === 1. Installing base package === |
| 250 | | |
| 251 | | 1. Download the tarballs from http://netfpga.org/beta/distributions/ [[BR]] |
| 252 | | For this you need a !FosWiki account (register [http://www.netfpga.org/foswiki/bin/view/System/UserRegistration?skin= here]). Download the newest netfpga_base_beta package (currently 2.0.0). Untar: |
| 253 | | |
| 254 | | tar -xvf netfpga_base_beta_2_0_0.tar.gz |
| 255 | | |
| 256 | | Untarring the package will create the directory NF2. |
| 257 | | |
| 258 | | 2. Edit files/ symlinks as per compatibility page [[BR]] |
| 259 | | |
| 260 | | Create links: |
| 261 | | {{{ |
| 262 | | ln /usr/bin/lspci /sbin/lspci |
| 263 | | ln /usr/bin/setpci /sbin/setpci |
| 264 | | ln -s /usr/lib/libpcap.so.0.8 /usr/lib/libpcap.so.0.8.3 |
| 265 | | }}} |
| 266 | | |
| 267 | | test_loopback_drop/run has been renamed run.pl in the most recent package. In this file you just need to change 'usleep 250' to 'sleep 0.00025'. |
| 268 | | |
| 269 | | NF2/lib/C/kernel/Makefile is already in working condition - you don't need to modify it. |
| 270 | | |
| 271 | | *Stopped at this point: The IRQF_SHARED flag is deprecated in kernels greater than 2.6.24. Instead, the SA_SHIRQ flag is used in include/linux/interrupt.h . in Ubuntu compatibility page** |
| 272 | | |
| 273 | | 3. Allocate more kernel memory [[BR]] |
| 274 | | The NetFPGA requires 256M memory allocated. This can be done through /boot/grub/menu.lst on older kernels. For the newer versions of Ubuntu (e.g. 9.10), which uses Grub 2, you need to edit /boot/grub/grub.cfg from |
| 275 | | {{{ |
| 276 | | GRUB_CMDLINE_LINUX_DEFAULT="quiet splash |
| 277 | | }}} |
| 278 | | to |
| 279 | | {{{ |
| 280 | | GRUB_CMDLINE_LINUX_DEFAULT="vmalloc=256M |
| 281 | | }}} |
| 282 | | |
| 283 | | 4. Edit and run user_account_setup.pl [[BR]] |
| 284 | | The script sets environmental variables. It is located under /NF2/lib/scripts/user_account_setup/ . Running it as is will throw an error saying |
| 285 | | |
| 286 | | cat: /usr/local/NF2/bashrc_addon: No such file or directory |
| 287 | | |
| 288 | | The easiest thing to do is to edit the script to contain the exact path to your NF2 directory and bashrc_addon: |
| 289 | | {{{ |
| 290 | | else |
| 291 | | { |
| 292 | | print "Copying the NF2 directory to your user account\n"; |
| 293 | | `cp -r /home/netfpga/NF2 ~`; |
| 294 | | } |
| 295 | | |
| 296 | | print "Adding the NetFPGA Enviornment Variables to your .bashrc\n"; |
| 297 | | `cat /home/netfpga/NF2/bashrc_addon >> ~/.bashrc\n`; |
| 298 | | }}} |
| 299 | | Then it will run correctly (here NF2 is already in the home directory) |
| 300 | | {{{ |
| 301 | | $ ./user_account_setup.pl NF2 directory already exists copy has been canceled |
| 302 | | Adding the NetFPGA Enviornment Variables to your .bashrc |
| 303 | | }}} |
| 304 | | |
| 305 | | |
| 306 | | trying to use the 9.10 compatible stuff, found in http://www.netfpga.org/foswiki/bin/view/NetFPGA/OneGig/RealTimeSwitch |
| 307 | | |
| 308 | | ---- |
| 309 | | ---- |
| 310 | | = Installing Ubuntu, ~~take 3~~ take 4 = |
| 311 | | After installing 9.10 server (AMD64) and 9.10 server 32 bit, we realized that there may be a "few" complications coming from the changes that were made in the more recent versions of Ubuntu (e.g. Grub 2, discarding "antiquated" libraries, ect), so things were rolled back to installing Ubuntu 9.04 server 32 bit. This should simplify *many* things. Hopefully. |
| 312 | | |
| 313 | | == 1. Installation of the OS == |
| 314 | | 1. Go to the BIOS settings, make CD drive highest boot priority |
| 315 | | |
| 316 | | 2. Follow the default choices given for intallation, save the following few: |
| 317 | | |
| 318 | | * no setup of LVM for disk partitioning |
| 319 | | * additional packages - install OpenSSH server |
| 320 | | * hostname: netfpga1 |
| 321 | | * user: netfpga |
| 322 | | |
| 323 | | 3. When rebooting, go back to the BIOS settings to make sure the HDD is included in the boot priority list. 'x' adds excluded devices to the boot priority list. |
| 324 | | |
| 325 | | 4. Who's afraid of a little command line? If you are, y' shoulda stuck with desktop edition. |
| 326 | | |
| 327 | | === Host info === |
| 328 | | From past experience, the following info will come in handy when troubleshooting stuff... |
| 329 | | * uname -a shows: 2.6.28-11-server i686 GNU/Linux |
| 330 | | * IP address: 192.168.203.187 (DHCP - May change) |
| 331 | | * bootloader: grub |
| 332 | | |
| 333 | | == 2. Installing drivers (NetFPGA, building NF2.ko module) == #BuildingNF2 |
| 334 | | The host is named "Netfpga1" because it will be a NetFPGA host. So this is sort of important. |
| 335 | | |
| 336 | | * Ubuntu Compatibility |
| 337 | | * http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/Guide |
| 338 | | |
| 339 | | 0. Get a Foswiki account so you can download the packages you need at their [http://www.netfpga.org/foswiki/bin/view/System/UserRegistration?skin= registration page]. |
| 340 | | |
| 341 | | 1. Update packages, then install the following dependencies as per the [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/UbuntuCompatibility Comaptibility page]: |
| 342 | | * build-essential |
| 343 | | * linux-headers-`uname -r` |
| 344 | | * ncurses-dev |
| 345 | | * libnet1-dev |
| 346 | | * libxml-simple-perl |
| 347 | | * libio-interface-perl |
| 348 | | * liblist-moreutils-perl |
| 349 | | * liberror-perl |
| 350 | | * libnet-rawip-perl |
| 351 | | * sun-java6-jre |
| 352 | | * sun-java6-jdk |
| 353 | | * libpcap0.8 |
| 354 | | * libpcap0.8-dev |
| 355 | | |
| 356 | | 2. Download the base package from http://netfpga.org/beta/distributions/. The page will prompt you for your !FosWiki username and password. The newest is netfpga_base_beta_2_0_0. Untar: |
| 357 | | {{{ |
| 358 | | tar xvf netfpga_base_beta_2_0_0.tar.gz |
| 359 | | }}} |
| 360 | | This will create a directory called NF2 in your home directory. |
| 361 | | |
| 362 | | 3. create symlinks: |
| 363 | | {{{ |
| 364 | | ln /usr/bin/lspci /sbin/lspci |
| 365 | | ln /usr/bin/setpci /sbin/setpci |
| 366 | | }}} |
| 367 | | |
| 368 | | 4. Acquire libpcap.so.0.9.4 (find on bottom of this page), slap it into /usr/lib, and create symlinks, as such: |
| 369 | | {{{ |
| 370 | | ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0.9 |
| 371 | | ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0 |
| 372 | | ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so |
| 373 | | }}} |
| 374 | | |
| 375 | | |
| 376 | | 5. Edit files by following the compatibility page. Some of the stuff there is outdated, so keep an eye out for the following: |
| 377 | | |
| 378 | | * /test_loopback_drop/run is /test_loopback_drop/run.pl |
| 379 | | * edit "usleep 250" to "sleep 0.00025" in /test_loopback_drop/run.pl |
| 380 | | * Don't change anything on /NF2/lib/C/kernel/Makefile |
| 381 | | * Likewise, NF2/lib/C/download/nf2_download.c need not be edited. |
| 382 | | |
| 383 | | 6. Edit /boot/grub/menu.lst to allocate more kernel memory: |
| 384 | | {{{ |
| 385 | | title Ubuntu 9.04, kernel 2.6.28-16-generic |
| 386 | | uuid (BIG number here) |
| 387 | | kernel /vmlinuz-2.6.28-16-generic root=UUID=(BIG number here) ro quiet splash vmalloc=256M |
| 388 | | }}} |
| 389 | | |
| 390 | | 7. Edit /home/netfpga/NF2/projects/router_kit/sw/rkd.cc to include limits.h |
| 391 | | |
| 392 | | 8. Replace asm/semaphores with linux/semaphores in: |
| 393 | | * /home/netfpga/NF2/lib/C/kernel/nf2main.c |
| 394 | | * /home/netfpga/NF2/lib/C/kernel/nf2_control.c |
| 395 | | |
| 396 | | 9. `make` and `make install` in user's NF2 directory (you need to be root) |
| 397 | | |
| 398 | | == 3. Regression testing == |
| 399 | | The selftest will not work on Ubuntu, so we skip right to the Regression tests. |
| 400 | | |
| 401 | | 1. Reprogramming the CPCI |
| 402 | | |
| 403 | | In cpci_reprogram.pl line 216, change the offset from 5 to 4: |
| 404 | | {{{ |
| 405 | | # Search for the NetFPGA device ID |
| 406 | | if ($words[4] eq $NF2_device_id) <<---over here |
| 407 | | { |
| 408 | | push (@device_list, $words[0]); |
| 409 | | } |
| 410 | | }}} |
| 411 | | |
| 412 | | Then run cpci_reprogram.pl (run from user home directory): |
| 413 | | {{{ |
| 414 | | /home/netfpga/NF2/lib/scripts/cpci_reprogram/cpci_reprogram.pl --all |
| 415 | | }}} |
| 416 | | Add the same command to /etc/rc.local so this is done automatically each time the machine is rebooted. |
| 417 | | |
| 418 | | 2. Rename interfaces. |
| 419 | | |
| 420 | | Since you need eth1 and 2 for the regression test, you need to change eth0 to eth2. Change "eth0" to "eth2" in /etc/udev/rules.d/70-persistent-net.rules so it looks similar to this: |
| 421 | | {{{ |
| 422 | | SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:30:48:b1:9d:a9", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" |
| 423 | | |
| 424 | | # PCI device 0x8086:0x109a (e1000e) |
| 425 | | SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:30:48:b1:9d:a8", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2" |
| 426 | | }}} |
| 427 | | |
| 428 | | The changes will take effect after you reboot the machine. |
| 429 | | |
| 430 | | 3. Much is hard-coded for CentOS. Change /sbin/arp to /usr/sbin/arp in NF2/projects/router_kit/regress/test_03_arp_table/setup |
| 431 | | |
| 432 | | 4. running the regression test |
| 433 | | |
| 434 | | The following command can be used to run the tests, which will take about 45 -60 min (according to the Guide). |
| 435 | | {{{ |
| 436 | | ~/NF2/bin/nf2_regress_test.pl |
| 437 | | }}} |
| 438 | | |
| 439 | | Expect some things to fail; the tests even fail on the CentOS pre-installed. |
| 440 | | |
| 441 | | == 4. Installing !OpenFlow components == |
| 442 | | |
| 443 | | '''Prequisite packages''' |
| 444 | | * autoconf |
| 445 | | * libtool |
| 446 | | * pkg-config |
| 447 | | * linux-source (2.6.28) |
| 448 | | |
| 449 | | Following the instructions of [http://www.openflowswitch.org/foswiki/bin/view/OpenFlow/Deployment/HOWTO/LabSetup Labsetup] |
| 450 | | we cloned the 0.8.9 branch of the their git repository, I'll include the steps because their pages don't stay up very long. |
| 451 | | |
| 452 | | {{{ |
| 453 | | $ git clone git://openflowswitch.org/openflow.git |
| 454 | | $ cd openflow |
| 455 | | $ git checkout -b openflow_089_rev4 origin/release/0.8.9 |
| 456 | | }}} |
| 457 | | |
| 458 | | Assuming all the prerequisites are installed, the next step is to compile the source. We're trying to make two kernel objects, and a bit file for the netfpga. |
| 459 | | |
| 460 | | {{{ |
| 461 | | $ ./boot.sh |
| 462 | | $ ./configure --with-l26=/lib/modules/`uname -r`/ --enable-hw-tables=nf2 |
| 463 | | $ make |
| 464 | | }}} |
| 465 | | |
| 466 | | boot.sh will fail to run if the listed prerequisites are not installed (there may be more preqs based on what was installed prior, build-tools, kernel headers, etc...). |
| 467 | | |
| 468 | | The original configure line read "./configure --with-l26=--with-l26=/lib/modules/`uname -r`/build --enable-hw-tables=nf2", while the extra --with-l26 is an obvious error, there are others. The configure script stuffs the argument of --wtih-l26 into a $path variable, then looks for the subdirectories $path/build and $path/source. Specifying --with-l26 as ../`uname -r`/build causes configure to fail, since there is no source directory in the build directory. Instead what they wanted was the directory 1 level up. |
| 469 | | |
| 470 | | In our first attempt we just ran make and went on to the next step, however when we tried to insert the modules one of them failed. The problem was with the ofdatapath_netfpga.ko module. When we tried to insert it (with insmod) it would complain about a missing symbol. |
| 471 | | Dmesg would show an even every time you tried to insert the module that said |
| 472 | | |
| 473 | | {{{ |
| 474 | | WARNING: "nf2k_reg_read" |
| 475 | | WARNING: "nf2k_reg$ ./boot.sh |
| 476 | | }}} |
| 477 | | |
| 478 | | Consulting a reference from the Stanford [$ https://mailman.stanford.edu/pipermail/openflow-discuss/2009-January/000051.html mailing list] on netfpga, we determined that the Module.symvers file needed to be edited. We needed to copy the entries from the ~/NF2/lib/c/kernel/Modules.symvers into the ~/openflow/datapath/linux-2.6/Modules.symvers. The NF2 directory comes from two tar files: |
| 479 | | * netfpga_base_beta_2_0_0_0.tar.gz |
| 480 | | * netfpga_openflow_switch.0_8_9-1.tar.gz - This file was chosen to match the version of the Netfpga openflow source we're compling. |
| 481 | | Go to http://netfpga.org/beta/distributions/ in order to download these packages (you'd need your !FosWiki account). |
| 482 | | |
| 483 | | Making this change produces the bits files and modules that will be needed in the next section. |
| 484 | | |
| 485 | | After a sucessfull complie we needed to load the openflow bitfile onto the netfpga |
| 486 | | {{{ |
| 487 | | # cpci_reprogram.pl |
| 488 | | # nf2_download ~/openflow/datapath/hwtable_nf2/openflow_switch.bit |
| 489 | | }}} |
| 490 | | |
| 491 | | The actual paths may vary but the basic process is the same. This is fairly straight forward assuming the netfpga tools were already built. |
| 492 | | |
| 493 | | Once that was done we inserted the modules (required root privileges) |
| 494 | | {{{ |
| 495 | | # /sbin/insmod ./openflow/datapath/linux-2.6/ofdatapath.ko |
| 496 | | # /sbin/insmod ./openflow/datapath/linux-2.6/ofdatapath_netfpga.ko |
| 497 | | }}} |
| 498 | | |
| 499 | | At this point we're ready to try testing the netfpga as a open flow switch. I'm going to pause here and rearrange the networks that the specfic netfpgas belong to. |
| 500 | | |
| 501 | | The host has been renamed to of1 and was given an official place in the WINLAB network with IP addr. 192.168.200.86 |
| 502 | | |
| 503 | | Next, you create the OpenFlow datapaths. What they refer to as ./utilities can be found under your openflow directory. We didn't assign a MAC address to the interface like they did in the tutorial. |
| 504 | | {{{ |
| 505 | | # ./utilities/dpctl adddp nl:0 |
| 506 | | }}} |
| 507 | | |
| 508 | | Then, add the NetFPGA interfaces. Here we encounter another typo in the instructions - there is no nf2c4. |
| 509 | | {{{ |
| 510 | | # ./utilities/dpctl addif nl:0 nf2c0 |
| 511 | | # ./utilities/dpctl addif nl:0 nf2c1 |
| 512 | | # ./utilities/dpctl addif nl:0 nf2c2 |
| 513 | | # ./utilities/dpctl addif nl:0 nf2c3 |
| 514 | | }}} |
| 515 | | |
| 516 | | Then you make the !OpenFlow module listen to your controller. For this setup the controller is a PC running SNAC with the IP address 192.168.203.75. secchan can be found under your openflow directory. |
| 517 | | {{{ |
| 518 | | # ./secchan/ofprotocol nl:0 tcp:192.168.203.75:6633 |
| 519 | | }}} |
| 520 | | |
| 521 | | === Building the L2 Tunnel === |
| 522 | | |
| 523 | | ==== 2/15/10 ==== |
| 524 | | |
| 525 | | James guessed that maybe the tunneling code was rolled up into the 0.9.1 version of the NF2 driver. This was incorrect. |
| 526 | | |
| 527 | | Following the directions [#BuildingNF2 above], James rebuilt the nf2 drivers, and installed the updated module. We did not notice any changes in the open flow behavior. But the project directory for tunneling was missing. |
| 528 | | |
| 529 | | ==== 2/16/10 ==== |
| 530 | | |
| 531 | | The code for the netfpga to do tunneling is present. It includes a different bitfile and some diffrent startup scritps. They claim to be running off version 0.8.9 of somethig (I'm guessing the open flow code) |
| 532 | | |
| 533 | | The main script that needs to be modified is the of_setup_to_site1_and_site2.sh script. I've modified the script to reflect our setup: |
| 534 | | |
| 535 | | {{{ |
| 536 | | ### Variable Setting |
| 537 | | SRCIP1=192.168.200.87 #For nf2c1 Change accordingly. |
| 538 | | #SRCIP2=192.168.100.50 #For nf2c2 Change accordingly. |
| 539 | | SRCMAC1=00:4e:46:32:43:01 |
| 540 | | #SRCMAC2=00:4e:46:32:43:02 |
| 541 | | DSTIP1=10.50.0.62;# IP address of the tunneling target site 1 |
| 542 | | #DSTIP2=12.13.14.15; # IP address of the tunneling target site 2 |
| 543 | | # Your local Gateway MAC address |
| 544 | | DSTMAC1=00:4E:46:32:43:01 |
| 545 | | #DSTMAC2=00:d0:05:5d:24:02 |
| 546 | | |
| 547 | | TUNIF1=nf2c1 # The tunneling target site 1 |
| 548 | | #TUNIF2=nf2c2 # The tunneling target site 2 |
| 549 | | }}} |
| 550 | | |
| 551 | | Examining the rest of the script seems to require a setreg tool, that isn't installed as of yet. Ubuntu claims is comes form the package mono-devel. |
| 552 | | |
| 553 | | === Centos Revisited === |
| 554 | | |
| 555 | | ==== 2/22/10 ==== |
| 556 | | |
| 557 | | The ubuntu experiment is failing terribly. The NF2 modules are throwing errors left and right about dma timeouts, and buffer overruns. |
| 558 | | James is attempting to build the openflow switch on the Centos Refrence installations that came with the the Tyan boxes from the reference vendor. |
| 559 | | |
| 560 | | The vendor already ran |
| 561 | | {{{ |
| 562 | | yum install netfpga-base |
| 563 | | }}} |
| 564 | | |
| 565 | | From the factory the netfpga packages that were installed are as follows: |
| 566 | | {{{ |
| 567 | | netfpga-base.i386 2.0.0-beta installed |
| 568 | | netfpga-kernel.i386 2.0.0-beta installed |
| 569 | | netfpga-repo.noarch 1-1_CentOS5 installed |
| 570 | | netfpga-utils.i386 2.0.0-beta installed |
| 571 | | netfpga_lib.i386 1.1-2 installed |
| 572 | | }}} |
| 573 | | |
| 574 | | James added one additional package to their netfpga installation |
| 575 | | {{{ |
| 576 | | netfpga-openflow_switch.i386 0_9_0-1 installed |
| 577 | | }}} |
| 578 | | |
| 579 | | This mostly populates the /root/NF2/projects/openflow subdir with stuff from the yum repository. |
| 580 | | |
| 581 | | |
| 582 | | In order to build the openflow modules from the 0.8.9~2 [http://openflowswitch.org/downloads/openflow-0.8.9~2.tar.gz tarball], we need a bunch of tools not installed in the refrence CentOS system, this step is copied directly form [http://openflowswitch.org/wk/index.php?title=CentOS_Install CentOS instructions]: |
| 583 | | {{{ |
| 584 | | sudo yum install git automake pkgconfig libtool gcc |
| 585 | | }}} |
| 586 | | |
| 587 | | Apparently you also need to compile automake from sources: |
| 588 | | {{{ |
| 589 | | wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.63.tar.gz |
| 590 | | tar xvzf autoconf-2.63.tar.gz |
| 591 | | cd autoconf-2.63 |
| 592 | | ./configure --prefix=/usr |
| 593 | | make |
| 594 | | sudo make install |
| 595 | | }}} |
| 596 | | |
| 597 | | = Ubuntu 10.04 = |
| 598 | | Current Ubuntu NetFPGAs run Ubuntu version 8.04 and !OpenFlow ver. 1.0. Here we try to move to newer distributions w/ !OpenFlow Ver. 1.0. The hardware we use here is the NetFPGA cube, another version of the NetFPGA pre-built solution. A user named 'test' with root privs was added for this purpose. |
| | 5 | Setup of the NetFPGA using earlier distributions can be found in the Diaries. |
