[wiki:Internal/OpenFlow/ return to OpenFlow index] = Setting up the host for the NetFPGA = Here we provide the links and directions followed to set up a working NetFPGA host. The directions can be found on the following site: http://netfpga.org/netfpgawiki/index.php/Guide == Installing CentOS == CentOS is the recommended OS for use with NetFPGA. The most current version is 5.3, which is not listed as a version of CentOS that works with NetFPGA, but was used anyways. === CentOS 5.3 === The latest version is CentOS 5.3. For the installation, the net install iso was chosen because of its small size. This can be found on any mirror site. * CentOS mirrors : http://isoredirect.centos.org/centos/5/os/i386/ * Network installation of CentOS : http://www.tonybhimani.com/2007/12/20/centos-51-network-install-instructions/ === CD install of CentOS 5.2 === The .iso images can be found here: http://vault.centos.org/5.2/isos/i386/ You only need CD's 1 through 4. k3b on Ubuntu was used to burn the images since the images burned with Windows seemed to become corrupt. It seems that you need to manually configure the network interface for eth0. * Network interface configurations: http://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-networkscripts-interfaces.html * VLANs on CentOS: http://www.tummy.com/journals/entries/jafo_20061130_063931 Before installing the netFPGA base package, make sure to update the kernel, since the dependencies for NetFPGA software require a newer version of the kernel than the one for 5.3. To install the basic drivers, install teh yum repository and GPG key, followed by the netFPGA base package. {{{ rpm -Uhv http://netfpga.org/yum/el5/RPMS/noarch/netfpga-repo-1-1_CentOS5.noarch.rpm yum install netfpga-base /usr/local/NF2/lib/scripts/user_account_setup/user_account_setup.pl }}} The directions then go over installing CAD tools, but since we are just trying to make the FPGA into an !OPenFlow switch, we skip over that section of the guide. cd into the NetFPGA directory, then make, make install, and reboot. {{{ make #a bunch of output showing status make install #more outputs reboot }}} after the machine has rebooted, check that drivers are loaded. The PID should show up: {{{ [root@localhost ~]# /sbin/lsmod | grep nf2 nf2 22540 0 }}} Finally, verify the interfaces with `/sbin/ifconfig -a` at the terminal. This is the output for our setup: {{{ eth0 Link encap:Ethernet HWaddr 00:1C:C0:B4:EC:D4 inet addr:192.168.202.102 Bcast:192.168.203.255 Mask:255.255.252.0 inet6 addr: fe80::21c:c0ff:feb4:ecd4/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:825 errors:0 dropped:0 overruns:0 frame:0 TX packets:212 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:76080 (74.2 KiB) TX bytes:21489 (20.9 KiB) Interrupt:50 Base address:0xa000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:1162 errors:0 dropped:0 overruns:0 frame:0 TX packets:1162 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2004680 (1.9 MiB) TX bytes:2004680 (1.9 MiB) nf2c0 Link encap:Ethernet HWaddr 00:4E:46:32:43:00 BROADCAST MULTICAST MTU:1986 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:58 nf2c1 Link encap:Ethernet HWaddr 00:4E:46:32:43:01 BROADCAST MULTICAST MTU:1986 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:58 nf2c2 Link encap:Ethernet HWaddr 00:4E:46:32:43:02 BROADCAST MULTICAST MTU:1986 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:58 nf2c3 Link encap:Ethernet HWaddr 00:4E:46:32:43:03 BROADCAST MULTICAST MTU:1986 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) Interrupt:58 sit0 Link encap:IPv6-in-IPv4 NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) }}} == NetFPGA on Ubuntu (11/7) == Ultimately, the second NetFPGA host was built using Ubuntu 8.04 (kernel 2.6.24-16). To make things neat, user 'netfpga' was created for this process. Make sure the user is added to the sudoers list. === installing NetFPGA drivers === First off, you need an account at netfpga.org to download the .tar files needed to install the netfpga drivers. The files needed are netfpga_base_beta_x_x.tar.gz and netfpga_lib.tar.gz. With an account, they can be downloaded from here: http://netfpga.org/beta/distributions/ Untar and the files with `tar -xvf `. They will untar into one directory called ''NF2''. Do not make and install yet - At this point, follow the directions on the [http://www.netfpga.org/netfpgawiki/index.php/Ubuntu_Compatibility Ubuntu compatibility] page. They ask you to make a few changes on the files located in NF2. A few things to note regarding the directions: * to edit /boot/grub/menu.lst as per the instructions, use the directions for modifying grub.conf for CentOS, found here: http://www.netfpga.org/netfpgawiki/index.php/Install_Software_1.0#CentOS_Installation_Instructions * Environmental variables can be added by the command {{{ /usr/local/NF2/lib/scripts/user_account_setup/user_account_setup.pl }}} as per the [http://netfpga.org/netfpgawiki/index.php/Guide#Verify_the_software_and_hardware Guide], however you might need to edit /usr/local/NF2 in user_account_setup.pl to specify the exact path to your NF2 directory: {{{ else { print "Copying the NF2 directory to your user account\n"; `cp -r /home/netfpga/test/NF2 ~`; } print "Adding the NetFPGA Enviornment Variables to your .bashrc\n"; `cat /home/netfpga/test/NF2/bashrc_addon >> ~/.bashrc\n`; }}} Then you can make and install the drivers. Note, the selftest can't be run in Ubuntu. === installing !OpenFlow on NetFPGA === General directions are found here: http://netfpga.org/netfpgawiki/index.php/OpenFlow-NetFPGA-090 In order to install !OpenFlow on the FPGA you need both the netFPGA !OpenFlow switch driver and the actual !OpenFlow software. * netFPGA-OpenFLow-switch package : http://netfpga.org/beta/distributions/netfpga_openflow_switch.0_9_0-1.tar.gz (you need an account to access it) * !OpenFlow : use git (directions below) Don't bother with the ./install.sh stuff under the '' Usage '' section of the directions found at netfpga.org, since they are custom-fitted for CentOS. Instead, the directions found at [http://openflowswitch.org/wk/index.php?title=CentOS_Install openflowswitch.org] should be referenced. The directions need to be modified slightly for Ubuntu: 1. ''' Install Dependencies ''' The following are needed to install everything properly (you should be able to `apt-get install` these): * git-core (recommended) * curl - dependency for Git that does not get installed properly due to a known bug in some versions of Ubuntu * autoconf * automake * libtool * gcc * pkg-config The stuff needed to build kernel modules: * linux-kernel-devel * kernel-package * linux-libc-dev 2. ''' Install ''' [[BR]] The commands, as per the CentOS install page at openflow.org: {{{ git clone git://openflowswitch.org/openflow cd openflow ./boot.sh }}} Reboot the machine, then build the kernel modules: {{{ ./configure --with-l26=/lib/modules/`uname -r`/build --enable-hw-tables=nf2 make sudo make install }}} 3. ''' Check that all is well/ the kernel modules exist: ''' the command {{{ ls /datapath/linux-2.6 | grep ofdatapath }}} should bring up the following 2 things: * ofdatapath.ko * ofdatapath_netfpga.ko == Regression testing == Connecting the SATA feedback loop seems to prevent the test from loading correctly, throwing the following error: {{{ [root@localhost ~]# /root/NF2/lib/C/download/nf2_download ~/NF2/bitfiles/selftest.bit Found net device: nf2c0 Bit file built from: nf2_top_par.ncd Part: 2vp50ff1152 Date: 2008/ 7/21 Time: 17: 0:39 After resetting Programming interface, expected status to be 1 (FIFO empty). However status & 0x102 is 0x102 Fatal Error, exiting... }}} In addition, the Ethernet cables used for the cross-connection should be crossover, not straight through. Once the SATA cable was removed, and the test run with the command {{{ ~/NF2/projects/selftest/sw/selftest -n }}} the proper output was returned. == Installing the !OpenFlow implementation (!OpenFlow-NetFPGA-090) == After successfully installing the drivers, you can install the !OpenFlow switch with the following command: {{{ [root@localhost ~]# yum install netfpga-openflow_switch }}} = Setting up the NetFPGA host = The goal of this project is to set up two NetFPGA hosts as !OpenFlow switches capable of tunneling. The hosts we are going to be using are the NetFPGA pre-built quad-core rack PCs ([http://www.accenttechnologyinc.com/product_info.php?products_id=5 found here]). Although the PC's come with CentOS pre-installed with the proper drivers needed for the NetFPGAs, we want to use Ubuntu. This section will document the steps needed to build a host running Ubuntu 9.10 Server Edition. Note: We saved the HDD w/ CentOS that came with the PC. === References === Following pages will be used to set the hosts up. * NetFPGA Installation Guide : http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/Guide * NetFPGA Ubuntu compatibility page : http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/UbuntuCompatibility * !OpenFlow tunneling : http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/TunnelingOpenFlowNetFPGA090 === Host info === * Distro used for PC: Ubuntu 9.10 Server Edition 32bit * IPaddr: 192.168.203.187 (this is dynamic so it might change) * user: netfpga == Setup Instructions == === 1. Installing Ubuntu === 1. Change boot priority of CD drive in BIOS - default will not let you boot into the install CD 2. Follow most of the defaults, except for the following: * Disk partitioning - choose regular automatic partitioning * When prompted with the option to install additional packages, install OpenSSH-server 3. Update packages 4. Install dependencies for Ubuntu compatibility, as per the [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/UbuntuCompatibility compatibility page]. The packages are: == 2. Installing NetFPGA Drivers == Steps described by the NetFPGA tunneling page: 1. Install base package 2. Install !OpenFlow components === 1. Installing base package === 1. Download the tarballs from http://netfpga.org/beta/distributions/ [[BR]] For this you need a !FosWiki account (register [http://www.netfpga.org/foswiki/bin/view/System/UserRegistration?skin= here]). Download the newest netfpga_base_beta package (currently 2.0.0). Untar: tar -xvf netfpga_base_beta_2_0_0.tar.gz Untarring the package will create the directory NF2. 2. Edit files/ symlinks as per compatibility page [[BR]] Create links: {{{ ln /usr/bin/lspci /sbin/lspci ln /usr/bin/setpci /sbin/setpci ln -s /usr/lib/libpcap.so.0.8 /usr/lib/libpcap.so.0.8.3 }}} test_loopback_drop/run has been renamed run.pl in the most recent package. In this file you just need to change 'usleep 250' to 'sleep 0.00025'. NF2/lib/C/kernel/Makefile is already in working condition - you don't need to modify it. *Stopped at this point: The IRQF_SHARED flag is deprecated in kernels greater than 2.6.24. Instead, the SA_SHIRQ flag is used in include/linux/interrupt.h . in Ubuntu compatibility page** 3. Allocate more kernel memory [[BR]] The NetFPGA requires 256M memory allocated. This can be done through /boot/grub/menu.lst on older kernels. For the newer versions of Ubuntu (e.g. 9.10), which uses Grub 2, you need to edit /boot/grub/grub.cfg from {{{ GRUB_CMDLINE_LINUX_DEFAULT="quiet splash }}} to {{{ GRUB_CMDLINE_LINUX_DEFAULT="vmalloc=256M }}} 4. Edit and run user_account_setup.pl [[BR]] The script sets environmental variables. It is located under /NF2/lib/scripts/user_account_setup/ . Running it as is will throw an error saying cat: /usr/local/NF2/bashrc_addon: No such file or directory The easiest thing to do is to edit the script to contain the exact path to your NF2 directory and bashrc_addon: {{{ else { print "Copying the NF2 directory to your user account\n"; `cp -r /home/netfpga/NF2 ~`; } print "Adding the NetFPGA Enviornment Variables to your .bashrc\n"; `cat /home/netfpga/NF2/bashrc_addon >> ~/.bashrc\n`; }}} Then it will run correctly (here NF2 is already in the home directory) {{{ $ ./user_account_setup.pl NF2 directory already exists copy has been canceled Adding the NetFPGA Enviornment Variables to your .bashrc }}} trying to use the 9.10 compatible stuff, found in http://www.netfpga.org/foswiki/bin/view/NetFPGA/OneGig/RealTimeSwitch ---- ---- = Installing Ubuntu, ~~take 3~~ take 4 = After installing 9.10 server (AMD64) and 9.10 server 32 bit, we realized that there may be a "few" complications coming from the changes that were made in the more recent versions of Ubuntu (e.g. Grub 2, discarding "antiquated" libraries, ect), so things were rolled back to installing Ubuntu 9.04 server 32 bit. This should simplify *many* things. Hopefully. == 1. Installation of the OS == 1. Go to the BIOS settings, make CD drive highest boot priority 2. Follow the default choices given for intallation, save the following few: * no setup of LVM for disk partitioning * additional packages - install OpenSSH server * hostname: netfpga1 * user: netfpga 3. When rebooting, go back to the BIOS settings to make sure the HDD is included in the boot priority list. 'x' adds excluded devices to the boot priority list. 4. Who's afraid of a little command line? If you are, y' shoulda stuck with desktop edition. === Host info === From past experience, the following info will come in handy when troubleshooting stuff... * uname -a shows: 2.6.28-11-server i686 GNU/Linux * IP address: 192.168.203.187 (DHCP - May change) * bootloader: grub == 2. Installing drivers (NetFPGA) == The host is named "Netfpga1" because it will be a NetFPGA host. So this is sort of important. * Ubuntu Compatibility * http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/Guide 0. Get a Foswiki account so you can download the packages you need at their [http://www.netfpga.org/foswiki/bin/view/System/UserRegistration?skin= registration page]. 1. Update packages, then install the following dependencies as per the [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/UbuntuCompatibility Comaptibility page]: * build-essential * linux-headers-`uname -r` * ncurses-dev * libnet1-dev * libxml-simple-perl * libio-interface-perl * liblist-moreutils-perl * liberror-perl * libnet-perl-rawip * sun-java6-jre * sun-java6-jdk * libpcap0.8 * libpcap0.8-dev 2. Download the base package. The newest is netfpga_base_beta_2_0_0. Untar: {{{ tar xvf netfpga_base_beta_2_0_0.tar.gz }}} This will create a directory called NF2 in your home directory. 3. create symlinks: {{{ ln /usr/bin/lspci /sbin/lspci ln /usr/bin/setpci /sbin/setpci }}} 4. Acquire libpcap.so.0.9.4 (find on bottom of this page), slap it into /usr/lib, and create symlinks, as such: {{{ ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0.9 ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0 ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so }}} 5. Edit files by following the compatibility page. Some of the stuff there is outdated, so keep an eye out for the following: * /test_loopback_drop/run is /test_loopback_drop/run.pl * edit "usleep 250" to "sleep 0.00025" in /test_loopback_drop/run.pl * Don't change anything on /NF2/lib/C/kernel/Makefile * Likewise, NF2/lib/C/download/nf2_download.c need not be edited. 6. Edit /boot/grub/menu.lst to allocate more kernel memory: {{{ title Ubuntu 9.04, kernel 2.6.28-16-generic uuid (BIG number here) kernel /vmlinuz-2.6.28-16-generic root=UUID=(BIG number here) ro quiet splash vmalloc=256M }}} 7. Edit rkd.cc to include limits.h 8. Replace asm/semaphores with linux/semaphores 9. `make` and `make install` in user's NF2 directory (you need to be root) == 3. Regression testing == The selftest will not work on Ubuntu, so we skip right to the Regression tests. 1. Reprogramming the CPCI In cpci_reprogram.pl line 216, change the offset from 5 to 4: {{{ # Search for the NetFPGA device ID if ($words[4] eq $NF2_device_id) <<---over here { push (@device_list, $words[0]); } }}} Then run cpci_reprogram.pl (run from user home directory): {{{ /home/netfpga/NF2/lib/scripts/cpci_reprogram/cpci_reprogram.pl --all }}} Add the same command to /etc/rc.local so this is done automatically each time the machine is rebooted. 2. Rename interfaces. Since you need eth1 and 2 for the regression test, you need to change eth0 to eth2. Change "eth0" to "eth2" in /etc/udev/rules.d/70-persistent-net.rules so it looks similar to this: {{{ SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:30:48:b1:9d:a9", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" # PCI device 0x8086:0x109a (e1000e) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:30:48:b1:9d:a8", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2" }}} The changes will take effect after you reboot the machine. 3. Much is hard-coded for CentOS. Change /sbin/arp to /usr/sbin/arp in NF2/projects/router_kit/regress/test_03_arp_table/setup 4. running the regression test The following command can be used to run the tests, which will take about 45 -60 min (according to the Guide). {{{ ~/NF2/bin/nf2_regress_test.pl }}}