[wiki:Internal/OpenFlow/ return to OpenFlow index] = NetFPGA !OpenFlow switch = According to the guides, !OpenVswitch provides a kenel implementation of the OF switch, whereas Stanford's reference switch is purely userspace. The former is faster, the latter more customizable. This is the latest set of instructions for installing NetFPGA drivers on Ubuntu (10.10), and making it work as an !OpenFlow switch. It is assumed that your working directory is /opt/. == I. NetFPGA setup == === 1.1 Download sources/dependencies === `cd` to /opt/ and fetch the following: * netfpga_full_2_2_0.tar.gz from: http://www.netfpga.org/releases/netfpga_full_2_2_0.tar.gz [[BR]] REF: http://netfpga.org/foswiki/NetFPGA/OneGig/Releases * netfpga_openflow_switch_1_0_0-4.tar.gz from: http://netfpga.org/beta/distributions/netfpga_openflow_switch_1_0_0-4.tar.gz [[BR]] REF: http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/OpenFlowNetFPGA100 * dependencies {{{ apt-get update apt-get install build-essential linux-headers-`uname -r` ncurses-dev libnet1-dev libxml-simple-perl libio-interface-perl liblist-moreutils-perl liberror-perl git-core libpcap0.8 libpcap0.8-dev }}} === 1.2 Compatibility fixes === The following changes have to be made in order to compile the drivers on Ubuntu. * Add the following symlinks: {{{ ln /usr/bin/lspci /sbin/lspci ln /usr/bin/setpci /sbin/setpci }}} * /opt/netfpga/lib/C/kernel/nf2util.h : add in the main #ifdef block: {{{ // Replace SA_SHIRQ with IRQF_SHARED on newer kernels #ifndef SA_SHIRQ #define SA_SHIRQ IRQF_SHARED #endif }}} * /etc/default/grub: add "vmalloc=512m" to GRUB_CMDLINE_LINUX_DEFAULT and do `update-grub` For the kernel: * If using kernel version 2.6.35-25, upgrade to 2.6.35 (`uname -ar` should show you which one you are running): {{{ apt-get install linux-headers-2.6.35-30-generic linux-image-2.6.35-30-generic }}} * Reboot. After reboot, check that the kernel flags are correct: {{{ dmesg | grep vmalloc [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.35-30-generic root=UUID=c90285a2-48de-4ab0-950f-241817516ad8 ro vmalloc=512m console=tty0 console=ttyS0,115200n8 [ 0.000000] vmalloc : 0xdf7fe000 - 0xff7fe000 ( 512 MB) }}} === 1.3 Compile drivers === * export environment variables: {{{ export NF_ROOT=/opt/netfpga #(wherever your netfpga base files are) source ${NF_ROOT}/bin/nf_profile }}} * from /opt/netfpga: `make`; `make install` After compilation, load kernel module with `modprobe nf2`. If all goes well, you should see the an entry if you check with `lsmod | grep nf2`. At this point, you can load bitfiles and such by first running `cpci_reprogram` and then using nf_download. You should see something like this: {{{ # cpci_reprogram.pl --all Loading the CPCI Reprogrammer on NetFPGA 0 Loading the CPCI on NetFPGA 0 CPCI on NetFPGA 0 has been successfully reprogrammed # nf_download /opt/netfpga/bitfiles/reference_switch.bit Found net device: nf2c0 Bit file built from: nf2_top_par.ncd;HW_TIMEOUT=FALSE Part: 2vp50ff1152 Date: 2011/ 1/20 Time: 6: 8:48 Error Registers: 0 Good, after resetting programming interface the FIFO is empty Download completed - 2377668 bytes. (expected 2377668). DONE went high - chip has been successfully programmed. CPCI Information ---------------- Version: 4 (rev 1) Device (Virtex) Information --------------------------- Project directory: reference_switch Project name: Reference Switch Project description: Reference Switch Device ID: 3 Version: 1.0.0 Built against CPCI version: 4 (rev 1) Virtex design compiled against active CPCI version }}} == II. !OpenFlow setup == === 2.1 dependencies === * packages: {{{ apt-get install autoconf libtool }}} * Use git to pull NetFPGA-specific !OpenFlow sources to /opt/: {{{ git clone git://openflowswitch.org/openflow.git cd openflow git checkout -b 1.0.0-netfpga origin/devel/tyabe/1.0.0-netfpga }}} === 2.2 build !OpenFlow === {{{ ./boot.sh cd ./hw-lib/nf2 wget http://openflowswitch.org/downloads/netfpga/openflow_switch.bit.100_3.tar.gz tar zxvf openflow_switch.bit.100_3.tar.gz ==> we need newer one from NF site:http://www.netfpga.org/releases/netfpga_openflow_switch_1_0_0-3.tar.gz cd ../../ ./configure --enable-hw-tables=nf2 make make install }}} After compilation, copy of_start.sh and of_stop.sh from /opt/netfpga/projects/openflow_switch/sw/. ---- See below for old instructions. ---- = Setting up NetFPGA hosts: Ubuntu 10.04 = Current Ubuntu NetFPGAs run Ubuntu version 8.04 and !OpenFlow ver. 1.0. Here we try to move to newer distributions w/ !OpenFlow Ver. 1.0. The hardware we use here is the NetFPGA cube, another version of the NetFPGA pre-built solution. As per the [http://www.openflowswitch.org/wk/index.php/CentOS_NetFPGA_Install !OpenFlow portion of the setup], a user named 'openflow' with root privs was added for this purpose. Setup of the NetFPGA using earlier distributions can be found in the Diaries. '''Linux distro:''' [[BR]] Ubuntu Server 10.04, w/ xfce4 WM, openssh-server [[BR]] kernel: 2.6.32-21-generic-pae '''Packages:''' [[BR]] NetFPGA base 2.1.0 [[BR]] Tunneling OpenFlow NetFPGA 1.0.0 === Installing Ubuntu. === 1. set boot priority of CD drive to 1st in BIOS 2. If desktop edition, boot as a live CD and install from there. Trying direct install just results in the system proceeding to live CD Mode (10.04). Server edition can be installed directly w/out issues. === I: Setup - NetFPGA drivers. === ==== 1. dependencies. ==== There is an updated dependency list: {{{ build-essential ncurses-dev libnet1-dev libxml-simple-perl libio-interface-perl liblist-moreutils-perl liberror-perl libnet-rawip-perl sun-java6-jre sun-java6-jdk libpcap0.8-dev }}} '''Installing java packages.''' In 10.04, sun-java6-jre, jdk are obsoleted and moved to the partner repository. This repository can be added w/ the following (ref: https://help.ubuntu.com/community/Repositories/CommandLine): 1. copy "deb !http://archive.canonical.com/ lucid partner" sans quotes to /etc/apt/sources.list 1. run apt-get update. 1. run apt-get as usual for the packages. ==== 2. Kernel memory allocation. ==== Ubuntu versions beyond 8.04 (assume anything w/ kernels beyond v 2.6.28) use GRUB 2. GRUB 2 gets rid of menu.lst, so kernel memory must be allocated via the following method (ref. http://ohioloco.ubuntuforums.org/showthread.php?p=8229620): Add "vmalloc=512m" to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub so it looks like this: {{{ GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vmalloc=512m" }}} Then run `update-grub` to regenerate /boot/grub/grub.cfg with the new parameter (you never want to modify grub.cfg directly). ==== 3. Symlinks. ==== Download libpcap.so.0.9.4, found below. Move it into /usr/lib. Run ubuntu_setup.sh (also attached below), or manually create the following links: {{{ ln /usr/bin/lspci /sbin/lspci ln /usr/bin/setpci /sbin/setpci ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0.9 ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0 ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so }}} ==== 4. Edit files. ==== Download Netfpga code: http://netfpga.org/foswiki/NetFPGA/OneGig/Releases ----- * edit "usleep 250" to "sleep 0.00025" in /test_loopback_drop/run.pl * edit /home/netfpga/NF2/projects/router_kit/sw/rkd.cc to include limits.h '''NOTE: this edit is no longer required for version 2.1.1 of the package. ''' ==== 5. Setting Variables. ==== After setting NF_ROOT to the netfpga directory, source `nf_profile`, located in ${NF_ROOT}/bin/ {{{ export NF_ROOT=/home/openflow/netfpga #(wherever your netfpga base files are) source ${NF_ROOT}/bin/nf_profile }}} ==== 6. Moment of truth. ==== `make` then `make install`. [[BR]] Then, check for nf2 module with `lsmod | grep nf2`: {{{ nf2 13425 0 }}} and for nf2cn, n=0:3 with `ifconfig -a`. {{{ nf2c0 Link encap:Ethernet HWaddr 00:4e:46:32:43:00 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 nf2c1 Link encap:Ethernet HWaddr 00:4e:46:32:43:01 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 nf2c2 Link encap:Ethernet HWaddr 00:4e:46:32:43:02 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 nf2c3 Link encap:Ethernet HWaddr 00:4e:46:32:43:03 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:16 }}} ==== 7. add the cpci reprogrammer to startup routine ==== Add the following to /etc/rc.local: {{{ /home/openflow/netfpga/lib/scripts/cpci_reprogram/cpci_reprogram.pl --all }}} The selftest ''does not'' work in Ubuntu. so don't try it. == II: Setup - !OpenFlow components. == The !OpenFlow wiki on CentOS NetFPGA setup were followed in conjunction with the netfpga site wiki for the Tunneling !OpenFlow NetFPGA. Stes are reiterated here since nothing is permanent. * [http://netfpga.org/foswiki/bin/view/NetFPGA/OneGig/TunnelingOpenFlowNetFPGA100 Tunneling !OpenFlow NetFPGA ] * !OpenFlow setup on [http://www.openflowswitch.org/wk/index.php/Ubuntu_Install Ubuntu] and [http://www.openflowswitch.org/wk/index.php/CentOS_NetFPGA_Install CentOS] ==== 1. dependencies. ==== {{{ autoconf libtool pkg-config linux-source git-core automake m4 }}} ==== 2. !OpenFlow package installation ==== This happens in the same directory as the netfpga install. From git repository (automatically pulls the newest version): {{{ git clone git://openflowswitch.org/openflow.git cd openflow ./boot.sh }}} To make NetFPGA compatible: {{{ ./configure --enable-hw-lib=nf2 }}} Remove avahi-daemon and disable IPv6: {{{ sudo apt-get remove avahi-daemon }}} in /etc/sysctl.conf add {{{ net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 }}} and in /etc/modprobe.d/blacklist.conf {{{ blacklist net-pf-10 blacklist ipv6 }}} ==== 3. NetFPGA tunnel package ==== Download and untar in `/home/openflow/`. Move `tunneling_openflow_switch.bit` and `reg_defines_tunneling_openflow_switch.h` to your_openflow_directory/hw-lib/nf2/. The files should be located under `bitfiles/` and `/projects/tunneling_openflow_switch/lib/C/` in the netfpga directory, respectively. Do a make and make install in the openflow directory. ==== 4. Regression test ==== The setup should be wired as such {{{ eth1 <----> nf2c0 eth2 <----> nf2c1 }}} The test can be run with the following command: {{{ nf_regress_test.pl --project tunneling_openflow_switch }}} As of now, getting this error: {{{ root@node1-2:~/netfpga/lib/scripts/cpci_reprogram# /home/akoshibe/netfpga/bin/n_regress_test.pl --project tunneling_openflow_switch NetFPGA environment: Root dir: /home/openflow/netfpga Project name: tunneling_openflow_switch Project dir: /home/openflow/netfpga/projects/tunneling_openflow_switch Work dir: /tmp/root Root directory is /home/openflow/netfpga Running tests on project 'tunneling_openflow_switch'... Running global setup... FAIL Output was: NetFPGA environment: Root dir: /home/openflow/netfpga Project name: tunneling_openflow_switch Project dir: /home/openflow/netfpga/projects/tunneling_openflow_switch Work dir: /tmp/root Found net device: nf2c0 Error: Virtex design compiled against a different CPCI version Active CPCI version : 4 (rev 1) Device built against: 3 (rev 4) Bit file built from: nf2_top_par.ncd Part: 2vp50ff1152 Date: 2010/ 5/ 4 Time: 15:49:32 Error Registers: 0 Good, after resetting programming interface the FIFO is empty Download completed - 2377668 bytes. (expected -1). DONE went high - chip has been successfully programmed. CPCI Information ---------------- Version: 4 (rev 1) Device (Virtex) Information --------------------------- Project directory: tunneling_openflow_switch Project name: Tunneling OpenFlow Switch Project description: Tunneling OpenFlow Switch Device ID: 0 Version: 0.1.0 Built against CPCI version: 3 (rev 4) Download Failed: 256 global/setup exited with value 1 }}} === bitfile issues === cpci_reprogram utilizes bitfiles located in /usr/local/netfpga/bitfiles. this is due to the hard-coded path in the cpci Perl scripts. The bitfile versions must match between all cpci.bit and any project bitfiles involved, or else it will throw the above errors. The bitfiles attached should all work - Download the tarball and extract in /usr/local/netfpga/bitfiles. [[BR]] [[BR]] [[BR]] [wiki:Internal/OpenFlow/ return to OpenFlow index]