wiki:Internal/OpenFlow/HostSetup

return to OpenFlow index

NetFPGA Host Setup

This is the latest set of instructions for installing NetFPGA drivers on Ubuntu (10.10), and making it work as an OpenFlow switch. It is assumed that your working directory is /opt/.

I. NetFPGA setup

This section describes the steps required to build v.2.2.0 of the NetFPGA kernel module on Ubuntu. The same steps can be used to build more recent versions. Steps to build the older version of the drivers can be found here.

1.1 Download sources/dependencies

In order to download tarballs, you need a Foswiki account. Get one ​here if you don't have one.

cd to /opt/ and fetch and untar the following:

or, if you want the tunneling OpenFlow switch fetch the following in place of netfpga_openflow_switch_1_0_0-4.

  • netfpga_tunneling_openflow_icmp_arp_1_0_0.tar.gz, from the same link as above.
  • install dependencies 
    apt-get update
apt-get install build-essential linux-headers-`uname -r` ncurses-dev libnet1-dev libxml-simple-perl libio-interface-perl liblist-moreutils-perl liberror-perl git-core libpcap0.8 libpcap0.8-dev

1.2 Compatibility fixes

The following changes have to be made in order to compile the drivers on Ubuntu.

  • Add the following symlinks: 
    ln /usr/bin/lspci /sbin/lspci
ln /usr/bin/setpci /sbin/setpci
  • /opt/netfpga/lib/C/kernel/nf2util.h : add after the first #ifndef/#define block: 
    // Replace SA_SHIRQ with IRQF_SHARED on newer kernels
#ifndef SA_SHIRQ
#define SA_SHIRQ IRQF_SHARED
#endif
  • /etc/default/grub: add "vmalloc=512m" to GRUB_CMDLINE_LINUX_DEFAULT and do update-grub

For the kernel:

  • If using kernel version 2.6.35-25, upgrade to 2.6.35 (uname -ar should show you which one you are running): 
    apt-get install linux-headers-2.6.35-30-generic linux-image-2.6.35-30-generic
  • Reboot. After reboot, check that the kernel flags are correct: 
    dmesg | grep vmalloc
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.35-30-generic root=UUID=c90285a2-48de-4ab0-950f-241817516ad8 ro vmalloc=512m console=tty0 console=ttyS0,115200n8
[    0.000000]     vmalloc : 0xdf7fe000 - 0xff7fe000   ( 512 MB)

1.3 Compile drivers

  • export environment variables: 
    export NF_ROOT=/opt/netfpga  #(wherever your netfpga base files are)
source ${NF_ROOT}/bin/nf_profile

  • from /opt/netfpga: make; make install

After compilation, load the kernel module: 

modprobe nf2

If all goes well, you should see the an entry if you check with lsmod | grep nf2. At this point, you can load bitfiles and such by first running cpci_reprogram and then using nf_download to program the NetFPGA with the correct bitfile. Your session should go something like this: 

# cpci_reprogram.pl --all
Loading the CPCI Reprogrammer on NetFPGA 0
Loading the CPCI on NetFPGA 0
CPCI on NetFPGA 0 has been successfully reprogrammed
# nf_download /opt/netfpga/bitfiles/reference_switch.bit
Found net device: nf2c0
Bit file built from: nf2_top_par.ncd;HW_TIMEOUT=FALSE
Part: 2vp50ff1152
Date: 2011/ 1/20
Time:  6: 8:48
Error Registers: 0
Good, after resetting programming interface the FIFO is empty
Download completed -  2377668 bytes. (expected 2377668).
DONE went high - chip has been successfully programmed.
CPCI Information
----------------
Version: 4 (rev 1)

Device (Virtex) Information
---------------------------
Project directory: reference_switch
Project name: Reference Switch
Project description: Reference Switch

Device ID: 3
Version: 1.0.0
Built against CPCI version: 4 (rev 1)

Virtex design compiled against active CPCI version

II. NetFPGA OpenFlow switch

The above instructions apply to any recent version of NetFPGA driver (v2.1.2 - 3.0.1). The following steps are needed in order to use the NetFPGA as an OpenFlow switch supporting v1.0 of the OpenFlow protocol. While any version of the NetFPGA drivers /should/ work, the OpenFlow components were built atop v2.2.0 of the NetFPGA drivers.

These instructions assume that you have already gotten the NetFPGA up and running on an Ubuntu box, as per section 1 of this guide.

According to the guides, OpenVswitch provides a kernel implementation of the OF switch, whereas Stanford's reference switch is purely userspace. The former is faster, the latter more customizable. The NetFPGA OpenFlow switch utilizes the latter, but can probably be used with OpenVswitch as well (though yet to be tested with a node).

2.1 dependencies

  • Install dependencies : 
    apt-get install autoconf libtool
  • Use git to pull NetFPGA-specific OpenFlow sources to /opt/: 
    git clone git://openflowswitch.org/openflow.git
cd openflow
git checkout -b 1.0.0-netfpga origin/devel/tyabe/1.0.0-netfpga

2.2 build OpenFlow

All bitfiles belong in the directory /opt/openflow/hw-lib/nf2. Steps 3 and 4 below download the OpenFlow switch bitfiles into this directory. 

./boot.sh
cd ./hw-lib/nf2
wget http://openflowswitch.org/downloads/netfpga/openflow_switch.bit.100_3.tar.gz
tar zxvf openflow_switch.bit.100_3.tar.gz ==> we need newer one from NF site:http://www.netfpga.org/releases/netfpga_openflow_switch_1_0_0-3.tar.gz
cd ../../
./configure --enable-hw-lib=nf2
make
make install

After compilation, copy of_start.sh and of_stop.sh from /opt/netfpga/projects/openflow_switch/sw/. Given that you have a OpenFlow controller up and running, you can connect the switch to the controller with the following: 

./of_start.sh x.x.x.x:6633

Where x.x.x.x is the controller's IP address and 6633 the control port number. This automatically generates a random DPID for the switch; if you want to specify a DPID, modify the script such that ofdatapath takes the -d parameter for DPID, e.g: 

/opt/openflow/udatapath/ofdatapath --detach punix:/var/run/dp0 -d 001010222324 -i nf2c0,nf2c1,nf2c2,nf2c3

You can then run the same command as: 

./of_start.sh 172.16.0.14:6633 000000101010

assigning the switch the DPID 000000101010. Making it so that the script takes DPID as an argument from the command-line is left as an exercise to the reader. ./of_stop.sh turns the OpenFlow components off.

III. NetFPGA Tunneling OpenFlow switch

This is the L2 tunnel version of the OpenFlow bitfiles for the NetFPGA. This section is based on notes taken for the following setup:

  • Ubuntu 11.04
  • netfpga_tunneling_openflow_icmp_arp_1_0_0.tar.gz
  • netfpga_full_3_0_0.tar.gz

Setup is described fairly well in the ​NetFPGA documentation, although a few steps necessary to work out kinks from their transitions between versions are not mentioned.

  • In /opt/netfpga/lib/C/kernel/nf2main.c, line 190 should be changed from 
    init_MUTEX(&card->state_lock);
    to 
    sema_init(&card->state_lock, 1);

This reflects changes made to newer kernels (2.6.37 and above) like the one used in 11.04. Note, this is already taken care of in newer NetFPGA code (>3.0.0).

  • When moving header and bitfiles into the OpenFlow directory, rename and move them to match start script conventions. In /opt/openflow/hw-lib/nf2: 
    mkdir sw
cp /opt/netfpga/projects/tunneling_openflow_icmp_arp/lib/C/reg_defines_tunneling_openflow_icmp_arp.h sw/reg_defines_tunneling_openflow_switch.h
cp /opt/netfpga/bitfiles/tunneling_openflow_icmp_arp.bit sw/tunneling_openflow_switch.bit

The following are outdated, use with caution

Setting up NetFPGA hosts: Ubuntu 10.04

Current Ubuntu NetFPGAs run Ubuntu version 8.04 and OpenFlow ver. 1.0. Here we try to move to newer distributions w/ OpenFlow Ver. 1.0. The hardware we use here is the NetFPGA cube, another version of the NetFPGA pre-built solution. As per the ​!OpenFlow portion of the setup, a user named 'openflow' with root privs was added for this purpose.

Setup of the NetFPGA using earlier distributions can be found in the Diaries.

Linux distro:
Ubuntu Server 10.04, w/ xfce4 WM, openssh-server
kernel: 2.6.32-21-generic-pae

Packages:
NetFPGA base 2.1.0
Tunneling OpenFlow NetFPGA 1.0.0

Installing Ubuntu.

  1. set boot priority of CD drive to 1st in BIOS
  2. If desktop edition, boot as a live CD and install from there. Trying direct install just results in the system proceeding to live CD Mode (10.04). Server edition can be installed directly w/out issues.

I: Setup - NetFPGA drivers.

1. dependencies.

There is an updated dependency list: 

build-essential ncurses-dev libnet1-dev libxml-simple-perl libio-interface-perl liblist-moreutils-perl liberror-perl libnet-rawip-perl sun-java6-jre sun-java6-jdk libpcap0.8-dev

Installing java packages. In 10.04, sun-java6-jre, jdk are obsoleted and moved to the partner repository. This repository can be added w/ the following (ref: ​https://help.ubuntu.com/community/Repositories/CommandLine):

  1. copy "deb http://archive.canonical.com/ lucid partner" sans quotes to /etc/apt/sources.list
  2. run apt-get update.
  3. run apt-get as usual for the packages.

2. Kernel memory allocation.

Ubuntu versions beyond 8.04 (assume anything w/ kernels beyond v 2.6.28) use GRUB 2. GRUB 2 gets rid of menu.lst, so kernel memory must be allocated via the following method (ref. ​http://ohioloco.ubuntuforums.org/showthread.php?p=8229620):

Add "vmalloc=512m" to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub so it looks like this: 

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vmalloc=512m"

Then run update-grub to regenerate /boot/grub/grub.cfg with the new parameter (you never want to modify grub.cfg directly).

3. Symlinks.

Download libpcap.so.0.9.4, found below. Move it into /usr/lib. Run ubuntu_setup.sh (also attached below), or manually create the following links: 

ln /usr/bin/lspci /sbin/lspci
ln /usr/bin/setpci /sbin/setpci
ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0.9 
ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so.0
ln -s /usr/lib/libpcap.so.0.9.4 /usr/lib/libpcap.so

4. Edit files.

Download Netfpga code: ​http://netfpga.org/foswiki/NetFPGA/OneGig/Releases

  • edit "usleep 250" to "sleep 0.00025" in /test_loopback_drop/run.pl
  • edit /home/netfpga/NF2/projects/router_kit/sw/rkd.cc to include limits.h

NOTE: this edit is no longer required for version 2.1.1 of the package.

5. Setting Variables.

After setting NF_ROOT to the netfpga directory, source nf_profile, located in ${NF_ROOT}/bin/ 

export NF_ROOT=/home/openflow/netfpga  #(wherever your netfpga base files are) 
source ${NF_ROOT}/bin/nf_profile

6. Moment of truth.

make then make install.
Then, check for nf2 module with lsmod | grep nf2: 

nf2                    13425  0

and for nf2cn, n=0:3 with ifconfig -a. 

nf2c0     Link encap:Ethernet  HWaddr 00:4e:46:32:43:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:16 

nf2c1     Link encap:Ethernet  HWaddr 00:4e:46:32:43:01  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:16 

nf2c2     Link encap:Ethernet  HWaddr 00:4e:46:32:43:02  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:16 

nf2c3     Link encap:Ethernet  HWaddr 00:4e:46:32:43:03  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:16

7. add the cpci reprogrammer to startup routine

Add the following to /etc/rc.local: 

/home/openflow/netfpga/lib/scripts/cpci_reprogram/cpci_reprogram.pl --all

The selftest does not work in Ubuntu. so don't try it.

II: Setup - OpenFlow components.

The OpenFlow wiki on CentOS NetFPGA setup were followed in conjunction with the netfpga site wiki for the Tunneling OpenFlow NetFPGA. Stes are reiterated here since nothing is permanent.

1. dependencies.

autoconf libtool pkg-config linux-source git-core automake m4

2. OpenFlow package installation

This happens in the same directory as the netfpga install. From git repository (automatically pulls the newest version): 

git clone git://openflowswitch.org/openflow.git
cd openflow
./boot.sh

To make NetFPGA compatible: 

./configure --enable-hw-lib=nf2

Remove avahi-daemon and disable IPv6: 

sudo apt-get remove avahi-daemon

in /etc/sysctl.conf add 

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

and in /etc/modprobe.d/blacklist.conf 

blacklist net-pf-10
blacklist ipv6

3. NetFPGA tunnel package

Download and untar in /home/openflow/. Move tunneling_openflow_switch.bit and reg_defines_tunneling_openflow_switch.h to your_openflow_directory/hw-lib/nf2/. The files should be located under bitfiles/ and /projects/tunneling_openflow_switch/lib/C/ in the netfpga directory, respectively. Do a make and make install in the openflow directory.

4. Regression test

The setup should be wired as such 

 eth1 <----> nf2c0 
 eth2 <----> nf2c1

The test can be run with the following command: 

nf_regress_test.pl --project tunneling_openflow_switch

As of now, getting this error: 

root@node1-2:~/netfpga/lib/scripts/cpci_reprogram# /home/akoshibe/netfpga/bin/n_regress_test.pl --project tunneling_openflow_switch

NetFPGA environment:
  Root dir:       /home/openflow/netfpga
  Project name:   tunneling_openflow_switch
  Project dir:    /home/openflow/netfpga/projects/tunneling_openflow_switch
  Work dir:       /tmp/root

Root directory is /home/openflow/netfpga
Running tests on project 'tunneling_openflow_switch'...
  Running global setup... FAIL
Output was:

NetFPGA environment:
  Root dir:       /home/openflow/netfpga
  Project name:   tunneling_openflow_switch
  Project dir:    /home/openflow/netfpga/projects/tunneling_openflow_switch
  Work dir:       /tmp/root

Found net device: nf2c0
Error: Virtex design compiled against a different CPCI version
  Active CPCI version : 4 (rev 1)
  Device built against: 3 (rev 4)
Bit file built from: nf2_top_par.ncd
Part: 2vp50ff1152
Date: 2010/ 5/ 4
Time: 15:49:32
Error Registers: 0
Good, after resetting programming interface the FIFO is empty
Download completed -  2377668 bytes. (expected -1).
DONE went high - chip has been successfully programmed.
CPCI Information
----------------
Version: 4 (rev 1)

Device (Virtex) Information
---------------------------
Project directory: tunneling_openflow_switch
Project name: Tunneling OpenFlow Switch
Project description: Tunneling OpenFlow Switch

Device ID: 0
Version: 0.1.0
Built against CPCI version: 3 (rev 4)

Download Failed: 256


global/setup exited with value 1

bitfile issues

cpci_reprogram utilizes bitfiles located in /usr/local/netfpga/bitfiles. this is due to the hard-coded path in the cpci Perl scripts. The bitfile versions must match between all cpci.bit and any project bitfiles involved, or else it will throw the above errors. The bitfiles attached should all work - Download the tarball and extract in /usr/local/netfpga/bitfiles.




return to OpenFlow index

Last modified 13 years ago Last modified on Nov 22, 2011, 7:25:45 PM

Attachments (3)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.