wiki:Documentation/fSDN/eNetFpgaTutorial

Version 10 (modified by nkiran, 14 years ago) ( diff )

—

Draft - WIP

An overview of the set up and some "hello world" experiments (with OMF scripts) to get started on NetFPGA/OpenFlow experimentation on SB9

Hardware Set up


Figure 1: Ivan's arch drawing goes here

The switch labeled 'sw-sb-09', a Pronto 3240, provides the central connectivity backplane between all hosts/NetFPGAs in the sandbox. Each host (node1-1..node1-9) is connected to the top switch through one 1GbE data connection on the interface 'exp0'. Also, a second GbE interface - 'control' - on the host is used exclusively for experiment control (incl. ssh/telnet sessions). The control connection is connected to an external control switch.

The first 7 hosts (node1-1..node1-7) each contain a 4x1GbE NetFPGA installed on a PCI slot. The NetFPGA has 4 connections to the top switch, each corresponding to its 4-GbE ports nf2c0-nf2c3. The NetFPGAs also have two 3-Gbps SATA interfaces, presently not connected to anything. If you have a convincing argument (experiment-related, of course) for connecting those up to anything, you'd need to get in touch with an ORBIT chief.

The Pronto switch, which we will call 'top switch' hereon, is a OpenFlow enabled switch and can be run in native or OpenFlow mode by controlling its boot configuration. When in native mode, it runs the Pica8 Xorplus switch software. However, the testbed constructs a higher-level interface to control both the mode of operation (OpenFlow/native), and the configuration of the most relevant functions in the two modes. The interface is simple and HTTP based, and enables the experimenter to control connectivity and other parameters without opening a ssh/telnet session and using the switch CLI - more details in later section.

Some external reference links about above hardware:

Software Details

Custom disk images exist that are preloaded with several of the software necessary for experimenting with the above hardware. These are the available options, which can be further customized using the 'save' function of OMF framework ( refer to the '​How to save...' tutorial):

  • Image 1
  • Image 2

….

  • Image k

Modifying or Freshly Installing NetFPGA/OpenFlow Software

The above images provide a basic software platform to get started with NetFPGA-OpenFlow experimentation. We may also add new images occasionally to update the software with their new stable releases. Beyond that, if you need to do a custom install of NetFPGA/OpenFlow software, you can choose to either build starting from one of the existing images above, or refer to the original instructions from the NetFPGA and OpenFlow development groups online:

The following page documents the installation steps for the the base image 'X' on Ubuntu 10.10:

Gateware

Controlling Sandbox Topology through the Top Switch

ORBIT provides HTTP request API abstraction to controlling the configuration of the top switch. The service implementing this API can be accessed by an experimenter from within the SB9 network, including the console and experiment nodes, at the following address: 

http://nox.orbit-lab.rutgers.edu:5052/network
Or simply as:
http://nox:5052/network

That base URL renders an XML response detailing all requests implemented by the service. Parameter values, such as to address a particular port (port #) on a particular switch (switch IP address), be passed as URL arguments in the HTTP request. For example, to retrieve the details of particular port (say 5) on the top switch (IP=10.19.0.253), the requested URL is: 

http://nox:5052/network/portStat?switch=10.19.0.253&port=5

The service supports the following main features for the native mode (refer to XML response from the base URL for complete list and API specification):

  1. Getting/setting the VLAN ID on any port(s)
  2. Assigning two or more VLAN IDs on a port, making it a trunk
  3. Getting per port statistics (in, out packet counts) and configuration (VLAN) information
  4. Enabling/disabling a port
  5. Turning on/off default switch topology protocols - Spanning Tree Protocol (STP)
  6. Saving the per-port configuration of the entire switch
  7. Restoring switch configuration to prior saved version

In addition, it supports the following requests to support the OpenFlow mode:

  1. Change the mode of the switch from native to OpenFlow mode (and vice versa)
  2. Set the OpenFlow mode parameters including: control port, controller IP, switch ID (dpid), and the transport mode for secure channel

Q. Does the switch have to be in native mode to invoke the setvsi/getvsi requests?

NetFPGA-OpenFlow Experimentation Tutorials using OMF

Tutorial 1: NetFPGA Self-Test

Tutorial 2: NetFPGA (NIC, Switch, Router) Regression Tests

Tutorial 3: NetFPGA-OpenFlow Switch Regression Tests

Tutorial 4: NetFPGA-Switch Load Test

Status log

  • Starting with James' disk image 'netfpga.ndz'. Targeting single image with NetFPGA software, OpenFlow software, Controller sw (NOX), tools
  • Designed two simple testing configurations:
    1. loopback configuration of NetFPGA (with ports c0-3) by looping corresponding ports on top switch to get c0↔c1, and c2↔c3
    2. 4-port switch configuration by connecting 4 hosts to ports c0-c3 via the top switch
  • Intermediate image with NOX and OpenFlow with support for NETFPGA-based switch
  • They require NOX to be run from src/ dir… The 'make install' fails with several Makefile errors - via illegal duplicate installs. Though these can be fixed(and was able to), nox_core still has to be run from the install root. Not an issue for progress…
  • (3/8/11) Trying to pass the switch regression tests on reference NetFPGA switch implementation for tutorial!#2 - ​steps outlined here. Requires a 4 GbE ports - attempting to achieve this via 4 virtual interfaces riding on the data port exp0, connected to the NetFPGA via the top switch. Yet to run the regression tests…have to see if the CentOS scripts hold up on Ubuntu.
  • (3/8/11) Next step is to get the NOX controller going with simple topology control for the top switch. Require the following information:
    1. How to change the firmware on the Pronto switch to OpenFlow? Also, a secchannel to controller.
    2. Is there a simple component for NOX that allows setting up topology/flow rules?
  • Ivan has a service to configure the top switch from Pronto native to OpenFlow mode and vice versa. Invoked over http, the service uses telnet and reboots in reqd. configuration. However, he prefers using VLANs on the switch for topology control, configured through an existing HTTP-based service. We can control the NetFPGA switches via OpenFlow. The other option for controlling flows on OpenFlowNetFPGA switch is using dpctl on the host. This last option allows for using simple OMF scripting for dynamic control of flows.
    • HTTP-based service to configure the Pronto switch is accessible from within ORBIT at: ​http://nox:5052/network. Interface is simple, for example to set a vlan id on a particular port you do: http://nox:5052/network/setvlan?switch=IP&port=X&vlan=Y
  • (3/9/11) Set up a disk image with NetFPGA and OpenFlow starting from a baseline Ubuntu image. It had become hard to understand what modules preexisted on the existing netfpga image, and dependencies were getting messed up as well. I'll probably go for a separate image for the controller to keep things clean. Created two special user accounts (with full sudo access) on this new image, one each for OpenFlow and the NetFPGA set ups. User accounts mainly created to closely track the install process described by those two developer groups. The set of pages with instructions used to do this step:
  • (3/9/11) Session terminal log​ - there were several changes in the steps to perform Ubuntu equivalents of the steps listed above for CentOS.
  • (3/9/11) Next step is to make sure we can run all regression tests defined by developers on this image. Remains to be seen if I can get away with the virtual interfaces for the 4-port GbE NIC they require for the regression tests. Then ensure interaction of the NetFPGA-Openflow switch with NOX controller.
  • (3/10/11) Ran the selftest with the 4 ports connected upto the switch. PHY kept failing mainly due to the indirect links via the switch. The corresponding 4 ports on the switch were on same VLANs and traffic was getting duplicated. Selftest code checks for exactness of source packet and definite ordering of packets by cycling through a sequence of patterns in the payload. After setting the VLANs to match selftest connection set up, the PHY still failed because of misconnection of NetFPGA ports to switch ports. After setting this right, there are still a small number of bad pkts (25/~250K), not sure why. Switch drops or undue delays, or bitflips are the guesses. Then again, it's a little hard to read the Verilog tests.
  • (3/10/11) Next is the regression tests, but I was unable to hookup the regression configuration that requires a 2-4 port GbE connected with the NetFPGA ports. The virtual ifaces outlined above would work only if the ifaces can be assigned to specific vlans. 'vconfig' can do this. The 4 virtual ifaces connect to the switch on a trunk port that trunks vlans 1,2,3,4 one for each of the NetFPGA ports. Configuring the switch for this trunk port fails at present — couldn't figure out how to specify a list of vlans to be set for the port.

  • (3/11/11) VLANS assigned to host virtual interfaces using vconfig. However, the names assigned to these ifaces is inflexible, and quite sadly, the NetFPGA regression tests are hard-coded to use 2 ethernet ports (non NetFPGA) on host names 'eth1' and 'eth2'. Modified one of the regression tests - reference_router (in a copied project dir 'reference_nic_mod' - to replace eth1 with exp0.1 created by vconfig, and similarly eth2 with exp0.2 - perl one liner to the rescue: 
perl -p -i -e 's/oldstring/newstring/g' `grep -ril oldstring *`
  • (3/11/11) But the router regression tests still FAIL. NF ifaces nf2c0-nf2c3 are on switch ports 1-4 and are assigned VLANS 1-4, respectively. The exp0 iface is on switch port 5, and is assigned VLAN trunk 1,2. The virtual interfaces on host, exp0.1 & exp0.2, are correspondingly assigned VLAN ids 1 & 2. The regression tests are throwing up all kind of errors, unexpected packets, missing packets, register read errors… Some of the errors I think are attributable to failure to execute 'usleep' - need to change those to 'sleep' equivalents. But there are 'unexpected pkt' errors on unconnected NetFPGA ports like 'nf2c2'??? Have to check this out correctly before going onto OpenFlow controller. If we get this going, then we have 2 configurations set up just using the top switch and by setting VLANs : 1.) selftest with nf2c0↔nf2c1, nf2c2↔nf2c3, and 2.) switch/router/nic regression with nf2c0↔exp0.1,nf2c1↔exp0.2

Attachments (5)

Note: See TracWiki for help on using the wiki.