[#bottom jump to bottom] [[BR]]
= The IP8800 Packet Switch = 
The IP8800/S3640 is a smart switch built by NEC capable of running firmware compliant with !OpenFlow networking. WINLAB utilizes IP8800 switches in its experimental network infrastructure. This is a quick-start guide (hopefully) suitable for anyone trying to learn how to deploy the NEC IP8800 on the network as an !OpenFlow device.

== Index == #index
 [#preamble Before we begin...] [[BR]]
 [#I I Introduction] [[BR]]
  [#Serial 1.1 Connection via serial] [[BR]]
  [#Modes 1.2 User modes] [[BR]]
  [#Help 1.3 Finding Information] [[BR]]
  [#lookup 1.4 device information lookup] [[BR]]
  [#unix 1.5 UNIX commands] [[BR]]
  [#power 1.6 Powering off] [[BR]]
 [#II II Basic Configuration Tasks] [[BR]]
  [#Contexts 2.1 Contexts] [[BR]]
  [#vlans 2.2 VLANs and VLAN Interfaces] [[BR]]
  [#Ports 2.3 Ports] [[BR]]     
  [#Telnet 2.4 Telnet] [[BR]]
 [#III III Miscellaneous Configuration Tasks] [[BR]]
  [#acls 3.1 ACLs and SNMP] [[BR]] 
  [#dhClient 3.2 DHCP Relay Agent] [[BR]]
  [#dhServer 3.3 DHCP Server] [[BR]]
  [#mirror 3.4 Port Mirroring] [[BR]]
  [#accounts 3.5 Users and Passwords] [[BR]]
 [#IV IV OpenFlow Switching] [[BR]]
  [#intro 4.1 What is OpenFlow? ] [[BR]]
  [#outline 4.2 Outline] [[BR]]
  [#ofconf 4.3 Configuring the switch] [[BR]]
  [#sdprep 4.4 Preparing the SD card] [[BR]]
  [#flash 4.5 Flashing the switch] [[BR]]
  [#ofcom 4.6 OpenFlow Commands] [[BR]]
  
   
----
== Before we begin... == #preamble

A few things before moving onto business. Firstly, Here are the assumptions that are made in this handbook: 
 * You are using the NEC IP8800/S3640-48T2XW switch. Commands vary even amongst similar lines of devices. 
 * You are familiar with, and running some form of *nix on your PC (We are, for the most part, using Ubuntu unless specified) 
 * You have a serial port on your machine, and have either minicom or kermit installed on your machine.

''' References ''' [[BR]]
The full IP8800 operations manual can be found online. Then why am I taking my time to make this handbook? For one, the online manual is in Japanese. Even if you are literate in Japanese, the manual is quite extensive, and doesn't cover the !OpenFlow side of things. So, to sum things up -- the purpose of this guide is to provide quick reference to the switches' basic functionalities so that no-body has to dig through the handbook to find out, say, how to safely shut the switch down. But, for those of you who want to check out the whole shebang, here are the links:

 * Vol. 1: http://www.nec.co.jp/ip88n/s36_sw/html/cfguide/index.html     
 * Vol. 2: http://www.nec.co.jp/ip88n/s36_sw/html/cfguide2/index.html   
 * Vol. 3: http://www.nec.co.jp/ip88n/s36_sw/html/cfguide3/index.html     

The switches may also come with 2 CDs containing the English translated version of the manual. The !OpenFlow manual comes with the firmware.  

'''Conventions'''[[BR]]
Text in `console font` and in quotations are commands that can be entered at the command line. When in quotations, the quotes should be omitted. Words surrounded by carets are various options for a command. If delimited by pipes (|), one out of the several commands separated by pipes can be entered in that spot. For example: 
{{{
interface gigabitethernet <src port> <tx|rx|both> 
}}}
Here, <src port> is a value between 0/1 and 0/48, and can be followed by keyword `tx`, `rx`, or `both`. 

Each section outlined in the index starts with a short overview, followed by step-by-step instructions (if applicable), and ends with troubleshooting error messages that are associated with each.
 
[[BR]][[BR]]
----

= I Introduction = #I 
Many "Smart" switches, including the IP8800 series switches, must be configured using the Command-line-interface(CLI). This usually involves a serial, telnet, or ssh session through a terminal. This chapter explains how to connect to the switch via serial, and covers some basic commands and operational tasks to get started with these switches.  

[[BR]]

== 1.1 Connecting via serial == #Serial
Telnet is not enabled by default on the switches. Therefore, initial configurations must be done through a serial connection. The steps below describe how to establish a serial connection with the switch using kermit, as its default settings allow it to connect without any modifications. Few laptops have serial ports these days, so have a desktop or a USB-to-serial converter handy. 

If not using kermit, the following settings are required if settings must be modified:
{{{
  * speed: 9600
  * data bits: 8
  * stop bits: 1 
  * Flow control: none
  * Parity: none
}}}  

=== Steps. ===
1. ''Name your switch.'' People names are easier to remember than IP addresses or machine-like names i.e. as01-hh-alex, and are easier to use in discussion if everyone agrees on names. Note, the names Bob, Nancy, and Andy are already taken. 

2. ''Establish a connection.'' Connect your PC serial port to the port labeled "console" on the switch using a RS-232 cable, and issue the following commands as root from a terminal:
{{{
 # kermit
 C-Kermit>set line /dev/ttyS0
 C-Kermit>set carrier-watch off
 C-Kermit>connect
}}}

The following is also valid:
{{{
 kermit -l /dev/ttyS0
 set carrier-watch off
 connect
}}}

If all goes well, you should then see something like below. If the login prompt doesn't come up, hitting the enter key should bring it up. 

{{{
 Connecting to /dev/ttyS0, speed 0
  Escape character: Ctrl-\ (ASCII 28, FS): enabled
 Type the escape character followed by C to get back,
 or followed by ? to see other options.
 ----------------------------------------------------


 login:

}}}

3. ''Login.'' The default username is "operator", and the password is blank. Once logged in, you will be brought to the CLI. Commands are entered at the caret. 

{{{
login: operator

Copyright (c) 2005-2008 ALAXALA Networks Corporation. All rights reserved.


> 
}}}     

4. ''exiting.'' `Ctrl-\ q` ends the kermit session. The command `exit` or  `logout` will just log you out of the switch, but will not end the session.   

=== Troubleshooting serial connections ===
 * If using kermit, make sure you are root when you start up your connection. minicom does not require you to be root.  

 * Kermit will balk if you try to log in while there is a session going on. Make sure that no one else is logged in via serial, and that you haven't forgotten to log out during a previous session.      

[#index return to index]
[[BR]]

== 1.2 User modes == #Modes
=== Enabled Mode ===
You are in initially in user mode when you log into the switch. Only a limited set of commands for status lookup are accessible from user mode. In order to gain full access to device (e.g. for configuration) you must be in ''enabled'' mode. For all intensive purposes, this is equivalent to superuser mode in *nix. The command `enable` allows a user to enter enabled mode. The whole command need not be entered at the prompt, as the switch is capable of command interpretation.

{{{
> ena     <<---whole command is 'enable' -- the switch assumes and autocompletes commands.
#
}}}   
Note that once in enabled mode, the caret becomes a pound. Command "`exit`" exits you from enabled mode (and any other modes or contexts, such as configuration mode explained next).   

=== Configuration Mode ===
Once in enabled mode, switches can be configured by entering ''configuration mode''. All configurations, from ACLs to port speed, are done from this mode, or, as we may interchangeably call, ''context''. Configuration mode may be entered via the command `configure`. 

{{{
# configure
(config)# 
}}}  

We will be delving further into the details of configuration starting with [#II chapter II] of the guide.

[#index return to index]
[[BR]]
== 1.3 Finding Information == #Help
Typing `?` at the prompt will display all of the basic commands that are available. Typing `?` after a articular command will display the available flags. 

{{{
sw-sb09> reload ?
  -f             Restart without confirmation
  dump-image     Memory dump is extracted and preserved
  no-dump-image  Memory dump is not extracted and preserved
  stop           Stop, but do not restart
  <cr>
}}}
Here, `<cr>` means hit `return`.  

An alphabetized list of commands and their usage can be found [http://www.nec.co.jp/ip88n/s36_sw/html/commindx/index.html here], although this is also in Japanese. 

The switch also recognizes some UNIX commands (see the [#unix UNIX commands] section for details). For such commands (and !OpenFlow commands such as `showswitch`), "`command --help`" or the flag `-h` may also be used to get information about a certain command.    

[[BR]]
== 1.4 Device information lookup == #lookup
This is primarily done through the `show` command. Some useful lines:
 * `show configuration` - all current configurations, e.g. hostname, VLANs, ACLs, ports
 * `show port statistics` - port status (interface name, up, down, ect.)
 * `show port vlan` - list all port-to-VLAN associations
 * `show version` - System information e.g. serial number, OS version
   
For a complete list, do "show ?". 

In configuration mode, the `show` command will display the configurations for the current device element you are configuring:
{{{
(config)# interface gigabitethernet 0/27    <<---entering port 0/27 config prompt
(config-if)# show
interface gigabitethernet 0/27
  switchport mode access
  switchport access vlan 4
!
}}}

Again, more details about the ''hows'' of configuration can be found in [#II chapter II] of the guide. 

[[BR]]
== 1.5 UNIX commands == #unix
The core OS of the switch is NetBSD 2.0. Therefore, some *nix commands can be used at the CLI. See /usr/local/bin, /usr/local/sbin, /usr/bin and /usr/sbin for a list of available commands. For example, the up arrow key will let you access previously issued commands, as will `!`. 

'''some commands''' [[BR]]
Note, enabled mode is not true superuser (user "admin", not "root"), so your actions are quite restricted. However, some useful commands are:
 * find <rootdir> -/name <targetname> : find a *nix command
 * /usr/bin/clear : clear the terminal screen 
 * /usr/bin/who : list users logged in 
 * /sbin/sysctl : system specs/status
 * /sbin/dmesg : display system message buffer
 * scp : better (more intuitive, I think) than ftp 

If you feel compelled to read them, try the [http://netbsd.gw.com/cgi-bin/man-cgi/man?+ANY+NetBSD-2.0 Online NetBSD man pages]. The built-in man pages are not very functional. [[BR]][[BR]]  
          
'''text editors''' [[BR]] 
If you ''desperately'' need to, and can't do it any way else, things (e.g. your !OpenFlow config file, discussed in [#IV Chapter IV]) can be edited using `ed`. Be forewarned, this is not for the faint-hearted. 

[#index return to index]  
[[BR]]
== 1.6 Powering off == #power
Yanking the power on a switch suddenly can cause file corruption, so we want to properly stop all processes before cutting the power. To shut down a switch, issue the command `reload stop`. This halts the system (stops all processes), but does not power the hardware off. After the system halts, the rom prompt will appear. At this point, you'd have to manually turn the switch off. 
{{{
sw-sb09-da# reload stop
Dump information extracted? (y/n): y
old dump file(rmdump  07/21 21:16) delete OK? (y/n): y
sw-sb09-da#
ROM 00.03.52
Rom:                                      
}}}  

[[BR]][[BR]]
----
= II Basic Configuration Tasks = #II

This chapter covers the basics of switch configuration. We will first take a look at some of the more important contexts available from configuration mode. The overview will be followed by a look at lower layer constructs (ports, VLANs, IP addresses) with respect to the switch, and how to configure these elements. This chapter will end with the configuration of telnet so that we can access the switch over the network.      

== 2.1 Contexts == #Contexts
'''Entering Contexts''' [[BR]]
All configurations are done by entering various contexts from the (config)# prompt. Some contexts we want to focus on are:

 * `vlan n` - create/configure standard VLANs, n being the VLAN ID (some number)
 * `interface vlan n` - create/ configure a virtual interface with both Layer 2 and 3 characteristics 
 * `interface gigabitethernet 0/n` - configure specific port, n being number between 1 and 48
 * `line <vty|console> vtyrange<0-b>` - configure telnet or console parameters, respectively  

The details of each context will be introduced in the following sections.  

when you enter a context, the prompt becomes "(config-if)#." Note, however, this prompt doesn't indicate what context you are using (i.e. which port you are setting up, what VLAN, ect); you just have to keep track of what you are doing. 

{{{
(config)# interface vlan 8              <<---start by entering the 'interface vlan' context             
!(config-if)# 
!(config-if)# interface gi 0/3           <<---switch to 'interface gigabitethernet' context 
!(config-if)#                            <<---note how prompt still shows up as "config-if"
}}} 

If you forget where you were in a context (i.e. which vlan you were configuring, ect.), typing "show" will display all of the configurations saved for that specific context (hopefully giving you enough information to let you recall what you were trying to configure). 

As you can see above, when there are unsaved changes in the configurations (e.g. a new vlan interface created, as in the case above), you will see a "!" before the prompt. This is there to remind you that you've made changes that must be saved if you want them to survive a reboot. The command "save" intuitively saves the settings. 

'''undoing configurations.''' [[BR]]
Re-running a command with "no" prepended to it will usually undo the configuration. Exceptions to this rule will be mentioned in the appropriate sections. 

'''exiting/ logging off. '''
The command `exit` lets you exit from contexts and enabled mode, or lets you log out of the switch if you were just in user mode.
{{{
!(config-if)# save
(config-if)# exit
(config)# exit
#
}}}

[#index return to index]
[[BR]]
== 2.2 VLANs and VLAN Interfaces == #vlans
=== Index === #vindex
This section is divided into the following sub-sections.
 [#vback 2.2.1. Background] [[BR]]
 [#create 2.2.2. VLAN creation] [[BR]]
 [#vname 2.2.3. Naming VLANs] [[BR]]
 [#toports 2.2.4. Associating VLANs to ports] [[BR]]
 [#vints 2.2.5. IP address assignment] [[BR]]
 [#delv 2.2.6. VLAN deletion] [[BR]]
=== 2.2.1. Background === #vback
VLANs are a technology defined by IEEE 802.1. In terms of the switch, VLANs allow a switch to group the hosts it sees on its port based on their L2 addresses. The L2 addresses are associated with the port that it is seen on; therefore, the port that the host is attached to determines which group, or VLAN, the host belongs in. 

The "vlan" context allows you to create and configure dot1Q VLANs that can be associated with ports. The "interface vlan" context allows you to configure network layer features of a VLAN by tying it to a virtual interface on layer 3. The latter context is typically used to configure IP addresses associated with the switch.  

=== 2.2.2. VLAN creation === #create
The switch comes with VLAN 1 by default, with all ports associated with it. New VLANs can be created by entering the `vlan` context for a non-existent VLAN ID from the config prompt. Entering the `interface vlan` context for a nonexistent VLAN will also do the same thing.  

''creating multiple vlans.'' Multiple VLANs can be created at once with the command `vlan <vlan range> <vlan IDs>`. The `interface range vlan` context, however, cannot be used in this manner; The range version of the interface vlan context can only be used to configure groups of VLANs that already exist. The following error will be thrown if you try to apply the context to a non-existent VLAN, in this case, VLAN 7:
{{{
(config)# interface range vlan 7-38
interface : Not found VLAN-ID <7>.
}}}

more information on the `interface vlan` context can be found in [#vints section 2.2.5].

''Example.'' The following illustrates typical steps for creating VLANs. We first generate VLANs 1-8, then 11-48, and finally, 50:
{{{
(config)# vlan 1-8 
!(config-vlan)# vlan 11-48
!(config-vlan)# vlan 50
!(config-vlan)# save
(config-vlan)# 
}}}

[#vindex return to section index] [[BR]]
[#index return to main index] [[BR]]
=== 2.2.3. Naming VLANs === #vname
Names are a good way to keep track of the various VLANs you create. VLANs are named through the `vlan` context with the `name` command. 
{{{
(config-vlan)# vlan 2
(config-vlan)# name "Internal"
}}}  

=== 2.2.4. Associating VLANs to ports === #toports
Port VLAN membership is configured through the `interface gigabitethernet <port>` context using the `switchport access vlan <VLAN ID>` command. 
{{{
(config)# interface gigabitethernet 0/25                 
(config-if)# switchport mode access 
(config-if)# swi acc vlan 3           <<-- full command is "switchport access vlan 3"   
}}}

Here, "switchport mode access" specifies that the traffic to and from this port will not be tagged. More on port configuration is discussed in [#Ports section 2.3]. 

[#vindex return to section index] [[BR]]
[#index return to main index]
[[BR]]
=== 2.2.5. IP address assignment === #vints
VLAN interfaces are abstractions that allow you to configure higher layer services and protocols that act upon ports based on their VLAN membership. One way you'd find yourself using them is to assign a management IP address to your switch.
{{{
sw-sb09-da(config)# interface vlan 1
sw-sb09-da(config-if)# ip address 172.19.10.100 255.255.0.0
!sw-sb09-da(config-if)# 
}}}     
The commands above assign the switch an IP address of 172.19.10.100 with subnet mask 255.255.0.0. As this is assigned to VLAN 1's VLAN interface, this address is only visible to devices on VLAN 1.   

'''undoing static IP assignments to virtual interfaces''' [[BR]]
When you have [#dhClient IP-helpers] configured, you want to type "no ip helper-address" at the CLI before trying to remove the IP address or else you get this warning:   
{{{
interface vlan 3
  ip address 192.168.5.5 255.255.255.0
  ip helper-address 192.168.5.1
!
(config-if)# no  ip address 192.168.5.5 255.255.255.0
interface : IP interface is not defined.
}}}

[[BR]]
=== 2.2.6. VLAN deletion === #delv
'''Deleting a VLAN'''[[BR]]
Any configurations involving a VLAN you want to remove has to be undone before deleting the VLAN. not doing so will result in the following error when you issue the `no vlan <n>` context:
{{{ 
(config)# no vlan 4
vlan : Can't delete this configuration referred by other configuration.
}}}   
In this case, VLAN 4 has port 0/27 associated with it. Undoing the association allows you to delete the VLAN without issues.  
{{{
(config)# int gi 0/27
(config-if)# no sw acc vlan 4
!(config-if)# exit
!(config)# no vlan 4             <<-----second attempt
!(config)# 
}}}

[#vindex return to section index] [[BR]]
[#index return to main index]
[[BR]][[BR]]
== 2.3 Ports == #Ports 
=== Index === #pindex  
The following subsections are found here:
 [#pback 2.3.1. Background] [[BR]] 
 [#pmodes 2.3.2. port modes] [[BR]]
 [#assoc 2.3.3. VLAN association] [[BR]]
 [#Trunking 2.3.4. Configuring trunk ports] [[BR]]
 [#speed 2.3.5. port speed] [[BR]]

=== 2.3.1. Background === #pback
There are 48 standard RJ-45 ports and 2 fiber ports on the switch recognized as 'geth' (1Gbs) and 'tengeth' (10Gbps) by the device, respectively. We focus only on the 'geth' interfaces. The context to configure these ports is `interface gigabitethernet <port>`, where <port> is a value between 0/1 and 0/48. `interface range gigabitethernet <port>` allows you to configure multiple ports at once. 
 

=== 2.3.2. port modes === #pmodes 
The switchports are access ports by default. Access ports restrict L2 broadcasts by limiting the ports that get flooded to those in the same VLAN affiliation as the port from which the frames originated. The `switchport mode <port type>` context is used to change the manner in which traffic to/from certain ports are handled. Off the bat, 2 are important:    

 * `switchport mode dot1q-tunnel` - frames from the port will be tagged with a VLAN ID when it hits the trunk. If one port is set to this mode, it is assumed that all access ports are of this mode, regardless of whether you've touched them or not. 
 * `switchport mode trunk` - Sets the port as a trunk port. Frames hitting the port will be tagged with a 2-byte VLAN ID unless a native VLAN.

See section 2.3.3 for info on dot1q ports and trunk configuration.

=== 2.3.3. VLAN association === #assoc
Traffic from a port associated to a certain VLAN is only visible to other ports associated with the same VLAN (unless we're talking about trunk ports or using routers to bridge VLANs). The context `switchport access vlan <n>` is used to associate the port to a certain VLAN of ID <n>; See section [#toports 2.2.4] for an example.
[[BR]]   

[#pindex return to section index] [[BR]]
[#index return to main index] 
[[BR]]
=== 2.3.4. Configuring trunk ports === #Trunking
With "access mode" ports, VLANs with the same VLAN ID that are on different but connected switches would be recognized as different VLANs despite having the same VLAN ID. In order to allow the two same-ID VLANs to be treated as one, we need 1) ports set to `dot1q-tunneling` mode, and 2) trunk ports connecting the two switches.

1. ''Configure the trunk port.'' If ports 0/45-48 are to be trunk ports allowing the assage of frames from VLANs 3,27,and 28 (Remember, if you are just configuring one port, you can always use the singular "interface gigabitethernet" context.):
{{{
!(config-if)# interface range gi 0/45-48
!(config-if-range)# switchport mode trunk
!(config-if-range)# switchport trunk allowed vlan 3,27,28
}}}

 * "interface range gi 0/45-48" - configure all four ports at once with "interface range" context
 * "switchport mode trunk" - make the ports trunk ports
 * "switchport trunk allowed vlan 3,27,28" - allow the trunk to only carry frames from VLANs 3,27,or 28 (frames from these VLANs will get tagged)

'''Some extensions of the "switchport trunk allow" context."''' [[BR]]
As per the switch's help output:
{{{
sw-sb09(config-if-range)# switchport trunk allowed vlan ?
  <vlan id list>  <List style>. 1 to 4094. ex. "10-20", "30,40"
  add             add the VLANs to use this port
  remove          remove the VLANs to use these port
}}}

Summarized:
 * "switchport trunk...vlan add <VLAN IDs> - add VLANs to your current "allowed VLANs" list. 
 * "switchport trunk...vlan remove <VLAN IDs> - remove VLANs from your current "allowed VLANs" list     
 
2. ''Associate the trunk with a Native VLAN.'' Native VLANs allows hosts that do not understand VLAN tagging to function on a network implementing trunking (IEEE 802.1q). In context of the switch, the native VLAN is much like a trunk port version of an interface VLAN; It essentially assigns an IP address and VLAN to the switch's trunk port(s), and provides a means to transport non-tagged frames on the trunk.         
{{{
!(config-if-range)# switchport trunk native vlan 1
}}}

 * "switchport trunk native vlan 1" - make VLAN 1 the native VLAN for trunk port 0/48 (any frames from VLAN 1 will pass through the trunk untagged) 

By the way, this is the configuration for interface VLAN 1, which should be done ahead of time:
{{{
(config-vlan)# interface vlan 1
(config-if)# ip address 172.16.19.253 255.255.0.0
}}}

If you now go into a context for one of your trunk ports and do a "show", your configurations should look like this:
{{{
!(config-if-range)# interface gi 0/48
!(config-if)# show
interface gigabitethernet 0/48
  switchport mode trunk
  switchport trunk allowed vlan 1,3,27-28
  switchport trunk native vlan 1
}}} 

But finally...

3. '''Configure the ports associated with the trunked VLANs.''' The steps are same as configuring the ports for regular VLANs, except instead of "switchport mode access" we specify "switchport mode dot1q-tunnel" in this case:
{{{
(config)# int ran gi 0/13-22
(config-if-range)# sw mo dot1q-tunnel 
!(config-if-range)# sw acc vlan 3
!(config-if-range)# save             <<<-----VERY important if you want the configurations saved!
}}} 

[#pindex return to section index] [[BR]]
[#index return to main index] 
[[BR]]
=== 2.3.5 port speed === #speed
By default, the switch autonegociates. You can change this by going into configuration mode for each port, and changing settings. 

1. ''Shut port(s) to configure down.'' Shutting the ports down before configuring them allows the changes to take immediate effect upon bringing them back up. 
{{{  
sw-sb09(config)#  interface gi 0/13
sw-sb09(config-if)# shutdown
!sw-sb09(config-if)# 
02/06 03:35:48 E3 PORT GigabitEthernet0/13 25011100 1350:000000000000 Port disabled administratively.
}}}

It is normal to see an output message when you take down ports. 

2. ''Set speed.'' The sub-context "speed" allows you to configure the port speed. The syntax is the following:

 * `speed auto [10|100|1000]` - autonegociate. If a speed is specified, the port will be shut down by the switch if the host refuses to negotiate to the set speed. 
 * `speed [10|100|1000]` - force port to run at a specific speed, no autonegociation

Where the numbers denote Mbps. Here, we configure the port to go at 1000Mbps (1Gbps), and bring it back up:
{{{
!sw-sb09(config-if)# speed auto 1000   
!sw-sb09(config-if)# no shutdown      
}}}

[#pindex return to section index] [[BR]]
[#index return to main index] 

[[BR]][[BR]]
== 2.4 Telnet == #Telnet 
This section covers the steps needed to set up telnet access to your switch. Replace the VLAN ID's, IP addresses, and other parameters to fit your deployment.  
 
 1. Create a VLAN for telnet via the "vlan" context. For clarity (and good practice), we name our VLAN "telnet" so we know what it's used for. 
{{{
(config)# vlan 4
!(config-vlan)# name "telnet"
}}} 
 2. Configure IP for the telnet VLAN's VLAN interface.     
{{{
!(config-vlan)# interface vlan 4
!(config-if)# ip address 192.168.10.1 255.255.255.0
}}}
 3. Enable telnet logins to the switch using the "line vty" context. The numbers following the context denote the range of virtual terminals that the users have access to for telnet. Here we set the values to "0 2", meaning that 3 users can simultaneously telnet to the switch.   
{{{
!(config-if)# line vty 0 2
!(config-line)# exit
!(config)#   
}}}

'''Using Telnet''' [[BR]]
Once you've set up telnet on your switch, you can access the switch with the IP address you've assigned it during step 2 above. So for this config you'd issue the following command from the terminal:
{{{
$ telnet 192.168.10.1
}}} 

The username and password are the same for this prompt once you connect. 

[#index return to index] 

[[BR]][[BR]]
----
= III Miscellaneous Configuration Tasks = #III
This section covers some odds-and-ends that you may or may not need in terms of basic switch deployment, and that didn't really fit nicely with the past two sections. These bits tend to be a bit 'gorier' in terms of switch administration, and may require stuff covered in the past sections.  
[[BR]][[BR]]
== 3.1 ACLs and SNMP == #acls
=== Index === #acindex  
The following subsections are found here:
 [#acback 3.1.1. Background] [[BR]] 
 [#aconf 3.1.2. Configuring ACLs] [[BR]]
 [#snmpconf 3.1.3. Configuring SNMP] [[BR]]
 [#snmpg 3.1.4. Accessing SNMP data] [[BR]]

=== 3.1.1. Background === #acback
Access Control Lists (ACLs) dictate who and what can access various services provided by the device. One such service is Simple Network Management Protocol (SNMP), which allows you to look up and modify device information across the network. So - here we cut to the chase and cover both at the same time. Firstly, some references for SNMP: 
 * SNMP, courtesy of Wikipedia: http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol [[BR]]
 * MIB references: http://www.alvestrand.no/objectid/

=== 3.1.2. Configuring the ACL === #aconf
Use of SNMP (Simple Network Management Protocol) is not set by default; you must create access lists with IP addresses allowed to request information as a manager device. ACls are configured through the `access-list` context. The (basic) syntax is the following:

`access-list <lid> <seq> <permit|deny> <inet|any> <netmask>` 

where
 * lid -  list ID, a value between 1 - 2699
 * seq -  sequence number, a value between 1 - 4294967294 (can be omitted) 
 * inet - IP address of host or network address
 * netmask - subnet mask, 0.0.0.0 for hosts

`no access-list <lid>` deletes all entries with a particular list ID. 

'''example'''[[BR]]
{{{
sw-sb09-da(config)# access-list 1 permit 10.19.0.10 0.0.0.0                           
!sw-sb09-da(config)# access-list 1 permit 192.168.100.28 0.0.0.0                       
!sw-sb09-da(config)# access-list 2 permit 172.16.0.1 255.255.0.0                       
}}}
the resulting configs can be seen with `show access-list`:
{{{
!sw-sb09-da(config)# sh access-list                                                    
access-list 1 10 permit host 10.19.0.10                                                
access-list 1 20 permit host 192.168.100.28                                            
access-list 2 10 permit 172.16.0.1 255.255.0.0                       
}}}
doing a "no access-list 1" will remove the first two entries of this ACL. 

[[BR]]
=== 3.1.2. Configuring SNMP === #snmpconf
Now that you have an ACL, you can specify which ACL entries may have access to SNMP services. This is done by specifying ACL entries by list ID in the `snmp-server community` context. 
{{{
sw-sb09(config)# snmp-server community "NET" rw 1
}}}
here, "NET" is an arbitrary group name, `rw` means any member from the allowed group can read or modify SNMP parameters (as opposed to "ro", read-only), and "1" is the list ID of the group. 
[[BR]]
[#acindex return to section index] [[BR]]
[#index return to main index] 
[[BR]]
=== 3.1.3. Accessing SNMP data === #snmpg 
SNMP uses Management Information Bases (MIBs) to organize the data it can provide and modify on a device. SNMP data lookup is usually done by specifying the MIB values for a certain spec. The `snmp` context provides a set of tools for this - for a list of options, type "snmp ?". 

'''example''' [[BR]] 
here we do a system information query on the switch we are logged into:
{{{
sw-sb09# snmp walk 1.3.6.1.2.1.1            

Name: sysDescr.0
Value: ALAXALA AX3640S AX-3640-48T2XW-L [AX3640S-48T2XW] Switching software Ver. 
10.7 [OS-L3L]

Name: sysObjectID.0
Value: ax3640s

Name: sysUpTime.0
Value: 25724205

Name: sysContact.0
Value: 

Name: sysName.0
Value: sw-sb09

Name: sysLocation.0
Value: 

Name: sysServices.0
Value: 78
}}}
'''using snmpwalk''' [[BR]]
This is an utility for *nix machines, and can be used to prod the switch over the network, given you have the ACLs set up properly. 
[[BR]][[BR]]
[#acindex return to section index] [[BR]]
[#index return to main index] 

[[BR]][[BR]]
== 3.2 DHCP Relay Agent == #dhClient
A DHCP relay agent forwards DHCP messages between DHCP client and server residing on two different subnets and VLANs. This is a translation from the online NEC guide. 

1. configure client VLAN. This is the VLAN on which the DHCP clients live.
{{{
(config)# vlan 2
(config-vlan)# exit
(config)# interface gigabitethernet 0/5
(config-if)# switchport mode access
(config-if)# switchport access vlan 2
(config-if)# exit
(config)# interface vlan 2
(config-if)# ip address 10.1.0.1 255.255.0.0
(config-if)# exit
}}}

2. Next, set up the VLAN and interface connected to the DHCP server. 
{{{
(config)# vlan 3
(config-vlan)# exit
(config)# interface gigabitethernet 0/7
(config-if)# switchport mode access
(config-if)# switchport access vlan 3
(config-if)# exit
(config)# interface vlan 3
(config-if)# ip address 20.1.0.1 255.255.0.0
(config-if)# exit
}}}

3. set the DHCP server's address as the helper address on client VLAN. 
{{{
(config)# interface vlan 2
(config-if)# ip helper-address 20.1.0.10
(config-if)# exit
}}}

[#index return to main index] 

[[BR]][[BR]]
== 3.3 DHCP Server == #dhServer
The IP8800 comes with DHCP server capabilities. DHCP on the switch is provided on a VLAN basis, where the service is provided to hosts based on VLAN membership. DHCP configs for the basic DHCP server use the `service dhcp` and `ip dhcp` contexts.  

1. enable DHCP services for VLANs. Here, we want VLAN 1 to have the service. 
{{{
(config)# service dhcp vlan 1
}}}

2. specify address(es) excluded from address block. Here, IP addresses between 10.1 and 11.10 will not be handed out to clients.    
{{{
!(config)# ip dhcp excluded-address 192.168.10.1 192.168.11.10 
}}}

3. configure client pool. Members of VLAN 1 will be part of pool "Test1", be given IP's from the 192.168.11.0/24 block, and will have 10-minute leases. 
{{{
!(config)# ip dhcp pool Test1
!(dhcp-config)# network 192.168.11.0 255.255.255.0
!(dhcp-config)# lease 0 0 10 
}}}

[[BR]]
[#index return to main index] 

[[BR]][[BR]]
== 3.4 Port Mirroring == #mirror
Say that you want to monitor the traffic to/from a certain port. The switch can act as a network tap if you configure port mirroring. When configured to port mirror, the switch would send a copy of the frames it receives from certain ports to another port where you'd, say, connect a network analyzer. 

We needed to set up port mirroring to diagnose what was going wrong with the L2TP tunneling between the netFPGA hosts. 

'''Syntax'''[[BR]]
{{{
monitor session <session no.> source interface gigabitethernet <src port> <tx|rx|both> destination interface gigabitethernet <mirror port>
}}}

 * <session no.> - a number between 1-4 uniquely identifying the mirroring session. the switch can do a maximum of 4 sessions at once.
 * <src port> - the port number(s) of the port(s) you're trying to monitor. Can be a range of ports e.g. `0/5-6,27`.
 * <tx|rx|both> - mirror incoming, outgoing, or both traffics, respectively. "tx" and "rx" are with respect t the host connected to the port.
 * <mirror port> - this is the port connected to the analyzer.   

'''Errors'''[[BR]]
Rebooting the switch without `reload stop` sometimes corrupts the monitor configs. When this happens the monitor session will show up in your configurations but will not actually be there; Trying to delete it will result in this error:

{{{
(config)# no monitor session 1
monitor : Can not delete it because data is not corresponding.
}}}

Instantiating the same session again should fix things should this happen. [[BR]]
     
[#index return to main index] 
[[BR]][[BR]]
== 3.5 Users and Passwords == #accounts
'''adding a user''' [[BR]]
This is done with the `adduser` command as with any *nix machine (The new user is added to /etc/passwd as you expect). You need to be in enabled mode.
{{{
sw-sb09-da# adduser native no-flash                                                   
User(empty password) add done. Please setting password.    <<---this means "user added. please set password".                             
                                                                                       
Changing local password for native.                                                    
New password:                                                                          
Retype new password:  
!sw-sb09-da#
}}}

'''deleting a user'''[[BR]]
This is not done with deluser, but with `rmuser`. 

'''setting/clearing passwords'''[[BR]]
`password` <username> will allow you to reset password. `clear password <username>` allows you to delete passwords. replacing <username> with "enable-mode" allows you to set the password for user admin (enabled mode).   

'''Forgotten admin password''' [[BR]]
There is no recovery process akin to Cisco switches per se - pressing the reset button on the switch for 5 seconds will "default reset" the device. Once the switch reboots, you will be able to set new passwords - Default reset just clears the administration password, but will not wipe out your configurations (although the descriptions make it sounds like it does)   
 
[#index return to main index] 
[[BR]][[BR]]
----
side notes...IOS, disabling STP
{{{
Switch(config)#no spanning-tree vlan 1-1005                                     
Switch(config)#exit                                                             
Switch#                                                                         
1w4d: %SYS-5-CONFIG_I: Configured from console by console                       
Switch#sh spann                                                                 
Switch#sh spanning-tree                                                         
                                                                                
No spanning tree instance exists.       
}}}                                        
----
= IV !OpenFlow Switching = #IV
This section goes over the !OpenFlow aspects of the switch.  
[[BR]]

== 4.1 What is !OpenFlow? == #intro
!OpenFlow is a virtualization technology developed by Stanford. Ideally, it allows administrators and researchers to configure the switch's behavior freely, without constraints of TCP/IP or the native firmware. This is achieved by removing traffic switching decisions off of the switch and onto an external, highly configurable controller. More on !OpenFlow can be found at http://www.openflowswitch.org.  

The switch supports !OpenFlow in the form of virtual switches, or "datapaths" that overlay VLAN configurations; Each separate VLAN residing on the same device can be instantiated as a virtual switch that can be controlled by separate controllers and/or scripts, or be kept as a traditional VLAN that does commodity switching.   

[[BR]]
[#index return to main index]     
[[BR]]
[[BR]]
== 4.2 Outline == #outline
In order to make the switch !OpenFlow capable, you need to flash the switch with an SD card containing specialized firmware. The rough outline of the steps are the following:
 
 1. disable Spanning Tree (STP) 
 1. configure VLANs and ports, telnet
 1. copy firmware and license to SD card
 1. add openflow.conf to SD card
 1. reboot switch with SD card
 1. configure virtual switches  

Since it'd be easier to describe what's going on, this process will be using the configurations for [http://orbit-lab.org/wiki/Internal/OpenFlow/OrbitSwitches/sw-sb-combined sw-sb-01] as an example. You should take a bit of time to graze over the configs for the switch before going on.   
[[BR]]
[[BR]]
== 4.3 Configuring the switch == #ofconf
This section corresponds to steps 1 and 2 in the Outline. 

'''''1. Disable Spanning Tree (STP)'''''. STP compromises both !OpenFlow and ORBIT nodehandler/nodeagent functionalities, so both STP and Per-VLAN Spanning Tree (PVST) must be shut down using the `spanning-tree` context from the (config)# prompt. 
{{{
spanning-tree disable
no spanning-tree vlan 1,3,11-14,19-26
}}}
The first line disables STP. The second line disables PVST for each VLAN used. 
[[BR]][[BR]]
'''''2. configure VLANs and ports, telnet'''''. Because virtual switches are based on VLANs, all port and VLAN configs should be done before flashing. This part of the setup is just normal configuration using `vlan`, `interface vlan`, and `interface gigabitethernet` contexts. 

[[BR]]
[#index return to main index]  
[[BR]]
[[BR]]
== 4.4 Preparing the SD card == #sdprep
When flashing, the switch is booted off of an SD card containing the firmware, license, and a configuration file. Each switch should have come with an SD card; if not (or if the SD card is shoddy), it's ok to use another 128MB card. Anything larger may be affected by a known bug that causes filesystem corruption (see 'errors' section below ).  
 
This section is comprised of steps 3 and 4. Technically, it's better explained as three steps: 

'''''1. Copy license to SD card'''''. The switch will refuse to boot from an SD card without the proper license file. Each license file should be named after a particular switch's serial ID, which can be found on a silver stcker on the lower righthand corner on the front of the device, or through the `show ver` command:
{{{
> sh ver                                                                     
Date 2010/07/26 15:42:50 UTC                                                           
Model: AX3640S-48T2XW                                                                  
S/W: OS-L3L Ver. 11.1.C                                                                
H/W: Main board                                                                        
       AX-3640-48T2XWE-L [WA13CL48T2XVS400E05A005:80330300:307:1B514-1B512]            
     Module slot1 PS-M(AC)                                                             
       AX-F2430-PSA01 [WA0PSA010000C110005A005]                                        
     Module slot2 FAN-M                                                                
       AX-F2430-FAN01 [WA2FAN010000C140005A005]                                        
}}}   
The string in square brackets right underneath "H/W: Main board" is the full serial number of the switch. 

In this case, the proper license for this switch would be named 'WA13CL48T2XVS400E05A005.dat'. This should be renamed to 'license.dat' before being saved to the card.   

'''''2. Copy firmware image file to card'''''. The firmware is named 'k.img'. This doesn't need to be renamed. 

'''''3. Create openflow.conf'''''. Create a new file named 'openflow.conf' and save it to the card.    

With the three files saved, the card is ready for use.
[[BR]]
== 4.5 Flashing the switch == #flash
The switch has an SD slot next to the serial port. Slap the card in and reboot the switch with a `reload` as user admin.  
{{{
sw-sb09-da# reload
Dump information extracted? (y/n): y
old dump file(rmdump  07/21 21:16) delete OK? (y/n): y
}}}
If connected by serial, you should be able to see the following startup sequence, which should take about 2 minutes:
{{{
sw-sb09-da# 
ROM 00.03.52
.......................................................................................

BOOT 00.03.00
Loading from dev2 100%


login: 
}}}

'Loading from dev2' indicates that the switch has successfully found, and is booting off of, the firmware on the SD card. 

=== errors ===
If you forget the license, or use the wrong one (e.g. one whose name doesn't match the serial ID of the switch), The switch will restart in the ROM prompt. 

The switch may also boot with the default image if the image on the SD card is corrupt. This is usually indicated by the message "Loading from dev0" and not recognizing [#ofcom OpenFlow specific commands]. 

The worst case scenario is a corrupted SD card. This may be caused by various reasons, including buggy firmware, improper restart, or using a card over 128MB. Symptoms include:

 * the device not starting up/loading the default image
 * scrambled/truncated filenames (e.g. 'openflow.conf' showing up as 'openf~l.con') 
 * inability to create virtual switches

The most surefire way to fix this is to reformat the SD card. The easiest way to do it is to use a Windows machine. When the card shows up under "Devices", right click and go to 'Format'. A window saying "Format Secure Digital Storage Device" should pop up. Set "Allocation Unit size" to "default allocation size", and click 'Start'. You may need to confirm that you want to reformat the card. The card should be good to go in about a minute or so. 

[[BR]]
[#index return to main index]      
[[BR]]
[[BR]]

== 4.6 OpenFLow Commands == #ofcom
=== Index === #ofindex
these are the sub-sections to this section:
 [#ofback 4.6.1. Background] [[BR]] 
 [#setv 4.6.2. Virtual switch instantiation] [[BR]]
 [#delvsi 4.6.3. Virtual switch deletion] [[BR]]
 [#ctvsi 4.6.4. Manipulating virtual switches] [[BR]]
 [#ofmon 4.6.5. Monitoring OpenFlow stats] [[BR]]
 [#others 4.6.6. other commands] [[BR]]

The !OpenFlow capable firmware supports a suite of commands that can be used to configure !OpenFlow datapaths. These commands are located in /usr/local/bin/; try greping (is that a word?) for "vsi", or "port" for a list of commands. 

Once the switch is booted up, we can move to the final step of actually instantiating virtual switches using these commands.

[[BR]]
=== 4.6.2. Virtual switch instantiation === #setv

'''setvsi''' [[BR]]

sw-sb-01's data VLAns are all !OpenFlow virtual switches instantiated using the command `setvsi`. `setvsi` takes on the following minimal syntax:
{{{
setvsi <VLAN ID> <ports> tcp <IP addr:port> dpid <datapath ID>
}}}  

 * <VLAN ID> - the VLAN ID of the VLAN you want to overlay a virtual switch onto. Otherwise called the "VSI number"
 * <ports> - the ports, without the '0/', that are going to be part of the virtual switch
 * <IP addr:port> - IP address of the !OpenFlow controller you want the virtual switch to listen to, and port number (default is 6633)
 * <datapath ID> - 12-digit hex value defining the MAC address for the virtual switch. Always begins with '0x'

a full synopsis can be found by typing "setvsi" at the CLI. 

setvsi basically echos the whole command into openflow.conf. This is handy since you can follow the exact commands used to instantiate the virtual switches on a particular switch by simply doing a `cat /mnt/openflow.conf`. 

''' Some important things about VSI's... ''' [[BR]]
 * the VSI ID must match the VLAN ID.
 * the list of ports must include all ports associated to that VLAN ID.
 * VSI's cannot be instantiated on a native VLAN. 
 * If editing openflow.conf with a text editor, newline should be in UNIX convention (LF only).  

'''implementation''' [[BR]]
Here's sw-sb-01's conf file: 
{{{
sw-sb-01> cat /mnt/openflow.conf
setvsi 12 31,32,48.12 tcp 172.16.0.4:6633 dpid 0x001010123132
setvsi 14 33,34,48.14 tcp 172.16.0.4:6633 dpid 0x001010143134
setvsi 20 35,36,48.20 tcp 172.16.0.4:6633 dpid 0x001010203230
setvsi 22 37,38,48.22 tcp 172.16.0.4:6633 dpid 0x001010223232
setvsi 24 39,40,48.24 tcp 172.16.0.4:6633 dpid 0x001010243234
setvsi 26 41,42,48.26 tcp 172.16.0.4:6633 dpid 0x001010263236
}}}

Each line is the exact command that was typed at the prompt. As you can see, sw-sb-01 has 6 virtual switches overlaying VLANs 12,14,20,22,24,and 26. All of them are pointed towards IP address 172.16.0.4, the ORBIT !OpenFlow controller.  

'''ORBIT DPID conventions''' [[BR]]  
There are a few home-brew conventions that are used for the dpid's:
 * first 3 bytes are always '001010' 
 * the 4th byte is the VLAN ID, in decimal
 * the last two bytes are the VLAN ID in ascii hex notation. ASCII maps decimal 0-9 to hex 30-39.  

So, for example, if we have VLAN 12, the last 3 bytes will be '12', '31', and '32'. That's the dpid from the first setvsi line in the terminal output above. 

'''trunk ports''' [[BR]]
If you look at the list of ports from the conf file above, you'll notice that all of them end with "48.n", n being the VLAN ID for the virtual switch being instantiated. This value lets the switch know which VLAN ID to tag the traffic to/from a particular datapath with on a particular trunk port. For example, "48.12" means "tag virtual switch 12's traffic with VLAN ID 12 on trunk port 0/48." This is needed if you are using trunking the switch. 

[[BR]]
[#ofindex return to section index] [[BR]]
[#index return to main index] 
[[BR]]
=== 4.6.3. Virtual switch deletion === #delvsi
`deletevsi` deletes a certain virtual switch instantiation. The syntax is as follows:
{{{
deletevsi <VLAN ID>
}}}

nice and simple. 

[[BR]]
=== 4.6.4. Manipulating virtual switches === #ctvsi
You don't want to have to keep deleting and redoing virtual switches just to add or remove ports from an instantiation. 
 * `addport <VLAN ID> <port>` - add ports to virtual switch for VLAN of <VLAN ID>
 * `deleteport <VLAN ID> <port>` - delete ports from virtual switch for VLAN of <VLAN ID>

Nor do you want to keep setvsi-ing them because you didn't want a virtual switch one moment, and then you wanted it the next.
 * `disablevsi <VLAN ID>` - temporarily disable the virtual switch. As far as the switch is concerned, it's not there when it's disabled.
 * `enablevsi <VLAN ID>` - enable a disabled virtual switch. 

[[BR]]
[#ofindex return to section index] [[BR]]
[#index return to main index] 
[[BR]]
=== 4.6.5. Monitoring !OpenFlow stats === #ofmon
There are a few commands to pull stats about flows, virtual switches, ect. 
 * `showswitch <VLAN ID|cr> <detail|cr>` - displays virtual switch information. VLAN ID and "detail" are optional parameters. 
 * `showflow <VLAN ID|cr> <detail|cr>` - display current flows. Same deal with parameters. 
 * `showofinfo` - display all !OpenFlow and system (akin to doing a dmesg and sysctl -a back-to-back) stats

'''example''' [[BR]]
Displaying all datapaths:
{{{
sw-sb-01> showswitch
vlan    ports                                           secure channel      
----    -----                                           --------------      
12      31, 32, 48.12                                   connected           
14      33, 34, 48.14                                   connected           
20      35, 36, 48.20                                   connected           
22      37, 38, 48.22                                   connected           
24      39, 40, 48.24                                   connected           
26      41, 42, 48.26                                   connected           
}}}
Here you can see that all of the virtual switches are registered ("connected") with the controller. 
Now, if you want to focus on one virtual switch, say VLAN 12:
{{{
sw-sb-01> showswitch 12 detail
Virtual switch 12
  Datapath ID  : 68989104434(0x1010123132)
  Datapath name: sw-sb-01
  Port         : gigabitethernet 0/31 (link down)                                
                 gigabitethernet 0/32 (link down)                                
                 gigabitethernet 0/48.12 (link up, 1G full)                      
  Conn mode    : tcp              
  Controller   : 172.16.0.4:6633 (connected)
  Exact match  : 01-24, 49-50 hw 0 / hw max 3046
                 25-48        hw 0 / hw max 3046
  Exact match  : sw 0 / sw max 131072
  Wildcard     : sw 1 / sw max 100
  Emerg        : sw 0 / sw max 100
  Packet buff  : 256 packets / 4294967295 MB                                           
  Miss sendlen : 128 bytes                                                             
  Wcard mode   : S/W only                                                              
  Counter type : Packet Counter                                                        
  MAC learning : On                                                                    
  Min EQ       : 0    
  Max EQ       : 7                                                                     
  Default EQ   : 4                                                                     
  VSI Enable   : yes                                                      
}}}    

=== 4.6.6. other commands === #others
I really haven't used the other commands e.g, for queue manipulation, so I can't really describe them. But here's the list of what I believe is all the commands for !OpenFlow datapath manipulation:
{{{
-r-xr-xr-x  1 root  wheel   221139 Jun  9 05:20 addport                                
-r-xr-xr-x  1 root  wheel   211566 Jun  9 05:20 deleteflow                             
-r-xr-xr-x  1 root  wheel   221162 Jun  9 05:20 deleteport                             
-r-xr-xr-x  1 root  wheel   202225 Jun  9 05:20 deletevsi                              
-r-xr-xr-x  1 root  wheel   202579 Jun  9 05:20 disablevsi                             
-r-xr-xr-x  1 root  wheel   202513 Jun  9 05:20 enablevsi                              
-r-xr-xr-x  1 root  wheel   521925 Jun  9 05:20 max-queue-rate                         
-r-xr-xr-x  1 root  wheel      436 Jun  9 05:20 ofconf_mng.sh                          
-r-xr-xr-x  1 root  wheel   202688 Jun  9 05:20 settrace                               
-r-xr-xr-x  1 root  wheel   231122 Jun  9 05:20 setvsi                                 
-r-xr-xr-x  1 root  wheel   229570 Jun  9 05:20 showflow                               
-r-xr-xr-x  1 root  wheel     2146 Jun  9 05:20 showofinfo                             
-r-xr-xr-x  1 root  wheel   232831 Jun  9 05:20 showswitch                 
}}}


[[BR]]
[#ofindex return to section index] [[BR]]
[#index return to main index] 
===== -- ===== #bottom