== Exercise 2: Measuring Performance of a MobilityFirst Router ==

=== Objective ===

In this exercise, we will try to drive synthetic traffic through the router and measure key performance characteristics such as throughput and forwarding latency. Since MobilityFirst presents a hop-by-hop block data transport, we can vary the unit size of the data block and observe it's impact on the performance. We will also try to visualize the performance results using OMF's result service by installing an OML-enabled monitor on the routers.

=== Deploy the Network ===

This tutorial assumes that a 4 nodes topology has been already established in one of the Orbit sandboxes or the grid:

{{{
#!sh
Host (GUID 101) ---- MFR1 (GUID 1) ---- MFR2 (GUID 2) ---- Host2 (GUID 102)
}}}

[[CollapsibleStart(If not coming from the previous exercise follow these instructions to setup your network)]]

Once logged into the grid console:

{{{
#!sh

ssh username@console.grid.orbit-lab.org
}}}

From the console we will start by loading the Mobilityfirst image into the four nodes that have been assigned to you:

{{{
#!sh

omf load -i 'mf-release-latest.ndz' -t system:topo:mf-groupX
}}}

'''system:topo:mf-groupX''' represents the group of 4 nodes and '''mf-groupX''' has to be replaced by the group id assigned to you.

For example, '''mf-group1''' will load the image on nodes 'node1-1,node1-2,node2-1,node2-2'

If the output of your console looks similar to:

{{{
#!sh

INFO exp:  ----------------------------- 
 INFO exp:  Imaging Process Done 
 INFO exp:  4 nodes successfully imaged - Topology saved in '/tmp/pxe_slice-2014-10-15t02.10.16.594-04.00-topo-success.rb'
 INFO exp:  ----------------------------- 
 INFO EXPERIMENT_DONE: Event triggered. Starting the associated tasks.
 INFO NodeHandler: 
 INFO NodeHandler: Shutting down experiment, please wait...
 INFO NodeHandler: 
 INFO NodeHandler: Shutdown flag is set - Turning Off the resources
 INFO run: Experiment pxe_slice-2014-10-15t02.10.16.594-04.00 finished after 1:50
}}}

your nodes have been imaged correctly.

=== Deploy Network ===

Software and experiment control in the ORBIT testbed can be automated greatly using the OMF framework. An OMF control script is written in Ruby and allows the experimenter to specify the set of nodes, their network configuration, to specify software components and arguments, and to control their execution on one or more nodes. We will use an OMF script to bring up 4 ORBIT nodes to host our topology, with corresponding software components.

We will first introduce the main details of the scripts that will be run and then will step to the execution itself.

==== Software Component Specification ====

The following snippet shows the specification of the MobilityFirst router along with the required arguments:

{{{
#!ruby

defApplication('MF-Router', 'router') {|app|
	app.shortDescription = "Click-based MobilityFirst Access Router"
	app.path = "/usr/local/src/mobilityfirst/eval/orbit/tutorial/scripts/ARWrapper.sh" 
	# click options
	app.defProperty('num_threads', 'number of threads', "-t",{:type => :integer, :mandatory => true, :default => 4, :order => 1})
	app.defProperty('ctrl_port', 'port for Click control socket', "-c",{:type => :integer, :order => 2})
	# click config file 
	app.defProperty('config_file', 'Click configuration file', "-C",{:type => :string,:mandatory=> true})
	# keyword parameters used in click config file
	app.defProperty('my_GUID', 'router GUID', "-m",{:type => :string, :mandatory => true})
	app.defProperty('topo_file', 'path to topology file', "-f",{:type => :string, :mandatory => true})
	app.defProperty('core_dev', 'core network interface', "-d",{:type => :string,:mandatory => true})
	app.defProperty('GNRS_server_ip', 'IP of local GNRS server', "-s",{:type => :string,:mandatory => true})
	app.defProperty('GNRS_server_port', 'Port of GNRS server', "-p",{:type => :string,:mandatory => true})
	app.defProperty('GNRS_listen_ip', 'IP to listen for GNRS response', "-i",{:type => :string,:default => "0.0.0.0"})
	app.defProperty('GNRS_listen_port', 'port to listen for GNRS response', "-P",{:type => :string,:default => "10001"})
	app.defProperty('edge_dev', 'edge network interface', "-D",{:type => :string,:mandatory => true})
	app.defProperty('edge_dev_ip', 'IP assigned to edge interface', "-I",{:type => :string,:mandatory => true})
}

defApplication('MF-GNRS', 'gnrs') {|app|
	app.shortDescription = "GNRS Server"
	app.path = "/usr/local/src/mobilityfirst/eval/orbit/tutorial/scripts/GNRSWrapper.sh" 
	app.defProperty('log4j_config_file', 'log 4j configuration file', "-d",{:type => :string, :order => 1})
	app.defProperty('jar_file', 'server jar file with all dependencies', "-j" ,{:type => :string, :mandatory=> true, :default => "/usr/local/src/mobilityfirst/gnrs/jserver/target/gnrs-server-1.0.0-SNAPSHOT-jar-with-dependencies.jar", :order => 2})
	app.defProperty('config_file', 'server configuration file', "-c",{:type => :string, :mandatory=> true, :order => 3})
}

#Enable OML reporting by default
defApplication('MF-HostStack', 'hoststack') {|app|
	app.shortDescription = "MF host network stack"
	app.path = "/usr/local/bin/mfstack" 
	app.defProperty('log_level', 'log level', nil,{:type => :string, :mandatory => true, :order => 1, :default => "-e"}) # default is 'error'
	app.defProperty('config_file', 'stack configuration file', nil,{:type => :string, :mandatory => true, :order => 2})
}
}}}

As seen above, the router is configured with both 'core' and 'edge' interfaces. The core interfaces connect routers towards the core of the network, while the edge interface enables hosts to connect and access the MobilityFirst network.

Also seen above is the GNRS service related arguments that specify which server (IP and port) the router should use for in-network name resolution purpose, both for sending requests and to receive responses. By default it will listen on all interfaces and port 10001.

==== Setting up the Software Node Groups ====

The following shows how the node groups for the routers are setup in the OMF control scripts. Node groups allows experimenters to use single statements to set configuration and execute commands across all nodes in the group.

{{{
#!ruby

for i in 1..num_routers
	defTopology("topo:router_#{i}") { |t|
		aNode = routersTopo.getNodeByIndex(i-1)
		t.addNode(aNode)
		info aNode, " assigned role of router with GUID: #{i}"
	}
  
	defGroup("router_#{i}", "topo:router_#{i}") {|node|
		node.addApplication('MF-Router') {|app|
			app.setProperty('num_threads', router_threads)
			app.setProperty('config_file', click_conf)
			app.setProperty('my_GUID', router_guid[i-1])
			app.setProperty('topo_file', rtr_topo_file)
			app.setProperty('core_dev', core_dev)
			app.setProperty('GNRS_server_ip', GNRS_server_ip)
			app.setProperty('GNRS_server_port', GNRS_server_port)
			app.setProperty('GNRS_listen_ip', "192.168.100.#{i}")
			app.setProperty('GNRS_listen_port', GNRS_listen_port)
			app.setProperty('edge_dev', edge_dev)
			app.setProperty('edge_dev_ip', router_ether_if_ip[i-1])
		}

	  #If is the first router add the GNRS
	  if i == 1
      aNode = routersTopo.getNodeByIndex(i-1)
      info aNode, " will also host the GNRS server"
		  node.addApplication('MF-GNRS') {|app|
		    app.setProperty('log4j_config_file', GNRS_log_file)
		    app.setProperty('jar_file', GNRS_jar_file)
		    app.setProperty('config_file', GNRS_conf_file)
		  }
	  end
	
	  node.net.e0.ip = "192.168.100.#{i}"
	  node.net.e0.netmask = '255.255.255.0'
    
    node.net.w0.mode = "adhoc"
    node.net.w0.type = 'g'
    node.net.w0.channel = "11"
    node.net.w0.essid = "SSID_group_#{i}"
    node.net.w0.ip = "192.168.#{i}.1"
	}
end

#Create host groups
for i in 1..num_hosts
	defTopology("topo:host_#{i}") { |t|
		aNode = hostsTopo.getNodeByIndex(i-1)
		t.addNode(aNode)
		info aNode, " assigned role of client with GUID: #{100 + i}"
	}
  
	defGroup("host_#{i}", "topo:host_#{i}") {|node|
		node.addApplication('MF-HostStack') {|app|
			app.setProperty('config_file', hoststack_conf_file[i-1])
			app.setProperty('log_level', log_level)
		}
	  
    #node.net.e0.ip = "192.168.#{i}.#{i+100}"
	  #node.net.e0.netmask = '255.255.255.0'
    
    node.net.w0.mode = "adhoc"
    node.net.w0.type = 'g'
    node.net.w0.channel = "11"
    node.net.w0.essid = "SSID_group_#{i}"
    node.net.w0.ip = "192.168.#{i}.2"
	}
end
}}}
As can be seen above, properties which were defined in the MF-Router, MF-GNRS and MF-HostStack applications have been set here. Moreover, node interfaces have been set up, and IP addresses have been assigned to them. As we discussed earlier the router is configured with both edge and core interfaces. The ethernet interface is used to connect to the core of the network, and the wireless interface is for connection to the clients. On the other side, the client is equipped with wifi interface to connect to the access router.


[[CollapsibleEnd]]

We will assume the network described and initialized in Exercise 1 is up and functional. Here also we will use mfping to send packets between the hosts. In addition to the deployment specified in exercise 1, we install OML-enabled statistics monitor for MobilityFirst routers. 

The entire script is available as part of the tutorial package as orbit/tutorial/scripts/exercise2.rb

The key extensions over previous script are briefly discussed below:

==== Setting up the 'OML enabled Monitor on Routers Application' ====

The following snippet from the script shows the code added to set up the OML enabled Monitor on Routers Application and its arguments:

{{{
#!ruby
defApplication("mf_click_monitor", "mf_click_monitor") do |app|
	app.shortDescription = "OML enabld statistics monitor for MobilityFirst Routers"
	app.path = "/usr/local/bin/mf_click_mon"
	app.defProperty('ctrl_port', 'Port for Click control socket', nil,{:type => :string, :mandatory => true, :order => 1})
	app.defProperty('self-id', 'OML ID', nil,{:type => :string, :mandatory => true, :order => 2})
	app.defProperty('oml-config-file', 'OML configuration file', "--oml-config",{:type => :string,:mandatory=> true})
	app.defProperty('oml-domain', 'OML domain name', "--oml-domain",{:type => :string,:mandatory=> true})
end

self_id = "MonitorID"
oml_config_file = "/usr/local/src/mobilityfirst/eval/orbit/tutorial/conf/click-oml-config.xml"
oml_domain = "#{Experiment.ID}"

defGroup("router_monitors", "router_universe") {|node|
  node.addApplication('mf_click_monitor') {|app|
    app.setProperty('ctrl_port', router_control_port)
    app.setProperty('self-id', self_id)
    app.setProperty('oml-config-file', oml_config_file)
    app.setProperty('oml-domain', oml_domain)
  }
}
}}}

As seen above, the configuration also covers the set up of OML measurement points that we will use to track and visualize the forwarding performance of MFRs.

==== Running the Benchmark Application ====

The following snippet shows how the exercise runs the mfperf application and also changes the block size dynamically

==== Visualizing the Performance Data ====

'''Method 1:''' OMF framework supports a result service that allows experimenters to query data stored using the OML measurement framework. The query is performed over the web and requires the you know the hostname and port where the result service runs, and the ''experiment ID'' associated with this experiment - this is obtained from the output following the execution of the control script.

The result service supports either dumping of the entire database or a SQL-like querying option to selectively retrieve required measurement data. The below HTTP request shows an example query to retrieve certain fields of our above defined measurement point in the mfperf application:

{{{
#!sh
wget "http://<hostname>:<port>/result/queryDatabase?expID=testing_slice-2010-09-03t09.41.43+10.00&format=csv&query=select ts,msg_no,msg_length from msg_out"  -O msg_out.csv
}}}

Note that the URL used in wget, in particular the arguments, may require to be encoded to unambiguously represent special characters when using the HTTP protocol.

The downloaded data can now be easily visualized using a tool such as gnuplot. You can find a helper script in the tutorial package that plots they key performance data downloaded using the above query.

'''Method 2:''' Alternatively, the performance data may also be visualized using ''omf-web'', OMF's web-based visualization service. It also works in concert with the result service referenced in Method 1, and makes available a variety of graph widgets to visualize live-experiment data logged using OML. Detailed documentation on the installation and usage of omf-web can be found on the [https://github.com/mytestbed/omf_web omf-web github site].

Since this is installed on all ORBIT domains, we will only concern ourselves with defining the widget configuration required to bring up the live graphs for the performance data we are logging. Below is the contents of the simple two widget configuration file available with this tutorial package:

{{{
#!yaml
title: MobilityFirst Data Transfer Performance

# Port number the omf-web service should run at
port: 4041

# Root url of AM result2 service
result2_server: http://oml:5054

# Define tabs, widgets for visualisation below
#
tabs:
  # Data transfer throughput
  mfperf_tput:
    # Line chart widget, need to define columns in mapping section.   
    - name: OML TS SERVER
      type: line_chart
      data: msg_out
      mapping:
        # x-axis, y-axis and group_by.
        x: ts
        y: kbytes_per_sec
        group_by: oml_sender_id
  mfperf_rtt:
    # Line chart widget, need to define columns in mapping section.   
    - name: OML TS SERVER
      type: line_chart
      data: msg_out
      mapping:
        # x-axis, y-axis and group_by.
        x: msg_length
        y: transfer_time
        group_by: oml_sender_id
}}}

To bring up the visualization, start the basic omf-web service with the configuration file argument:

{{{
#!sh
omf-web-basic -c <config_yaml_file> <experiment id>
}}}