Exercise 2: Measuring Performance of a MobilityFirst Router

Objective

In this exercise, we will try to drive synthetic traffic through the router and measure key performance characteristics such as throughput and forwarding latency. Since MobilityFirst presents a hop-by-hop block data transport, we can vary the unit size of the data block and observe it's impact on the performance. We will also try to visualize the performance results using OMF's result service.

Deploy the Network

We will assume the network described and initialized in Exercise 1 is up and functional. Instead of the mfping application that was run manually, we will extend the OMF script to add the mfperf application that will drive the performance measurement of the router.

The entire script is available as part of the tutorial package as orbit/tutorial/scripts/exercise2.rb

The key extensions over previous script are briefly discussed below:

Setting up the 'mfperf Application'

The following snippet from the script shows the code added to set up the mfperf application and its arguments:

defApplication('test:app:mf:mfperf', 'mfperf') do |a|

  a.path = "/usr/local/bin/mfperf" 
  app.appPackage = "http://mobilityfirst.winlab.rutgers.edu/mf-orbit-tutorial.tar" 
  a.version(0, 9, 1)
  a.shortDescription = "MF protocol performance benchmark tool" 
  a.description = "This is targeted to be similar to the iperf benchmarking tool available for IP protocol. It generates MobilityFirst block packet traffic via the MF socket API and can be used to benchmark performance of MF routers and protocol stack implementations." 

  # Define the properties that can be configured for this application
  # 
  # syntax: defProperty(name = :mandatory, description = nil, parameter = nil, options = {})
  #
  a.defProperty('generator', 'Type of packet generator to use (cbr)', '-g', {:type => :string, :dynamic => false})
  a.defProperty('dst_GUID', 'GUID of the Destination', '--dst_GUID', {:type => :string, :dynamic => false})
  a.defProperty('my_GUID', 'GUID of this Source application', '--my_GUID', {:type => :string, :dynamic => false})
  a.defProperty("cbr:size", "Size of data block [bytes]", '--cbr:size', {:dynamic => true, :type => :integer})
  a.defProperty("cbr:rate", "Data rate of the flow [kbps]", '--cbr:rate', {:dynamic => true, :type => :integer})

  # Define the Measurement Points and associated metrics that are available for this application
  #
  a.defMeasurement('msg_out') do |m|
    m.defMetric('ts',:float)
    m.defMetric('msg_no',:long)
    m.defMetric('msg_length',:long)
    m.defMetric('dst_GUID',:string)
  end
end
}

As seen above, the configuration also covers the set up of OML measurement points that we will use to track and visualize the forwarding performance of MFRs.

Running the Benchmark Application

The following snippet shows how the exercise runs the mfperf application and also changes the block size dynamically

Visualizing the Performance Data

Method 1: OMF framework supports a result service that allows experimenters to query data stored using the OML measurement framework. The query is performed over the web and requires the you know the hostname and port where the result service runs, and the experiment ID associated with this experiment - this is obtained from the output following the execution of the control script.

The result service supports either dumping of the entire database or a SQL-like querying option to selectively retrieve required measurement data. The below HTTP request shows an example query to retrieve certain fields of our above defined measurement point in the mfperf application:

wget "http://<hostname>:<port>/result/queryDatabase?expID=testing_slice-2010-09-03t09.41.43+10.00&format=csv&query=select ts,msg_no,msg_length from msg_out"  -O msg_out.csv

Note that the URL used in wget, in particular the arguments, may require to be encoded to unambiguously represent special characters when using the HTTP protocol.

The downloaded data can now be easily visualized using a tool such as gnuplot. You can find a helper script in the tutorial package that plots they key performance data downloaded using the above query.

Method 2: Alternatively, the performance data may also be visualized using omf-web, OMF's web-based visualization service. It also works in concert with the result service referenced in Method 1, and makes available a variety of graph widgets to visualize live-experiment data logged using OML. Detailed documentation on the installation and usage of omf-web can be found on the ​omf-web github site.

Since this is installed on all ORBIT domains, we will only concern ourselves with defining the widget configuration required to bring up the live graphs for the performance data we are logging. Below is the contents of the simple two widget configuration file available with this tutorial package:

title: MobilityFirst Data Transfer Performance

# Port number the omf-web service should run at
port: 4041

# Root url of AM result2 service
result2_server: http://oml:5054

# Define tabs, widgets for visualisation below
#
tabs:
  # Data transfer throughput
  mfperf_tput:
    # Line chart widget, need to define columns in mapping section.   
    - name: OML TS SERVER
      type: line_chart
      data: msg_out
      mapping:
        # x-axis, y-axis and group_by.
        x: ts
        y: kbytes_per_sec
        group_by: oml_sender_id
  mfperf_rtt:
    # Line chart widget, need to define columns in mapping section.   
    - name: OML TS SERVER
      type: line_chart
      data: msg_out
      mapping:
        # x-axis, y-axis and group_by.
        x: msg_length
        y: transfer_time
        group_by: oml_sender_id

To bring up the visualization, start the basic omf-web service with the configuration file argument:

omf-web-basic -c <config_yaml_file> <experiment id>