Changes between Version 29 and Version 30 of Tutorials/oMF/tut1


Ignore:
Timestamp:
Oct 20, 2014, 9:33:39 PM (10 years ago)
Author:
wontoniii
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • Tutorials/oMF/tut1

    v29 v30  
    1 = Exercise 1: Simple !MobilityFirst Network Deployment and Test =
     1= Exercise 2: Measuring Performance of a !MobilityFirst Router =
    22
    33[[TOC(Tutorials/oMF*, depth=2)]]
    … …  
    88=== Objective ===
    99
    10 In this exercise we will establish a simple topology consisting of !MobilityFirst routers, hosts and applications, deploy the software components onto physical nodes, and run an end-to-end 'ping' application. At the end of the exercise, you should expect to acquire a general knowledge of the !MobilityFirst software components and how to deploy and configure them using the Orbit testbed.
     10In this exercise, we will try to drive synthetic traffic through the router and measure key performance characteristics such as throughput and forwarding latency. Since !MobilityFirst presents a hop-by-hop block data transport, we can vary the unit size of the data block and observe it's impact on the performance. We will also try to visualize the performance results using OMF's result service by installing an OML-enabled monitor on the routers.
    1111
    1212=== Pre-requisites ===
    … …  
    1616 * Some familiarity with the !MobilityFirst terminology.
    1717
    18 === Topology ===
    19 
    20 In this exercise we will use a simple linear topology consisting of two !MobilityFirst routers (MFR) that interconnect two hosts: Host1 will initiate a 'ping' communication and Host2 will respond to the ping request:
     18=== Deploy the Network ===
     19
     20This tutorial assumes that a 4 nodes topology has been already established in one of the Orbit sandboxes or the grid:
    2121
    2222[[Image(Tutorials/oMF:MFTurorialNetwork.png)]]
    2323
    24 === !MobilityFirst ORBIT Image ===
    25 
    26 The complete set of components from the latest release of the !MobilityFirst software is available as a compressed image within the ORBIT testbed for imaging nodes using the 'OMF' tool. The current pre-prepared image is built over Ubuntu 12.04 LTS distribution and will be moved to newer distributions as they become available and we have had a chance to test compatibility.
    27 
    28 A typical Orbit experiment requires the following six steps:
    29 
    30  1. [[CollapsibleStart(Create resource reservation)]][[Include(Documentation/Short/CreateRes)]][[CollapsibleEnd]]
    31 
    32  2. [[CollapsibleStart(Login into reserved domain: "ssh username@sb1.orbit-lab.org")]][[Include(Documentation/Short/Login)]][[CollapsibleEnd]]
    33 
    34  3. [[CollapsibleStart(Load an image on the nodes: "omf load -i baseline.ndz -t all")]] [[Include(Documentation/Short/LoadImage)]][[CollapsibleEnd]]
    35 
    36  4. [[CollapsibleStart(Turn the nodes on: "omf tell -a on -t all")]][[Include(Documentation/Short/TellOn)]][[CollapsibleEnd]]
    37 
    38  5. [[CollapsibleStart(Execute the experiment: "omf exec test:exp:tutorial:hello-world-wireless -- --res1 node1-1.sb1.orbit-lab.org --res2 node1-2.sb1.orbit-lab.org")]][[Include(Software/cOMF/aExec)]][[CollapsibleEnd]]
    39 
    40  6. Analyze the results
    41 
    42 While, most of the experiments follow the presented structure, for this specific tutorial some simplifications have been applied.
    43 
    44 From now on, the following assumptions are considered:
    45 
    46  * You will be working with resources belonging to the Orbid grid.
    47  * You have been assigned a group number, username and password.
    48 
    49 While for this experiment we are using the grid, it is not a strict a requirement and for the successful execution of the experiment any sandbox with at least 4 nodes could be employed (i.e. sb4, sb9, etc.).
     24If not coming from [wiki:Tutorials/oMF/tut1 exercise 1] follow these instructions on how to setup the topology. Running exercise 1 at least once before moving to exercise 2 is advised to understand the steps and software components involved.
     25
     26[[CollapsibleStart(4 nodes topology setup)]]
    5027
    5128First of all, log in into the grid console using SSH:
    … …  
    237214Moreover, resources such node interfaces and their corresponding IP addresses have to be set up in this phase of the experiment. As we discussed earlier the router is configured with both edge and core interfaces. An ethernet interface is used to connect to 2 core routers, while a wireless interface is used to provide access for the clients.
    238215
     216[[CollapsibleEnd]]
     217
     218==== Setting up the 'OML enabled Monitor on Routers Application' ====
     219
     220At this point, the network topology described and initialized in Exercise 1 is supposed to be ready and functional. In order to produce synthetic traffic, we will use mfping to send packets between the end hosts. In order to perform more advanced network measurements, other applications are also available, such as a modified version of the commonly used application ''iperf''. As per the goal of the exercise, we will use an OML-enabled statistics monitor for !MobilityFirst routers in order to collect usage statistics on the nodes.
     221
     222The key extensions over exercise 1's script are briefly discussed below. The following snippet from the script shows the code added to set up the OML enabled Monitor on Routers Application and its arguments:
     223
     224{{{
     225#!ruby
     226defApplication("mf_click_monitor", "mf_click_monitor") do |app|
     227        app.shortDescription = "OML enabld statistics monitor for MobilityFirst Routers"
     228        app.path = "/usr/local/bin/mf_click_mon"
     229        app.defProperty('ctrl_port', 'Port for Click control socket', nil,{:type => :string, :mandatory => true, :order => 1})
     230        app.defProperty('self-id', 'OML ID', nil,{:type => :string, :mandatory => true, :order => 2})
     231        app.defProperty('oml-config-file', 'OML configuration file', "--oml-config",{:type => :string,:mandatory=> true})
     232        app.defProperty('oml-domain', 'OML domain name', "--oml-domain",{:type => :string,:mandatory=> true})
     233end
     234
     235self_id = "MonitorID"
     236oml_config_file = "/usr/local/src/mobilityfirst/eval/orbit/tutorial/conf/click-oml-config.xml"
     237oml_domain = "#{Experiment.ID}"
     238
     239defGroup("router_monitors", "router_universe") {|node|
     240  node.addApplication('mf_click_monitor') {|app|
     241    app.setProperty('ctrl_port', router_control_port)
     242    app.setProperty('self-id', self_id)
     243    app.setProperty('oml-config-file', oml_config_file)
     244    app.setProperty('oml-domain', oml_domain)
     245  }
     246}
     247}}}
     248
     249As seen above, the OML enabled monitor will work with the !MobilityFirst router and it will enable us to track and visualize the forwarding performance of the MFRs in real time. In order to report statistics to the ORBIT's OML server, the monitor periodically queries the router through a socket control port (''ctrl_port'' in our script). Using the conveniency of OML we can define different characteristics of the stream of measurements going to the OML server. This is done using an XML configuration file; in our case the following configuration file has been preloaded on the nodes:
     250
     251{{{
     252#!xml
     253
     254<omlc id='click_mon'>
     255  <collect url='tcp:oml.orbit-lab.org:3003'>
     256    <stream name='packet_stats' mp='packet_stats' samples='1'>
     257    </stream>
     258    <stream name='routing_stats' mp='routing_stats' samples='1'>
     259    </stream>
     260    <stream name='link_stats' mp='link_stats' samples='1'>
     261    </stream>
     262  </collect>
     263</omlc>
     264}}}
     265
     266After running the experiment which should be able to retrieve statistics on a per sample base divided among different groups, based on the analyzed layer (i.e. network layer, link layer, physical layer).
    239267
    240268== Execute ==
    241269
    242 ==== Starting the !MobilityFirst Components ====
    243 
    244 The final part of the experiment script is dedicated to the execution events and commands. The following snippet shows the starting of the router software, the gnrs server and the client network protocol stack:
    245 
    246 {{{
    247 #!ruby
    248     onEvent(:ALL_UP_AND_INSTALLED) do |event|
    249    
    250     info "This is my first MobilityFirst experiment"
    251 
    252     info "Initializing resources"
    253     # clean up and initialize networking for routers
    254     for i in 1..num_routers
    255         # click router cleanup
    256         group("router_#{i}").exec("killall -9 click")
    257         # gnrs cleanup
    258         group("router_#{i}").exec("killall -9 java")
    259     end
    260 
    261     #clean up and initialize networking for hosts
    262     for i in 1..num_hosts
    263         group("host_#{i}").exec("killall -9 mfstack")
    264     end
    265     wait 20
    266    
    267     # bring up routers (along with gnrs servers)
    268     info "Bringing up routers..."
    269     for i in 1..num_routers
    270         group("router_#{i}").startApplications
    271     end
    272     wait 5
    273 
    274     info "Bringing up host stacks..."
    275     for i in 1..num_hosts
    276         group("host_#{i}").startApplications
    277     end
    278    
    279     info "Access the nodes to run a program"
    280 
    281     wait 10000
    282 
    283     Experiment.done
    284     end
    285 }}}
    286 
    287 To make sure, our experiment will not conflict with any prior running experiments on the node groups, all eventual residual processes are killed. After waiting for a reasonable time in order to allow the resources to reset and be ready for a new execution, first the routers are started (and the gnrs for the associated node), and then the hosts are brought up.
    288 
    289 
    290 ==== Executing the script ====
    291 
    292 First of all you will first need to turn your assigned nodes on:
    293 
    294 {{{
    295 #!sh
    296     omf tell -a on -t system:topo:imaged
    297 }}}
    298 
    299 All the aggregated definitions just presented are included in a single script that you have to download to your grid console. In order to download it to your console, copy and paste the following command:
    300 
    301 {{{
    302 #!sh
    303     wget www.winlab.rutgers.edu/~bronzino/downloads/orbit/exercise1.rb
     270==== Running the Benchmark Application ====
     271
     272To generate the traffic that will be reported by the routers, we will use the same ''mfping'' application as in the previous exercise.
     273
     274First of all, you will need to start the experiment via an OMF script. Download the script to the orbit console copying and pasting the following command in your terminal:
     275
     276{{{
     277#!sh
     278    wget www.winlab.rutgers.edu/~bronzino/downloads/orbit/exercise2.rb
    304279}}}
    305280
    … …  
    308283{{{
    309284#!sh
    310     omf exec exercise1.rb
    311 }}}
    312 
    313 The obtained output should resemble the following snippet:
    314 
    315 {{{
    316 #!sh
    317    
    318  INFO NodeHandler: OMF Experiment Controller 5.4 (git 3fb37b9)
    319  INFO NodeHandler: Reading configuration file /etc/omf-expctl-5.4/services.yaml
    320  INFO NodeHandler: Add domain http - http://internal1.orbit-lab.org:5054/
    321  INFO NodeHandler: Add domain http - http://repository1.orbit-lab.org:5054/
    322  INFO NodeHandler: Slice ID: default_slice (default)
    323  INFO NodeHandler: Experiment ID: default_slice-2014-10-15t02.12.19.869-04.00
    324  INFO NodeHandler: Message authentication is disabled
    325  INFO Experiment: load system:exp:stdlib
    326  INFO property.resetDelay: resetDelay = 230 (Fixnum)
    327  INFO property.resetTries: resetTries = 1 (Fixnum)
    328  INFO Experiment: load system:exp:eventlib
    329  INFO Experiment: load system:exp:winlib
    330  INFO Experiment: load exercise1.rb
    331  INFO Topology: Loaded topology '/tmp/pxe_slice-2014-10-15t02.10.16.594-04.00-topo-success'.
    332  INFO Topology: Loaded topology 'system:topo:imaged'.
    333  INFO exp: node1-1.grid.orbit-lab.org assigned role of router with GUID: 1
    334  INFO exp: node1-1.grid.orbit-lab.org will also host the GNRS server
    335  INFO exp: node1-2.grid.orbit-lab.org assigned role of router with GUID: 2
    336  INFO exp: node2-1.grid.orbit-lab.org assigned role of client with GUID: 101
    337  INFO exp: node2-2.grid.orbit-lab.org assigned role of client with GUID: 102
    338  INFO exp: Definition of resources completed
    339  INFO stdlib: Waiting for nodes (Up/Down/Total): 0/4/4 - (still down: node1-2.grid.orbit-lab.org,node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [0 sec.]
    340  INFO stdlib: Waiting for nodes (Up/Down/Total): 0/4/4 - (still down: node1-2.grid.orbit-lab.org,node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [10 sec.]
    341  INFO stdlib: Waiting for nodes (Up/Down/Total): 0/4/4 - (still down: node1-2.grid.orbit-lab.org,node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [20 sec.]
    342  INFO stdlib: Waiting for nodes (Up/Down/Total): 0/4/4 - (still down: node1-2.grid.orbit-lab.org,node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [31 sec.]
    343  INFO stdlib: Waiting for nodes (Up/Down/Total): 0/4/4 - (still down: node1-2.grid.orbit-lab.org,node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [41 sec.]
    344  INFO stdlib: Waiting for nodes (Up/Down/Total): 0/4/4 - (still down: node1-2.grid.orbit-lab.org,node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [51 sec.]
    345  INFO node2-2.grid.orbit-lab.org: Device 'net/w0' reported Not-Associated
    346  INFO node1-2.grid.orbit-lab.org: Device 'net/w0' reported Not-Associated
    347  INFO stdlib: Waiting for nodes (Up/Down/Total): 2/2/4 - (still down: node2-1.grid.orbit-lab.org,node1-1.grid.orbit-lab.org) [61 sec.]
    348  INFO node2-2.grid.orbit-lab.org: Device 'net/w0' reported 76:01:22:6E:DB:FD
    349  INFO ALL_UP: Event triggered. Starting the associated tasks.
    350  INFO exp: This is my first MobilityFirst experiment
    351  INFO exp: Initializing resources
    352  INFO exp: Request from Experiment Script: Wait for 20s....
    353  INFO node1-2.grid.orbit-lab.org: Device 'net/w0' reported 76:5D:54:9F:2E:AE
    354  INFO node2-1.grid.orbit-lab.org: Device 'net/w0' reported 76:01:22:6E:DB:FD
    355  INFO node1-1.grid.orbit-lab.org: Device 'net/w0' reported 76:5D:54:9F:2E:AE
    356  INFO exp: Bringing up routers...
    357  INFO exp: Request from Experiment Script: Wait for 5s....
    358  INFO exp: Bringing up host stacks...
    359  INFO exp: Access the nodes to run a program
    360  INFO exp: Request from Experiment Script: Wait for 10000s....
    361 }}}
    362 
    363 A few comments on the obtained output:
    364   * The experiment ID can be seen on one of the first lines, it will be useful for using OML to retrieve the output for OML-enabled experiments. (this experiment does not have any outputs.)
    365   * The GUIDs assigned to the nodes can be seen. For this particular snippet, the routers (nodes1-1 and 1-2 on the grid) are assigned GUID 1 and 2, and the hosts (nodes2-1 and 2-2 on the grid) have GUID 101 and 102. Since we have a simple topology in this experiment, only a single instance of GNRS server is running, which as shown above is hosted by router with GUID 1.
    366 
    367 === Test The Network ===
    368 
    369 Before proceeding to the next step, make sure your OMF script has reached the line:
     285    omf exec exercise2.rb
     286}}}
     287
     288Once your experiment is showing you the following line:
    370289
    371290{{{
    … …  
    374293}}}
    375294
    376 This will be the signal that the host and router components are up.
    377 
    378 Now you can proceed and using the previously opened consoles log in into the two host nodes (GUIDs 101 and 102) that will be used to run the simple 'mfping' application. In order to access a running Orbit node ssh into it as follow:
     295you can proceed running the ping application. If coming from exercise 1, you will only need to repeat the same procedure. For conveniency, the detailed steps on how to run the application follow.
     296
     297[[CollapsibleStart(Instructions on how to run the mfping application)]]
     298
     299You can now proceed: using the previously opened consoles log in into the two host nodes (GUIDs 101 and 102) that will be used to run the simple 'mfping' application. In order to access a running Orbit node ssh into it as follow:
    379300
    380301{{{
    … …  
    420341}}}
    421342
     343[[CollapsibleEnd]]
     344
     345
     346==== Visualizing the Performance Data ====
     347
     348'''Method 1:''' the OMF framework supports a result service that allows experimenters to query data stored using the OML measurement framework. The query is performed over the web and requires that you know the ''experiment ID'' associated with your experiment - this is obtained from the output following the execution of the control script. It should look something like this :
     349{{{
     350#!sh
     351Experiment ID: default_slice-2014-10-15t02.12.19.869-04.00
     352}}}
     353
     354The result service supports either dumping of the entire database or a SQL based querying option to selectively retrieve required measurement data. The below HTTP request shows an example query to retrieve the reported statistics from the OML enabled monitor for !MobilityFirst Routers. In order to see the results the following web page should be requested using any browser. The following URL should be typed in the browser replacing 'experimentID' with the ID associated with your experiment:
     355
     356{{{
     357#!sh
     358http://oml.orbit-lab.org:5054/result/dumpDatabase?expID=experimentID
     359}}}
     360In this case the hostname is "oml.orbit-lab.org" and the port number is "5054".
     361
     362'''Method 2:''' Alternatively, the performance data may be visualized using a tool called ''omf-web'', an OMF web-based visualization service. It also works in concert with the result service referenced in Method 1, and makes available a variety of graph widgets to visualize live-experiment data logged using OML. Documentation on the installation and usage of omf-web can be found on the [https://github.com/mytestbed/omf_web omf-web github site].
     363
    422364
    423365== Finish ==