Multi-Genenrator with real-time data logging

Multi-Generator (mgen) is an open source network utility capable of performing performance tests and measurements by generating and receiving UDP/TCP packets. This tool is already well documented under the Networks and Communication Systems Branch at US Navy Research.

An integrated version of mgen (version 5.02c) and Orbit Measurement Framework & Library (OML) has been built and tested in the Orbit environment. The following steps describe how to build and run this version of mgen.

Set up

1) Install the OML library and develpoment files.

apt-get update
apt-get install liboml2

While building mgen other dependencies may be required to complete the compilation. Simply search and install the missing libraries and continue compiling.

2) Download the source file for mgen and apply oml patch

tar -zxvf src-mgen-5.02c.tgz
mv src-mgen-5.02c mgen
cd mgen

patch -tb -p1 < mgen_oml.patch

3) Navigate to 'makefiles' directory and build.

cd mgen/makefile
make -f Makefile.linux mgen

4) use help for a listing of options

./mgen help
GPSSubscribe(): fopen() error: No such file or directory
GPSSubscribe(): fopen() error: No such file or directory
mgen: version 5.02c
mgen [ipv4][ipv6][input <scriptFile>][save <saveFile>]
     [output <logFile>][log <logFile>][hostAddr {on|off}
     [logData {on|off}][logGpsData {on|off}]
     [event "<mgen event>"][port <recvPortList>]
     [instance <name>][command <cmdInput>]
     [sink <sinkFile>][block][source <sourceFile>]
     [interface <interfaceName>][ttl <multicastTimeToLive>]
     [unicast_ttl <unicastTimeToLive>]
     [df <on|off>]
     [tos <typeOfService>][label <value>]
     [txbuffer <txSocketBufferSize>][rxbuffer <rxSocketBufferSize>]
     [start <hr:min:sec>[GMT]][offset <sec>]
     [precise {on|off}][ifinfo <ifName>]
     [queue <queueSize>][broadcast {on|off}]
     [convert <binaryLog>][debug <debugLevel>]
     [gpskey <gpsSharedMemoryLocation>]
     [boost] [reuse {on|off}]
     [oml <server:port[,omlFile]>]
MgenApp::ProcessCommands() OnCommand(help) error
mgen: error while processing startup commands

The errors shown above are just informatory and can be ignored.

All the options above (with the exception of 'oml') are explained in details in the reference guide.

The 'oml' option specifies the oml server and port number to send and record events into an sql file. A specific filename can be given as well otherwise if the field is left blank the filename will default to 'oml_mgen_test.sq3'. Example

./mgen event "listen udp 5124" oml oml:3003,test_file

The table below describes the parameters that are recorded

Parameter Type Description
oml_sender_id INTEGER identify which node the recorded data came from
oml_seq INTEGER oml sequence number
oml_ts_client REAL oml client time stamp
oml_ts_server REAL oml server time stamp
"proto" TEXT type of traffic → UDP, TCP
"direction" TEXT direction of traffic → sender, receiver
"flowid" INTEGER mgen flow id
"seq_num" INTEGER sequence number for each flow id
"src_addr" TEXT source IP address
"src_port" INTEGER source port
"dst_addr" TEXT destination IP
"dst_port" INTEGER destination port
"data_len" INTEGER length of data

Simple OML recording

1) Refer to the reference guide for complete details on using formatted scripts and options to run mgen. In this section we'll only describe

2) To demonstrate the OML recording feature, setup mgen on two nodes and run a script with the input option. For this example node6-6 and node15-6 on the grid were used and the corresponding script is showing the table.

Node Id Traffic Direction Script contents
node6-6 sender 0.0 ON 0 UDP SRC 5000 DST POISSON [10 1024:1200]
2.0 ON 1 UDP SRC 5001 DST POISSON [8 1024:1200]
4.0 ON 2 UDP SRC 5002 DST POISSON [6 1024:1200]
6.0 ON 3 UDP SRC 5003 DST POISSON [4 1024:1200]
8.0 ON 4 UDP SRC 5004 DST PERIODIC [1 256]
node15-6 receiver 0.0 LISTEN UDP 5000-5004

The sender script file (udp_send_script.mgn) configures the node to transmit UDP traffic to the receiver node which is configured to listen for UDP packet on port ranging from 5000 to 5004.

The format of the sender script is shown below separated into fields.

Event time Event Flow id Protocol Source port Destination IP and port Traffic pattern
0.0 ON 0 UDP SRC 5000 DST POISSON [10 1024:1200]

Likewise the receiver script (udp_recv_script.mgn) is condensed into a single liner

Event time Event Protocol Port list
0.0 LISTEN UDP 5000-5004

Check MGEN SCRIPT Format section of the reference page for an explanation of all the fields.

3) Run each script using the 'input' option of mgen and specify the oml options for recording.

node6-6> ./mgen input udp_send_script.mgn       oml oml:3003,udp_send_mgen

node15-6> ./mgen input udp_recv_script.mgn oml oml:3003,udp_recv_mgen

Each instance of mgen will start producing the text output. In the background it will also send packet infomation to the specified oml server and database. To quickly view the contents of the data files, run the following command from the console:

console> sqlite3 /var/lib/oml2/udp_recv_mgen.sq3 .dump
PRAGMA foreign_keys=OFF;
INSERT INTO "_senders" VALUES('et',1);
CREATE TABLE "_experiment_metadata" (oml_tuple_id INTEGER PRIMARY KEY, oml_sender_id INTEGER, oml_seq INTEGER, oml_ts_client REAL, oml_ts_server REAL, "subject" TEXT, "key" TEXT, "value" TEXT);
INSERT INTO "_experiment_metadata" VALUES(1,NULL,NULL,NULL,NULL,NULL,'table__experiment_metadata','0 _experiment_metadata subject:string key:string value:string');
INSERT INTO "_experiment_metadata" VALUES(2,NULL,NULL,NULL,NULL,NULL,'start_time','1528726472');
INSERT INTO "_experiment_metadata" VALUES(3,NULL,NULL,NULL,NULL,NULL,'table__mp__udp_recv_mgen','1 _mp__udp_recv_mgen proto:string flowid:int32 seq_num:int32 src_addr:string src_port:int32 dst_addr:string dst_port:int32 data_len:int32');
CREATE TABLE "_mp__udp_recv_mgen" (oml_tuple_id INTEGER PRIMARY KEY, oml_sender_id INTEGER, oml_seq INTEGER, oml_ts_client REAL, oml_ts_server REAL, "proto" TEXT, "flowid" INTEGER, "seq_num" INTEGER, "src_addr" TEXT, "src_port" INTEGER, "dst_addr" TEXT, "dst_port" INTEGER, "data_len" INTEGER);
INSERT INTO "_mp__udp_recv_mgen" VALUES(1,1,1,3.50581999868154525756e-01,2.95809999999999961861e-01,'UDP',0,4,'',5000,'',5000,1044);
INSERT INTO "_mp__udp_recv_mgen" VALUES(2,1,2,4.55220999661833047866e-01,0.3979,'UDP',0,5,'',5000,'',5000,1172);
INSERT INTO "_mp__udp_recv_mgen" VALUES(3,1,3,0.585224999114871,5.2790399999999992886e-01,'UDP',0,6,'',5000,'',5000,1192);
INSERT INTO "_mp__udp_recv_mgen" VALUES(4,1,4,1.40533899888396263122e+00,1.489031,'UDP',0,7,'',5000,'',5000,1041);
INSERT INTO "_mp__udp_recv_mgen" VALUES(5,1,5,1.59353199973702430725e+00,1.614587,'UDP',0,8,'',5000,'',5000,1110);
INSERT INTO "_mp__udp_recv_mgen" VALUES(6,1,6,1.68116699904203414916e+00,1.62387099999999984234e+00,'UDP',0,9,'',5000,'',5000,1198);
INSERT INTO "_mp__udp_recv_mgen" VALUES(7,1,7,1.72220599837601184844e+00,1.66488099999999983324e+00,'UDP',0,10,'',5000,'',5000,1176);
INSERT INTO "_mp__udp_recv_mgen" VALUES(8,1,8,1.89545799978077411651e+00,1.838213,'UDP',1,0,'',5001,'',5001,1049);
INSERT INTO "_mp__udp_recv_mgen" VALUES(9,1,9,1.91608599945902824401e+00,1.858813,'UDP',1,1,'',5001,'',5001,1127);
INSERT INTO "_mp__udp_recv_mgen" VALUES(10,1,10,2.05955099686980247497e+00,2.144107,'UDP',1,2,'',5001,'',5001,1169);
INSERT INTO "_mp__udp_recv_mgen" VALUES(11,1,11,2.0766779966652393341e+00,2.269656,'UDP',1,3,'',5001,'',5001,1080);
INSERT INTO "_mp__udp_recv_mgen" VALUES(12,1,12,2.10336599871516227722e+00,2.269673,'UDP',1,4,'',5001,'',5001,1064);
INSERT INTO "_mp__udp_recv_mgen" VALUES(13,1,13,2.15250899642705917358e+00,2.269681,'UDP',1,5,'',5001,'',5001,1075);
INSERT INTO "_mp__udp_recv_mgen" VALUES(14,1,14,2.19350999966263771057e+00,2.269688,'UDP',0,11,'',5000,'',5000,1180);
INSERT INTO "_mp__udp_recv_mgen" VALUES(15,1,15,2.24396899715065956115e+00,2.269796,'UDP',1,6,'',5001,'',5001,1073);
INSERT INTO "_mp__udp_recv_mgen" VALUES(16,1,16,2.24748399853706359863e+00,2.269805,'UDP',0,12,'',5000,'',5000,1161);
INSERT INTO "_mp__udp_recv_mgen" VALUES(17,1,17,2.26098299771547317504e+00,2.269826,'UDP',1,7,'',5001,'',5001,1148);
INSERT INTO "_mp__udp_recv_mgen" VALUES(18,1,18,2.2951769977808,2.269834,'UDP',0,13,'',5000,'',5000,1164);
INSERT INTO "_mp__udp_recv_mgen" VALUES(19,1,19,2.30865999683737754821e+00,2.26984,'UDP',1,8,'',5001,'',5001,1128);
INSERT INTO "_mp__udp_recv_mgen" VALUES(20,1,20,2.32708599790930747985e+00,2.269847,'UDP',1,9,'',5001,'',5001,1171);
INSERT INTO "_mp__udp_recv_mgen" VALUES(21,1,21,2.35045599937438964843e+00,2.293127,'UDP',0,14,'',5000,'',5000,1153);
INSERT INTO "_mp__udp_recv_mgen" VALUES(22,1,22,2.44623499736189842224e+00,2.388898,'UDP',0,15,'',5000,'',5000,1166);
INSERT INTO "_mp__udp_recv_mgen" VALUES(23,1,23,2.58714799955487251281e+00,2.52982499999999976836e+00,'UDP',1,10,'',5001,'',5001,1173);

Last modified 2 years ago Last modified on Jun 14, 2018, 2:03:32 PM

Attachments (2)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.