How-To: Extending OpenFlow with Vendor messages
This tutorial attempts to describe how to use the OpenFlow vendor message to create custom messages. We first provide an overview of the message structure. We then describe the vendor message implementations found in openflowj, the Java implementations of the OpenFlow protocol used in Floodlight and FlowVisor.
Before Beginning
Class names are shown in terminal type, and methods, in italics. We use the fully qualified names of the classes and packages, as found in Floodlight source. It is best to follow the tutorial with the Floodlight source sitting in front of you - to import the code to Eclipse, follow these instructions.
For working examples in this page we'll use snippets of the Nicira vendor messages, found in org.openflow.vendor.nicira of the Floodlight source. The Nicira extensions add OpenFlow role features, introduced in OpenFlow v1.2, to the v1.0 protocol used in Floodlight.
Contents
1. Overview: The Vendor Message
2. openflowj
3. The OpenFlow Message
4. Conclusion, or Links
1. Overview: The Vendor Message
OpenFlow provides a vendor message type as a way to offer third parties a way to customize the protocol without going out of spec. Although called the "Vendor message", this message type provides a handy way for a developer to implement and test out experimental features without wantonly modifying the protocol.
Vendor messages are identified by OpenFlow message type (OFType) value of 4. In addition to the standard OpenFlow header, The vendor type message has its own message header and a fully customizable payload.
A vendor message header contains a vendor ID and a data type. The vendor ID identifies the vendor implementing the custom message, and is typically the organizationally unique identifier (OUI) of the vendor. As an individual, you can randomly pick and use an integral value for this purpose and nothing would balk at this, but this is highly discouraged. The proper conduct for this case is either to use your institution's OUI (if it has one), or to apply for an ID from the Open Networking Foundation (see here for details).
The data type is used to indicate any subtypes that this message may have. For example, Nicira's vendor messages use Nicira's OUI (002320) as the vendor ID and comes in two types, a Request and Reply, indicated by the data type values of "10" and "11".
The rest of the message is the vendor message payload, and can be freely defined. To sum it up, the full OpenFlow vendor message takes on the following general format:
|<------OpenFlow header------>||<----------------------Vendor Data------------------------>| | ||<-Vendor message header-->||<---Vendor message payload---->| [OF ver(1)|OFType(1)|length(2)][Vendor ID(2-8)|dataType(4)][user-defined structures(varied)]
Where the numbers in the parenthesis denote the field size in bytes. The vendor message header and payload in combination make up the full payload of the OpenFlow message, the vendor data.
2. openflowj
In openflowj, the base interface and framework needed for creating vendor messages are found in the package org.openflow.protocol.vendor. Specifically, openflowj provides developers with a way to define custom vendor data classes. We start with a brief overview of the interface provided for this purpose and the general approach used for vendor data implementation, after which we describe the detailed requirements for implementing vendor data.
Interface OFVendorData
OFVendorData is the Java interface required by any class defining custom vendor data. OFVendorData imposes just a few methods on a class implementing it:
- getLength() : return length of the data
- readFrom(ChannelBuffer data, int length) : Read the vendor data from the specified ChannelBuffer
- writeTo(ChannelBuffer data) : Write the vendor data to the specified ChannelBuffer
These methods are needed for the serialization/deserialization of the message.
Structuring the code
A custom vendor message is comprised of the OpenFlow header that identifies it as a vendor type message, and the vendor data. The vendor data is usually implemented as a collection of classes - one base class that implements OFVendorData and defines the Vendor ID, and the various subclasses that extend the base class to implement the individual message types.
This organization isn't a requirement, but makes code reuse easier. The "nesting" of message classes can be thought of as the implementation of the message structure in layers - Each subclass implements either 1) message components that are encapsulated by components in its parent class, and/or 2) methods that assign values to variables declared in its super-classes. We will see the this in the code snippets to follow.
2.1 The Nicira vendor messages
As mentioned earlier, the Nicira vendor messages implement newer protocol features in an older version of OpenFlow that does not support it. This vendor message's two message types are implemented across four classes with the following parent-child relations:
OFNiciraVendorData--->OFRoleVendorData--+--->OFRoleRequestVendorData
|
---> = "parent class of" +--->OFRoleReplyVendorData
Where OFNiciraVendorData is the base class implementing OFVendorData, and OFRoleRequestVendorData and OFRoleReplyVendorData implement the two message types. This chart will hopefully make more sense with the following sections.
2.2 Vendor Data Components
What we refer to as the "vendor message header" is typically a set of variables defined in the class that implements OFVendorData.
For example, the vendor ID and data type are part of OFNiciraVendorData, the base class for all Nicira vendor messages:
public class OFNiciraVendorData implements OFVendorData {
public static final int NX_VENDOR_ID = 0x00002320;
/**
* The value of the integer data type
* at the beginning of the vendor data
*/
protected int dataType; (1)
...
As explained in the start of the section, OFNiciraVendorData is extended to implement the Request and Reply message types (OFRoleRequestVendorData and OFRoleReplyVendorData, respectively). Note how at (1) the value of dataType is not set within this class - this value is set through the base class's constructor, which takes an integer value:
/**
* Contruct Nicira vendor data with the specified data type
* @param dataType the data type value at the beginning of the vendor data.
*/
public OFNiciraVendorData(int dataType) {
this.dataType = dataType;
}
The values passed to this constructor are found in each subclass that represents a message type. We can see this in OFRoleReplyVendorData:
/**
* The data type value for a role reply
*/
public static final int NXT_ROLE_REPLY = 11;
/**
* Construct a role reply vendor data with an unspecified role value.
*/
public OFRoleReplyVendorData() {
super(NXT_ROLE_REPLY);
}
If we trace back, we learn that super() above refers to the constructor of OFRoleVendorData, a subclass of OFNiciraVendorData. OFRoleVendorData takes the value passed to it by OFRoleReplyVendorData and passes it to the constructor of its parent class.
You may wonder what the purpose of OFRoleVendorData is - for now, it is okay to think of it as the class that defines the single-integer vendor data payload common to both Request and Reply message types, and the values that it can take ("role values" as commented below):
public class OFRoleVendorData extends OFNiciraVendorData {
/**
* Role value indicating that the controller is in the OTHER role.
*/
public static final int NX_ROLE_OTHER = 0;
/**
* Role value indicating that the controller is in the MASTER role.
*/
public static final int NX_ROLE_MASTER = 1;
/**
* Role value indicating that the controller is in the SLAVE role.
*/
public static final int NX_ROLE_SLAVE = 2;
protected int role;
...
There is more to this class, but that is covered later, under serialization.
The Vendor ID and data type are the only requirements in terms of vendor header content. Given that the methods required by OFVendorData are provided, along with those required for message registration (which we cover next), the message implementation may be structured as needed.
2.3 Message Registration
As expected from the variable structure of vendor messages, a given vendor message must be registered with openflowj before it can handle your messages properly. Registration is a two step process:
- Vendor ID registration
- Message type registration
We can see this by taking a look at the method initVendorMessages() from Floodlight's core controller class Controller. We see that (1) the vendor ID is registered first, and (2),(3) followed by each class representing a message type:
// Configure openflowj to be able to parse the role request/reply vendor messages.
// first register the vendor ID
OFBasicVendorId niciraVendorId = new OFBasicVendorId( (1)
OFNiciraVendorData.NX_VENDOR_ID, 4);
OFVendorId.registerVendorId(niciraVendorId);
// then each data type, starting with reqest
OFBasicVendorDataType roleRequestVendorData = (2)
new OFBasicVendorDataType(
OFRoleRequestVendorData.NXT_ROLE_REQUEST,
OFRoleRequestVendorData.getInstantiable());
niciraVendorId.registerVendorDataType(roleRequestVendorData);
// then the reply
OFBasicVendorDataType roleReplyVendorData = (3)
new OFBasicVendorDataType(
OFRoleReplyVendorData.NXT_ROLE_REPLY,
OFRoleReplyVendorData.getInstantiable());
niciraVendorId.registerVendorDataType(roleReplyVendorData);
Where, as seen earlier, NXT_ROLE_REQUEST and NXT_ROLE_REPLY are the request and reply data type values for the two Nicira vendor message data types.
There are two things to point out here:
- Since vendor IDs may vary in length, we indicate the length in bytes that the vendor ID is when we instantiate the OFBasicVendorId. In (1) we provide the constructor with the value 4 along with the actual Vendor ID, indicating that the Nicira vendor ID is an integer (4 bytes long).
- As seen above in (2) and (3), the class implementing vendor data must provide an instantiator. The instantiator provides
OFBasicVendorDataTypewith a format that allows it to safely avoid making assumptions about the structure of the vendor data. The method getInstantiable() returns an instantiator for the class.
The second point indicates that we need a getInstantiable() (or something of equal function) in our vendor data class. The following snippet was taken from OFRoleRequestVendorData, but the structure will pretty much be the same for any vendor data class (e.g. replace OFRoleRequestVendorData below with your class):
protected static Instantiable<OFVendorData> instantiable =
new Instantiable<OFVendorData>() {
public OFVendorData instantiate() {
return new OFRoleRequestVendorData();
}
};
/**
* @return a subclass of Instantiable<OFVendorData> that instantiates
* an instance of OFRoleRequestVendorData.
*/
public static Instantiable<OFVendorData> getInstantiable() {
return instantiable;
}
A non-registered Vendor message data payload is interpreted simply as a byte array (an OFByteArrayVendorData object, to be precise), and cannot be cast to your message (sub)class(es) for further handling. Aside from throwing a ClassCastException if you try, this is inconvenient since you won't be able to invoke the class methods specific to your vendor data class for message-specific processing.
2.4 Message Serialization
As mentioned earlier, a class implementing OFVendorData must have a readFrom(ChannelBuffer data, int length) and writeTo(ChannelBuffer data) method for reading and writing the data from/to a ChannelBuffer. A getLength() method that returns the size in bytes of the vendor data, minus the Vendor ID length, is also required for your Vendor message to be properly sent and received. We do not count the OpenFlow header length since helper methods take it into account along with the Vendor ID length. Note, a getLength() function that returns the wrong value may not always prevent your controller from sending the message, but will definitely stop you from receiving it on the other end. So if your message is not being received correctly one of the first things you should check (after channel functionality) is your getLength() function.
The packet structure is determined by the order in which the various fields are written to the ChannelBuffer, so readFrom() and writeTo() should read/write the fields to/from the ChannelBuffer in the same order.
As an example, we can look at OFRoleVendorData, the parent class of the Nicira Request and Reply messages. Since both messages only differ in the values of the role field, the readFrom(), writeTo(), and getLength() methods for the final packet structure are defined in this class. The message structure looks like this:
|<-Vendor ID->||<-dataType->||<---payload(4)--->|
[OpenFlow Header][ 0x00002320 ][ 10|11 ][ 0|1|2 ]
Where the values within the square brackets separated by pipes are the various decimal values that the fields can take. For this message, what we had dubbed the vendor message payload in section 1 is one integral value of 4 bytes, used to indicate controller role. The packet structure is reflected in readFrom() and writeTo():
public void readFrom(ChannelBuffer data, int length) {
super.readFrom(data, length);
role = data.readInt();
}
public void writeTo(ChannelBuffer data) {
super.writeTo(data);
data.writeInt(role);
}
super in this case is OFNiciraVendorData, which takes care of reading and writing the integral dataType field:
@Override
public void readFrom(ChannelBuffer data, int length) {
dataType = data.readInt();
}
@Override
public void writeTo(ChannelBuffer data) {
data.writeInt(dataType);
}
As for getLength(), the total returned is the length of the two fields (8 bytes). looking at the two classes side-by-side makes this clear. First OFRoleVendorData:
@Override
public int getLength() {
return super.getLength() + 4;
}
Then OFNiciraVendorData:
@Override
public int getLength() {
return 4;
}
Now that we have vendor data, we can complete the full OpenFlow message — which brings us to the next section.
3. The OpenFlow Message
This section describes how to construct the full vendor type message once we have a structured vendor data object, or alternatively, parse a vendor type message containing a known (e.g. registered) type of vendor data.
The class OFVendor implements the full OpenFlow vendor message. OFVendor itself is a subclass of OFMessage, which implements the generic OpenFlow message header. As with other OpenFlow messages, the vendor messages are received by the controller via the OFChannelHandler, an inner class of Controller, and sent out via the write() methods in OFSwitchImpl, the class that represents a switch connected to the controller. Description of the full system is better left to another tutorial, as it will detract from our main topic.
3.1 Message construction
We sidestep from the Nicira messages for this section to look at OFVendorTest, the unit test for Vendor messages, found in /src/test/java of Floodlight source. Aside from their intended purpose, the unit tests are fairly good references for figuring out how various components of the controller and openflowj are instantiated and/or used.
We can reference the first few lines of testVendorData() of OFVendorTest to see how a vendor message may be constructed in full.
OFVendor msg = makeVendorMessage(ACME_VENDOR_ID); (1)
OFVendorData vendorData = new AcmeVendorData1((short)11, (short)22); (2)
msg.setVendorData(vendorData); (3)
msg.setLengthU(OFVendor.MINIMUM_LENGTH + vendorData.getLength()); (4)
We see four basic steps to the process:
- Create an "empty" OFVendor message. the method makeVendorMessage() is a helper function:
private OFVendor makeVendorMessage(int vendor) { OFVendor msg = (OFVendor) messageFactory.getMessage(OFType.VENDOR); (1) msg.setVendorDataFactory(new BasicFactory()); (2) msg.setVendor(vendor); (3) return msg; }This function is (1) wired to an OpenFlow message factory that supplies us with the empty OFVendor message, and also does some prep work on the OFVendor message, such as (2) setting its message factory and (3) the Vendor ID field. - Create the vendor data, and set its fields as needed.
- set the vendor data of the message created in 1. to the vendor data created in 2.
- set the length of the total message. OFVendor.MINIMUM_LENGTH counts just the length of the OpenFlow header (8 bytes) and the Vendor ID and is 12 bytes.
At this point, the message is ready to be sent. Sending the message to a switch is a one-liner, in theory:
sw.write(msg, null);
3.2 Reading message contents
OFVendor provides the following methods for deconstructing a message:
- getVendor() : return the Vendor ID
- getVendorData() : return the vendor data
Once the vendor data is extracted, it may be cast to the appropriate message type for further parsing. A good reference for this process is the method handleVendorMessage(), found in OFChannelHandler. It extracts the dataType from the message, and passes it to a switch statement to cast it to the correct message type and to pass it to appropriate methods:
int dataType = niciraVendorData.getDataType();
switch (dataType) {
case OFRoleReplyVendorData.NXT_ROLE_REPLY:
OFRoleReplyVendorData roleReplyVendorData =
(OFRoleReplyVendorData) niciraVendorData;
handleRoleReplyMessage(vendorMessage,
roleReplyVendorData);
break;
default:
log.warn("Unhandled Nicira VENDOR message; " +
"data type = {}", dataType);
break;
}
This method processes all vendor messages that the controller receives. As it is currently implemented to handle only Nicira vendor messages, it must be modified to also handle your messages. This can be as simple as adding your Vendor ID to the first switch statement in the method:
int vendor = vendorMessage.getVendor();
switch (vendor) {
case OFNiciraVendorData.NX_VENDOR_ID:
...
break;
case YourVendorData.VENDOR_ID:
....
break;
default:
log.warn("Unhandled VENDOR message; vendor id = {}", vendor);
break;
}
3.3 Troubleshooting messages
When things go wrong, messages will either be thrown away and never reach your handlers, or will throw exceptions that may or may not reach logging. One way to minimize this is to check that the messages can be built and read/written from/to ChannelBuffers within your controller. Note, a local check won't prevent everything. In general, providing the correct getLength() value helps to prevent many common errors, so it is important to check that this function is returning the right value. BasicFactory has a dumpBuffer() function that returns a hexdump of a message as a string that you can check more thoroughly to see if what you think you have implemented is actually what you have.
//where msg below is your Vendor message
ChannelBuffer bb = ChannelBuffers.dynamicBuffer();
bb.clear();
msg.writeTo(bb);
bb.resetReaderIndex();
//print vendor data as string
System.out.println(msg.getVendorData().toString());
//print a hexdump
System.out.println(BasicFactory.dumpBuffer(bb))
The above should produce something like below as output (demonstrated with a custom message not discussed here)
CPLCommandMsg[[UID=2561 peer=false SID=1 TID=2][RCmd=8 EType=any] Command=CONTINUE] 01040028 00000000 0000ffff 00000005 00000000 00000a01 01000000 02080000 00000000 00000000
4. Conclusion
In this tutorial, we tried to cover the basics of developing custom messages. While this page describes Vendor message creation, usage delves into controller service development. The following links provides information on getting started on this front:
Floodlight development
- Developers' Wiki (a collection of links): http://www.openflowhub.org/display/floodlightcontroller/For+Developers
- floodlight-dev mailing list: http://groups.google.com/a/openflowhub.org/group/floodlight-dev
Netty
Netty is the networking library used in Floodlight.
- The official docs: https://netty.io/Documentation/WebHome
- A big-picture description: http://www.znetdevelopment.com/blogs/2009/04/21/netty-using-handlers/
- A tutorial: http://seeallhearall.blogspot.com/2012/06/netty-tutorial-part-15-on-channel.html
