Go with the flow!
Last week I attended the ATCA Summit to present the concept of Application-Defined Networking (ADN). I also participated to a panel on the future of Openflow with distinguished colleagues from Cisco, Dell and Big Switch Networks. The lively discussion highlighted an important topic, the gap between Applications and Openflow.
I believe this gap should be closed and a key for this to happen is to reconcile the concept of flow from both an Application and a Network perspective.
Let me explain. To ensure data exchanges in a computer network, complex tasks such as routing are delegated to network devices from various different vendors while congestion control is delegated to the TCP stack of various operating systems. Internally networks manage individual packets while in the end systems, the TCP implementation and the application processes all speak in terms of flows. An Application-Defined Networking approach that aims at smoothing out communication and computation for (and by) the application must logically control the virtual networks that operate the application-level abstraction (flows) rather than packets.
Flow-based networking has been around for almost 10 years. In fact today the concept of flows is largely used inside the network since the advent on IP QoS in early 90’s. It enables packets from different flows to be handled differently by means of separate queues in switches, routers and network adapters, to achieve traffic shaping, fair queuing or Quality of Service.
Openflow exploits these features, particularly the lookup tables in Ethernet switches and routers. These flow-tables implement firewalls, address translation, quality of service or collect statistics. The issue is that the flow-tables implementations vary between different vendors. The goal of Openflow is to provide a standard way of managing flow-tables. This enables to partition the network traffic on a flow base. This way, the various different flows can be grouped and isolated, in order to be routed, processed or controlled independently.
The key issue is to give the correct (useful) definition of a flow in function of the application or the network perspective adopted.
- For example in the TCP case a flow is a virtual circuit, also known as a virtual connection or a byte stream identified by a 5-tuple.
- In packet switches, the flow may be identified by a single identifier IEEE 802.1Q Virtual LAN tagging in Ethernet networks or by a Label Switched Path in MPLS tag switching.
- In Openflow, the flow is not defined, the software is responsible of the definition. Openflow enables the manipulation of n-tuples for identifying the flows with the possibility to have more than 10 different fields to determine a flow!
This openness is both a strong point and a threat for Openflow; a topic of discussion at the ATCA panel on the Future of Openflow.
In the Cloud, the definition of the flow can be simply the pair of IP address, representing the end points (VMs) exchanging data. Indeed, a Cloud application is a distributed set of processes running on different virtual machines. When the processes need to exchange data with each other, with a storage component or a user they send sequences of packets through a socket directly attached to the TCP/IP layer of their operating system. From the application perspective the emitted flow of packets is identified within the network by the port number, the IP of the sending virtual machine and the port of the receiver, and the IP of the receiving machine. The TCP field is useful only at the end host level and all other fields, the MAC address of the virtual interface or of the physical NIC are useless for defining the flow after the next hop in the network. All these fields are modified by different equipments all along the path until destination.
But defining correctly a flow is only half of the way. In final analysis, what matters is to ensure the overall functionality of the application network and that is essentially moving data – efficiently – between different end points of this application.
In civil engineering, Traffic flow (ref. Wikipedia) ‘is the study of interactions between vehicles drivers and infrastructure (including highways, signage, and traffic control devices), with the aim of understanding and developing an optimal road network with efficient movement of traffic and minimal traffic congestion problems’.
Similarly, understanding and developing an optimal virtual network with efficient movement of traffic and minimal traffic congestion problems is a key for Application-Defined Networking.
For this, an ADN enabler analyzes the interactions between application flows (vehicles & drivers) and infrastructure (roads and highways). End to end packet flow can be represented as a path in a virtual network to model network performance. For example a water flow network can be used to conceptualize packet flow. Channels can be thought of as pipes, with the pipe capacity corresponding to bandwidth and flows corresponding to data throughput. This visualization can help understand flow dynamics, bottlenecks, queuing, and the unique requirements of tailored systems.
For Application Defined Networks, apps need to participate in defining the flows and paths, and future posts will elaborate the disadvantage of not having this.