VNU University of Engineering and Technology

Faculty of Electronics and Telecommunications

======================

SOFTWARE DEFINED NETWORKS

Lecture 5: SDN Challenges and Scalability

References: Lecture Notes from CIS5930 Software Defined Networking

Page 1 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU UET
 SDN challenges

Deployment challenges
 Device heterogeneity
 Interoperation with legacy systems
 Controller and apps scalability
Infrastructure research challenges
 NOS issue: network abstraction
 To make SDN accessible to a typical user
• Programming abstraction
• Configuration abstraction
 Programming abstraction support
• Update abstraction
• Modular composition abstraction
• Correctness verification, debugging, and testing
 Security
Page 2 > Software Defined Networks
Dr. Dinh Thi Thai Mai – VNU - UET
 Device heterogeneity

Heterogenous switches
 Number of packet-handling rules
 Range of matches and actions
 Multi-stage pipeline of packet processing
 Offload some control-plane functionality
Legacy systems: how to interoperate

access MAC IP
control look-up look-up

Page 3 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET3
 SDN Scalability

Controller scalability
 Controller is much slower than the switch (in processing packets)
 Processing packets leads to delay and overhead
 Need to keep most packets in the “fast path”

packets
Page 4 > Software Defined Networks
Dr. Dinh Thi Thai Mai – VNU - UET4
 SDN scalability: distributed controller, distributed apps

For scalability
Controller and reliability Controller
Application Application

Partition and replicate state

Network OS Network OS

Page 5 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET5
 NOS: Network abstraction

Centralized network view (global network graph)

Levels of details:
 Topology, flows, link usage statistics, where to cut off?
 Frequency of control updates, collecting statistics introduces overhead
 Does the data structure for NIB make a difference?

Page 6 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Programming Abstractions

Controller APIs are low-level

 Thin veneer on the underlying hardware
Need better languages
Controller
 Algorithmic programming
 Composition of modules
 Managing concurrency
 Querying network state
 Network-wide abstractions
 Debugging and testing
 Formal verification
Switches

Page 7 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET7
 Configuration abstraction?

Is programming abstraction in its perfect form good enough?

Users are system administrators.

Page 8 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Implementation software challenges: Update abstraction

How to install new rules and remove old rules so that a packet will only experience
one consistent network state?
 Need theory (update consistency model) and implementation (working system
based on theory)
Example:

Page 9 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Update abstraction

1. update I to forward S traffic to F3 while continuing to forwarding U and G traffic to

F1 and F traffic to F3
2. Wait until in-flight packets have been processed by F2
3. update F2 to deny SSH packets
4. update I to forward G traffic to F2 while continuing to forwarding U traffic to F1 and
S and F traffic to F3.
Page 10 > Software Defined Networks
Dr. Dinh Thi Thai Mai – VNU - UET
 Composition of modules

Many network tasks: routing, access control, traffic monitoring, etc

All require packet handling rules installed in the same flow table
 These rules may interact with one another, making it very difficult to decoupling
the high level tasks from implementation.
Significant challenges in specifying the tasks and realizing the tasks.

Page 11 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Testing and Debugging

OpenFlow makes programming possible

 Network-wide view at controller
 Direct control over data plane
Plenty of room for bugs
 Still a complex, distributed system
Need for testing techniques
 Controller applications
 Controller and switches
 Rules installed in the switches

Page 12 > Software Defined Networks

12
Dr. Dinh Thi Thai Mai – VNU - UET
 SDN security issues

Can the SDN programming system provide some security features, what should they
be?

Can NOS provide process like protection among SDN applications?

Can we have an integrated security mechanism for SDN?

Page 13 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 SDN controller scalability issue

Fundamental issue: the speed gap between data plane and control plane.

SDN controller

Switch OS Switch OS
Switch OS
Switch HW Switch HW
Switch HW
10-100 Gbps 10-1000Mbps
???
Page 14 > Software Defined Networks
Dr. Dinh Thi Thai Mai – VNU - UET
 SDN controller scalability issue

Control plane
1. Stress controller resources

2. Stress the control channel

Data plane

Data plane can overwhelm control plane by design

o Control channel or controller resources
SDN controller has a fundamental scalability issue

Page 15 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Solutions 1: Increase controller capacity - distributed
controllers

Control
Controlplane
plane
Control plane
1. Stress controller resources

2. Stress the control channel

Data plane

Flat structure multiple controllers

 ONIX (OSDI’10)
 ONOS(HostSDN’14)

Page 16 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Solutions 2: reduce traffic to controller -- Hierarchical controller

Root Control
Control plane 1. Stress controller resources

Local Control
2. Stress the control channel
Data plane Data plane

Hierarchical controller design

 Kandoo (HotSDN’12)

Page 17 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Solutions 2: Reduce traffic to controller – offload control to
switch

Root Control
Control plane 1. Stress controller resources

offload Control
2. Stress the control channel
Data plane Data plane

Offload to switch control plane

 Diffane (SIGCOMM’10)
 DevoFlow(SIGCOMM’11)

Page 18 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 ONIX

ONIX’s view of network components

 Physical infrastructure: switches, routers, etc
 Connectivity infrastructure: channels for messages.
 Onix: A distributed system running the controller
 Control logic: network management applications running on top on Onix

Page 19 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Onix architecture

Page 20 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix NIB

Holds a collection of network entities

 Can be viewed as a centralized graph with notification mechanism
Updates to the NIB are asynchronous.

Page 21 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix NIB API

Query: find entities

Create/destroy: create and remove entities
Access attributes: inspect and modify entities
Notification: receive update about changes
Synchronize: wait for updates being export to network elements and controllers
Configuration: configure how state is imported to and exported from the NIB
Pull: ask entities to be imported on-demand

Page 22 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix abstraction

Global view: Observe and control a centralized

network view (NIB) which contains all physical
network elements
Flow: the first packet and subsequent packets with
the same header are treated in the same way.
Switch: with flow tables <header: counters, actions>
Event-based operation: the controller operations are
triggered by routers or applications.

Page 23 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix API

The global view is represented as a network graph

Nodes represent physical network entities
Developers program over the network graph

• Write flow entry

• List ports
• Register for updat
……

Page 24 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Network Information Base

The NIB is the focal point of the system

 State for applications to access
 External state changes imported into it
 Local state changes exported from it

Page 25 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix scalability

A single physical controller won’t work for large network

 NIB will overrun the memory in one server
 CPU and bandwidth for one server will not be enough
Onix solution: Partition, aggregation, and consistency.
 Partition:
• Each Onix instance may have connections to a subset of network
elements
• Network control logic can configure to keep a subset of the NIB in
memory

Page 26 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix scalability

Partition, aggregation, and consistency.

 Aggregation:
• Each Onix instance can be configured to expose a subset of elements in
its NIB as an aggregate element (reduce fidelity) to another Onix
instance.
 Consistency and durability
• Control logic dictates the consistency requirement of the network state it
manages
 Two storage options
» Replicated transactions (SQL) storage
» One-hop memory-based DHT
• Control logic resolve conflicts when necessary.

Page 27 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix reliability

Network element and link failures

 Control logic reconfigures to deal with such failures
Management connectivity infrastructure failures
 Assumed reliable (remember Google B4 issue?)
Onix failures:
 Distributed coordination facilities to provide failover

Page 28 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onix Summary

Onix provides state distribution capability

The developers of management applications still have to understand the scalability
implications of their design

One of the earlier SDN controllers: the controller functionality and application
functionality are not clearly partitioned.

Page 29 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Distributed SDN controller Research issues?

Page 30 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Distributed SDN research issues

Network abstraction for distributed SDN

 Need concrete understanding of network abstraction in the current systems
Exploit existing distributed system techniques to address distributed network
abstraction issues
 Consistency, usability, synchronization, fault tolerance, etc
Adapting distributed system techniques specifically to SDN controller
 No need to reinvent the wheel

Page 31 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 ONOS: Towards an Open, Distributed SDN OS

Earlier NOS dodges the distributed system issues.

Earlier distributed NOS may try to reinvent the wheel.
ONOS is a second generation of distributed NOS, separating distributed systems
issues with network management issues
 We know how to distribute and maintain information in a distributed manner.
Many systems are available.
 Distributed NOS can utilize the existing distributed information systems and
focus on network management issues.

Page 32 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Distributed system building blocks

Distributed storage system

 Cassandra
 RAMcloud (in memory storage)
Distributed graph database (Titan)
Distributed event notification(HazelCast)
Distributed coordination service (Zookeeper)
Distributed system data structures and algorithms
 Distributed hash table (DHT)
 Consensus algorithm
 Failure detector
 Checkpointing
 Transaction

Page 33 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Onos architecture

Page 34 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Onos abstraction: Global network view

Page 35 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Onos summary

Use existing distributed system infrastructure.

Focus on making it efficient with known distributed system applications.
 E.g. how to maintain, lookup, and update the topology effectively

Page 36 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Kandoo: A framework for efficient and
scalable offloading of control applications

Page 37 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Local Apps

Page 38 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Local Apps

Page 39 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Where to run the local apps

Page 40 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Kandoo

Page 41 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
An example: Elephant flow rerouting

Page 42 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
An example: Elephant flow rerouting

Page 43 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Kandoo variations

Page 44 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Kandoo summary

2 levels of controller
Deal with the scalability issue by moving software closer to the data plane
Future: a generalized hierarchy
 Filling the gap between local and non-local apps
 Finding the right scope is quite challenging

Page 45 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Devoflow

DevoFlow: scaling flow management for high-performance networks,

SIGCOMM’2011.

OpenFlow is good; but fine-grain per flow management creates too much overhead
 Flow setup
 Statistics collection
Devoflow – a new paradigm to reduce the control and overhead while providing fine
control for important flows.

Page 46 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Dilemma

Control dilemma:
 Role of controller: visibility and mgmt capability
however, per-flow setup too costly
 Flow-match wildcard (existing hardware), hash-based:
much less load, but no effective control
Statistics-gathering dilemma:
 Pull-based mechanism: counters of all flows
full visibility but demand high BW
 Wildcard counter aggregation: much less entries
but lose trace of elephant flows
Aim to strike in between

Page 47 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Main Concept of DevoFlow

Devolving most flow controls to switches

 Use the default wildcard match
Maintain partial visibility
Keep trace of significant flows
Default v.s. special actions:
 Security-sensitive flows: categorically inspect
 Normal flows: may evolve or cover other flows
become security-sensitive or significant
 Significant flows: special attention
Collect statistics by sampling, triggering, and approximating

Page 48 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Design Principles of DevoFlow

Try to stay in data-plane, by default

Provide enough visibility:
 Esp. for significant flows & sec-sensitive flows
 Otherwise, aggregate or approximate statistics
Maintain simplicity of switches

Page 49 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Mechanisms

Control
 Rule cloning
 Local actions
Statistics-gathering
 Sampling
 Triggers and reports
 Approximate counters

Page 50 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Rule Cloning – identify elephant flow

ASIC clones a wildcard rule as an exact match rule for new microflows
 Timeout or output port by probability

Page 51 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Rule Cloning

ASIC clones a wildcard rule as an exact match rule for new microflows
 Timeout or output port by probability

Page 52 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Rule Cloning

ASIC clones a wildcard rule as an exact match rule for new microflows
 Timeout or output port by probability

Page 53 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Local Actions

Rapid re-routing: fallback paths predefined

Recover almost immediately
Multipath support: based on probability dist.
Adjusted by link capacity or loads

Page 54 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 Statistics-Gathering

Sampling
 Pkts headers send to controller with1/1000 prob.
Triggers and reports
 Set a threshold per rule
 When exceeds, enable flow setup at controller
Approximate counters
 Maintain list of top-k largest flows

Page 55 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
 DevoFlow Summary

Per-flow control imposes too many overheads

Balance between
 Overheads and network visibility
 Effective traffic engineering, network management
Switches with limited resources
 Flow entries, control-plane BW
 Hardware capability, power consumption

Page 56 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET
Thank you for your attention!

Page 57 > Software Defined Networks

Dr. Dinh Thi Thai Mai – VNU - UET