Open stack networking vlan, gre

Open stack networking vlan, gre

• 1.
  OpenStack networking - withOpen vSwitch VLAN, GRE Paul Sim Cloud Consultant paul.sim@canonical.com
• 2.
  Index ● Prior Knowledge ●OpenStack Networking - VLAN ● OpenStack Networking - GRE ● Security Group, Floating-IP, NameSpace ● Neutron ML2
• 3.
  Prior Knowledge -Network NameSpace without Network NameSpace Process with Network NameSpace Process Process Process Process Process Process Process Share Routing table Ford NameSpace Benz NameSpace Network Resources Network Resources BMW NameSpace Network Resources Network Resources Address Netfilter rules eth0 eth1 Network Resources eth2 eth0 eth1 eth2 Network NameSpace provides isolation of the system resources associated with networking. Thus, each network namespace has its own network devices, IP addresses, IP routing tables, /proc/net directory, port numbers, and so on. - http://lwn.net/Articles/531114/
• 4.
  Prior Knowledge -VLAN, GRE VLAN - Virtual LAN 802.1Q Header TPIC : 16bit - 0x8100 TCI : 16bit PCP : 3bit DEI : 1bit VID : 12bit (0 ~ 4095) GRE - Generic Routing Encapsulation 16 Bytes Header + IP header Key field : 32bit - identify an individual traffic flow within a tunnel
• 5.
  OpenStack Installation -Grizzly External network 192.168.122.0/24 eth0 eth0 Controller node eth0 Network node Quantum L3-agent Nova Keystone Glance Horizon Quantum openvswitch-agent Quantum metadata-agent Quantum - Server eth1 Compute node - 1 Compute node - 2 Quantum openvswitch-agent Quantum openvswitch-agent Nova compute Nova compute Quantum dhcpagent eth1 eth0 eth2 eth2 eth1 eth2 Management 192.168.20.0/24 Data 192.168.10.0/24 eth1 eth2
• 6.
  Network Topology ● ● ● ● ext_net :external network - 192.168.122.0/24 net_proj_one : “user_one” tenant - 50.50.1.0/24 net_proj_two : “user_one” tenant - 50.50.2.0/24 net_proj_new : “user_new” tenant - 60.60.1.0/24
• 7.
  Big picture -VLAN OpenStack Grizzly OpenvSwitch plug-in VLAN mode Network node net_proj_one net_proj_two Compute node - 1 net_proj_new VM tap~ qr~ tap~ qr~ qg~ qg~ br-ex qg~ VM tap~ tag: 1 qr~ br-int VM tap~ tag:2 tap~ tag:2 tap~ int-br-eth1 phy-br-eth1 br-eth1 int-br-eth1 Data 192.168.10.0 /24 eth1 br-int phy-br-eth1 eth1 br-eth1 eth0 OVS port OVS Bridge ● ● qg~~~ : external gateway interface qr~~~ : virtual router interface
• 8.
  VLAN - Computenode OpenStack Grizzly OpenvSwitch plug-in VLAN mode Compute node - 1 br-eth1 eth1 VM VM VM VM tap~ tag: 1 tap~ tag:2 tap~ tag:2 tap~ tag:3 veth pair phy-br-eth1 int-br-eth1 br-int Packet conversion mod_vlan_vid mod_vlan_vid Security Group[1]
• 9.
  VLAN - Computenode Packet conversion janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-eth1 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=90455.716s, table=0, n_packets=6, n_bytes=468, priority=2,in_port=2 actions=drop cookie=0x0, duration=89606.096s, table=0, n_packets=9484, n_bytes=2312018, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:1024,NORMAL cookie=0x0, duration=90456.248s, table=0, n_packets=6813, n_bytes=1325511, priority=1 actions=NORMAL janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=90458.482s, table=0, n_packets=64, n_bytes=4644, priority=2,in_port=1 actions=drop cookie=0x0, duration=89608.755s, table=0, n_packets=6499, n_bytes=1283680, priority=3,in_port=1,dl_vlan=1024 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=90459.075s, table=0, n_packets=9820, n_bytes=2323195, priority=1 actions=NORMAL openvswitch-agent.log Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-int', 'hard_timeout=0, idle_timeout=0,priority=3,in_port=1,dl_vlan=1024,actions=mod_vl an_vid:1,normal'] Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-ofctl', 'add-flow', 'br-eth1', 'hard_timeout=0, idle_timeout=0,priority=4,in_port=2,dl_vlan=1,actions=mod_vlan _vid:1024,normal']
• 10.
  VLAN - Networknode OpenStack Grizzly OpenvSwitch plug-in VLAN mode Network node tap~ Namespcae tap~ Namespcae qr~ qg~ qr~ qg~ veth pair br-int int-br-eth1 phy-br-eth1 br-ex eth0 Packet conversion net_proj_one mod_vlan_id net_proj_two Floating-IP(NAT) net_proj_new mod_vlan_id eth1 qg~ Namespcae br-eth1 qr~ tap~
• 11.
  VLAN - Networknode Packet conversion janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-int NXST_FLOW reply (xid=0x4): cookie=0x0, duration=7370.307s, table=0, n_packets=6, n_bytes=468, priority=2,in_port=6 actions=drop cookie=0x0, duration=7368.424s, table=0, n_packets=0, n_bytes=0, priority=3,in_port=6,dl_vlan=2048 actions=mod_vlan_vid:2,NORMAL cookie=0x0, duration=7367.991s, table=0, n_packets=764, n_bytes=191460, priority=3,in_port=6,dl_vlan=1024 actions=mod_vlan_vid:3, NORMAL cookie=0x0, duration=7369.073s, table=0, n_packets=0, n_bytes=0, priority=3,in_port=6,dl_vlan=500 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=7370.924s, table=0, n_packets=549, n_bytes=104066, priority=1 actions=NORMAL janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-eth1 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=7373.826s, table=0, n_packets=14, n_bytes=1104, priority=2,in_port=2 actions=drop cookie=0x0, duration=7372.725s, table=0, n_packets=13, n_bytes=922, priority=4,in_port=2,dl_vlan=1 actions=mod_vlan_vid:500,NORMAL cookie=0x0, duration=7371.663s, table=0, n_packets=519, n_bytes=103966, priority=4,in_port=2,dl_vlan=3 actions=mod_vlan_vid:1024, NORMAL cookie=0x0, duration=7372.09s, table=0, n_packets=9, n_bytes=634, priority=4,in_port=2,dl_vlan=2 actions=mod_vlan_vid:2048,NORMAL cookie=0x0, duration=7374.384s, table=0, n_packets=764, n_bytes=191460, priority=1 actions=NORMAL
• 12.
  Big picture -GRE OpenStack Grizzly OpenvSwitch plug-in GRE tunneling Network node qr~ qr~ VM Tunnel gre~ qg~ patch patch br-int qg~ Data 192.168.10.0 /24 tap~ br-tun qr~ tap~ qg~ VM tap~ tag: 1 patch tap~ net_proj_new br-tun net_proj_two gre~ net_proj_one Compute node - 1 tap~ tag:2 patch br-int br-ex eth0 OVS port OVS Bridge ● ● qg~~~ : external gateway interface qr~~~ : virtual router interface
• 13.
  GRE - Computenode OpenStack Grizzly OpenvSwitch plug-in GRE tunneling Compute node - 1 patch VM VM VM tap~ tag: 1 br-tun gre~ VM Tunnel tap~ tag:2 tap~ tag:2 tap~ tag:3 patch br-int Packet conversion mod_vlan_vid set_tunnel id Security Group[1]
• 14.
  GRE - Computenode Packet conversion janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=87770.027s, table=0, n_packets=0, n_bytes=0, priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00: 00 actions=mod_vlan_vid:1,output:1 cookie=0x0, duration=87770.09s, table=0, n_packets=8786, n_bytes=1893724, priority=4,in_port=1,dl_vlan=1 actions=set_tunnel:0x1,NORMAL cookie=0x0, duration=87769.693s, table=0, n_packets=3031, n_bytes=617650, priority=3,tun_id=0x1,dl_dst=fa:16:3e:db:08:63 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=87769.966s, table=0, n_packets=6320, n_bytes=4432680, priority=3,tun_id=0x1,dl_dst=fa:16:3e:e0:73:95 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=87771.753s, table=0, n_packets=2921, n_bytes=951454, priority=1 actions=drop
• 15.
  GRE - Networknode OpenStack Grizzly OpenvSwitch plug-in GRE tunneling Network node tap~ Namespcae tap~ Namespcae qr~ Namespcae qr~ qg~ patch patch br-int br-ex eth0 Packet conversion net_proj_one set_tunnel id net_proj_two Floating-IP(NAT) net_proj_new mod_vlan_id Tunnel gre~ qg~ qr~ br-tun qg~ tap~
• 16.
  GRE - Networknode Packet conversion janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=474674.446s, table=0, n_packets=7899, n_bytes=2572502, priority=3,tun_id=0x3,dl_dst=01:00:00:00:00:00/01:00:00:00:00: 00 actions=mod_vlan_vid:2,output:1 cookie=0x0, duration=473163.123s, table=0, n_packets=7876, n_bytes=2565284, priority=3,tun_id=0x4,dl_dst=01:00:00:00:00:00/01:00:00:00:00: 00 actions=mod_vlan_vid:3,output:1 cookie=0x0, duration=633937.826s, table=0, n_packets=10543, n_bytes=3426814, priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00: 00 actions=mod_vlan_vid:1,output:1 cookie=0x0, duration=473163.329s, table=0, n_packets=16484, n_bytes=3348666, priority=4,in_port=1,dl_vlan=3 actions=set_tunnel:0x4, NORMAL cookie=0x0, duration=474674.541s, table=0, n_packets=16864, n_bytes=3389132, priority=4,in_port=1,dl_vlan=2 actions=set_tunnel:0x3, NORMAL cookie=0x0, duration=633937.905s, table=0, n_packets=62044, n_bytes=37320316, priority=4,in_port=1,dl_vlan=1 actions=set_tunnel:0x1, NORMAL cookie=0x0, duration=472911.069s, table=0, n_packets=16335, n_bytes=3551350, priority=3,tun_id=0x4,dl_dst=fa:16:3e:89:fd:ce actions=mod_vlan_vid:3,NORMAL cookie=0x0, duration=474336.184s, table=0, n_packets=16360, n_bytes=3560332, priority=3,tun_id=0x3,dl_dst=fa:16:3e:d8:d5:29 actions=mod_vlan_vid:2,NORMAL cookie=0x0, duration=474674.351s, table=0, n_packets=525, n_bytes=52427, priority=3,tun_id=0x3,dl_dst=fa:16:3e:69:ca:97 actions=mod_vlan_vid:2,NORMAL cookie=0x0, duration=473162.912s, table=0, n_packets=197, n_bytes=19365, priority=3,tun_id=0x4,dl_dst=fa:16:3e:d6:b8:07 actions=mod_vlan_vid:3,NORMAL cookie=0x0, duration=633937.746s, table=0, n_packets=6207, n_bytes=630043, priority=3,tun_id=0x1,dl_dst=fa:16:3e:c7:ec:bd actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=474794.912s, table=0, n_packets=36912, n_bytes=7440964, priority=3,tun_id=0x1,dl_dst=fa:16:3e:8b:a6:d7 actions=mod_vlan_vid:1,NORMAL cookie=0x0, duration=636252.069s, table=0, n_packets=163, n_bytes=36046, priority=1 actions=drop
• 17.
  Security Group -VLAN, GRE FORWARD quantum-filter-top quantum-openvswi-local Security group is applied here quantum-openvswi-FORWARD quantum-openvswi-sg-chain quantum-openvswi-iTAP_NUMBER quantum-openvswi-sg-fallback quantum-openvswi-oTAP_NUMBER quantum-openvswi-sg-fallback
• 18.
  Security Group -VLAN, GRE Chain quantum-openvswi-sg-chain (4 references) target prot opt source destination quantum-openvswi-i21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 quantum-openvswi-o21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 quantum-openvswi-i7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 quantum-openvswi-o7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap21767f1f-45 --physdev-is-bridged PHYSDEV match --physdev-in tap21767f1f-45 --physdev-is-bridged PHYSDEV match --physdev-out tap7903fd30-74 --physdev-is-bridged PHYSDEV match --physdev-in tap7903fd30-74 --physdev-is-bridged Chain quantum-openvswi-i7903fd30-7 (1 references) target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED RETURN icmp -- 0.0.0.0/0 0.0.0.0/0 RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 RETURN udp -- 50.50.1.3 0.0.0.0/0 udp spt:67 dpt:68 quantum-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0 Chain quantum-openvswi-o7903fd30-7 (2 references) target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:DB:08:63 RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 DROP all -- !50.50.1.2 0.0.0.0/0 DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68 DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED RETURN all -- 0.0.0.0/0 0.0.0.0/0 quantum-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0 [1] Note, OpenStack uses iptables rules on the TAP devices such as “tap~~” to implement security groups,. However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an Open vSwitch port.
• 19.
  Network NameSpace janghoon@Network-node:~$ sudoip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 ifconfig lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 qg-fa243f49-d6 Link encap:Ethernet HWaddr fa:16:3e:9f:4b:63 inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe9f:4b63/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 qr-bc654dc2-f1 Link encap:Ethernet HWaddr fa:16:3e:c7:ec:bd inet addr:50.50.1.1 Bcast:50.50.1.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fec7:ecbd/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default 192.168.122.1 0.0.0.0 UG 0 0 0 qg-fa243f49-d6 50.50.1.0 * 255.255.255.0 U 0 0 0 qr-bc654dc2-f1 192.168.122.0 * 255.255.255.0 U 0 0 0 qg-fa243f49-d6
• 20.
  Floating-IP(NAT) - VLAN,GRE NameSpace janghoon@Network-node:~$ sudo ip netns show qdhcp-4c2f2346-ffaa-41a0-ab76-34cadf0163f5 qrouter-e1b88ce4-51e9-4744-be80-d70d04c6a59b qdhcp-c19e22a0-1700-4b3b-91e5-2c961ef0a353 qrouter-244fff3f-f935-4bdd-949d-739f1ce81dd0 qdhcp-f37b681a-4be8-47b8-8063-3d17d24ee1ae qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 Floating-IP(NAT) janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 iptables -L -n -t nat Chain quantum-l3-agent-PREROUTING (1 references) target prot opt source destination REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697 DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.1.2 Chain quantum-l3-agent-float-snat (1 references) target prot opt source destination SNAT all -- 50.50.1.2 0.0.0.0/0 to:192.168.122.51 Chain quantum-l3-agent-snat (1 references) target prot opt source destination quantum-l3-agent-float-snat all -- 0.0.0.0/0 SNAT all -- 50.50.1.0/24 0.0.0.0/0 0.0.0.0/0 to:192.168.122.50
• 21.
  Neutron ML2 The ModularLayer 2 (ML2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety of layer 2 networking technologies found in complex real-world data centers. It currently works with the existing openvswitch, linuxbridge, and hyperv L2 agents, and is intended to replace and deprecate the monolithic plugins associated with those L2 agents. Neutron ML2 Plugin TypeDriver Cisco Nexus Arista Flat OpenDaylight VxLAN Hyper-V GRE OpenvSwitch VLAN MechanismDriver pSwitch TypeDriver : TypeDrivers maintain any needed type-specific network state, and perform provider network validation and tenant network allocation. MechanismDriver : The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied given the specific networking mechanisms that have been enabled. https://wiki.openstack.org/wiki/Neutron/ML2
• 22.
  Neutron ML2 eth0 eth0 eth0 Network node Computenode - 1 Compute node - 2 Neutron ML2-agent Neutron ML2-agent Nova compute Nova compute Neutron L3-agent Neutron ML2 plugin Neutron metadataagent Neutron dhcpagent eth1 eth2 eth1 eth2 eth1 eth2