Downloaded 1,137 times
More Related Content
Similar to Google Cloud Networking Deep Dive
Google Cloud Networking Deep Dive
- 1. Google Cloud Networking Withoutnetworking, there is no cloud Prajakta Joshi, Product Manager, Google prajaktajoshi@google.com Srinivas Krishnan, Tech Lead/Manager, Google krishnan@google.com April 6th, 2017
- 2. 7 Cloud productswith 1 billion users
- 3. Google’s technology infrastructureand innovations available to all
- 4. Google probably hasthe best networking technology on the planet. — Peter Bakkum, Quizlet “ ”
- 5. FASTER (US, JP,TW) 2016 Unity (US, JP) 2010 SJC (JP, HK, SG) 2013 GCP Infrastructure 6 regions, 18 zones, over 100 points of presence, and a well-provisioned global network comprised of hundreds of thousands of miles of fiber optic cable. Edge points of presence (>100 - all are not shown on this map) Leased and owned fiber # # Future regions and number of zones Current regions and number of zones 3 3 2 3 3 3 3 3 2 4 3 3 2 Frankfurt Singapore S Carolina N Virginia Belgium London Taiwan Mumbai Sydney Oregon Iowa São Paulo Finland Tokyo Montreal California Netherlands 3 3 33 https://peering.google.com https://cloud.google.com/compute/docs/regions-zones/regions-zones
- 6. Software Defined Load Balancer Google Innovations inNetworking Software Defined Network Virtualization Software Defined DataCenters Software Defined WAN 2006 2008 2010 2012 2014 Google Global Cache Freedome Watchtower BwE Onix B4 Jupiter Andromeda Maglev QUIC Espresso Software Defined Edge
- 7. Google Cloud Networking Global Scale Applicationdelivery at scale globally or regionally Cloud Load Balancing Cloud CDN Cloud DNS Global private space, regional segmentation.VPC SDN network virtualization Global Networks Granular Subnetworks Connect to on-premises or another cloud Cloud VPN Cloud Router Cloud Interconnect Hybrid Cloud User control Network IAM roles Firewalls Stackdriver Security Policies Visibility / diagnostics Control
- 8. Global, Flexible VirtualPrivate Cloud (VPC)
- 9. Traditional VPC ● Regional USWest US East Traditional VPC 10.10.0.0/26 Traditional VPC Internet Web Application Server Web Application Server 10.50.0.0/26
- 10. Google Global VPC USWest 10.10.0.0/26 US East 10.50.0.0/26 Traditional VPC Traditional VPC Application Server Application Server Internet US West US East Google VPC 10.10.0.0/26 10.50.0.0/26 Application Server Application Server ● Connect workloads across any regions ● Access any region by interconnecting through a single location, through Google's backbone network Traditional VPC Google VPC
- 11. Subnet 10.10.0.0/20 Project Webserver Project Recommendation Project Analytics Webserver devOps Recommend devOps Security Network Admin OrganizationNode MY-ORG.com Database devOps VM instances VM instances VM instances Customer DC Interconnect (POP) Shared VPC Network (XPN) Private IP connectivity Shared network VPC (XPN)
- 12. VPC Network Peering VPCProducer NetworkVPC Consumer Network Project service-prod Project customer-prod Producer devOps Consumer devOps Consumer Security Network Admin Organization node example.com Serving Instance Compute Engine Cloud Load Balancing Backend Compute Engine Producer Security Network Admin Organization node SaaS.com 12
- 13. GCE VM distributedfirewalls ● Distributed: enforced on host => no choke points ● Ingress and Egress firewall (allow / deny rules) ● Stateful with Connection tracking ● Flexible grouping mechanism for applications: tagsGCP Firewall Traditional enterprise model Ingress firewall Egress firewallIngress firewall Internet Google Cloud model Compute Engine Compute Engine Compute Engine Compute Engine Red-tag Blue-tag
- 14. Scale, resiliency andoptimized app delivery across the globe
- 15. Google Global Loadbalancing HTTP(S) Load Balancing SSL proxy Network TCP/UDP Load Balancing Internal TCP/UDP Load Balancing Global Regional TCP proxy Delivered using Google Front End infrastructure (GFE)Delivered using Maglev Delivered using Andromeda ● Google Front Ends (GFEs)= Software-defined, distributed systems that sit in Google POPs and perform global load balancing in conjunction with other systems/control planes ● Andromeda = Google Cloud’s software-defined network virtualization stack ● Maglev = Distributed systems for network load balancing
- 16. GFE Your Backend compute Google’s globalhigh-quality network ISP Network Global LB Software-defined and globally distributed load balancing GFE VM VM VM Global LB Load Balancer GCP Datacenters Google Edge POP Google Network Google Edge POP Serving Instances Compute Engine Autoscaling Software-defined, globally distributed load balancing
- 17. Google Global HTTP(S)Load Balancing Maya in California 2001:db8::2 Bob in London 2001:db8::3 Shen in Singapore 2001:db8::4 Cloud DNS www.myapp.com 2001:db8::10 120.1.1.1 Region: US-West Serving Instances Compute Engine Autoscaling Region: Europe Serving Instances Compute Engine Autoscaling Region: Asia Serving Instances Compute Engine Autoscaling 10.240.0.0/16 192.168.0.0/16 10.2.0.0/16 Google Network www.myapp.com www.myapp.com www.myapp.com www.myapp.com 2001:db8:10 120.1.1.1 Google Edge POP Google Edge POP Google Edge POP 2001:db8:10 120.1.1.1 2001:db8:10 120.1.1.1 Google Global Load Balancing (IPv4/IPv6)
- 18. Google HTTP(S) loadbalancing performance 0 150 300 450 600 1,600,000 Multiple Regions (europe-west1-a, europe-west1-b, asia-east1-a) RPS Time (s) 1,200,000 800,000 400,000 0 1.3 million requests per second 50X Actual Traffic was of expected traffic Handled with:
- 19. Optimizing with ContentDelivery Networks (CDN) Google Cloud CDN
- 20. Google Cloud CDN Userin San Francisco Frontend Frontend US-Central Region Asia Region US-East region Google Cloud Storage Cloud CDN User in Iowa User in New York Stackdriver Monitoring & Logging Autoscaling Autoscaling Cloud Load Balancing HTTP(S) LB Cache hit! ● 80+ locations ● Single IP across multiple regions ● Enable CDN for HTTP(S) with single check box ● Caches content from instances and storage buckets ● Cache invalidation ● Custom cache keys ● No extra charge for SSL (TLS traffic) ● Industry leading performance (Cedexis)
- 21. Google Cloud CDN:latency performance View Cedexis data on CDN latency, throughput, availability at https://www.cedexis.com/google-reports/ *Lower is better
- 22.
- 23. Google Cloud Interconnect CarrierInterconnect Enterprise-grade connection through a large partner network of service providers, VPN required for RFC 1918 communication VPN Secure multi-Gbps connection over VPN tunnels Direct Peering Enterprise-grade connection between you and Google for your hybrid cloud workloads, VPN required for RFC 1918 communication Private Interconnect (new) Connect N X 10G transport circuits for private cloud traffic to Google Cloud at Google POPs, VPN not required for RFC 1918 communication
- 24. Hybrid cloud: TheHome Depot Home Depot Data Centers VPN Gateway Local Storage Local Compute Google Cloud Interconnect / VPN VPN Gateway Compute Engine Cloud Storage Local Compute API Gateway HTTP Cloud Load Balancing Compute Engine Google Cloud Interconnect ( Mutual TLS ) Learn more in Ravi’s talk (The Home Depot): RFC 1918 Public IPs
- 25. Hybrid/Multi-cloud automation Internal loadbalancing templates Google Cloud provider
- 26.
- 27. Cloud networking securityblueprint Secure VPC Cloud Interconnect 3rd party virtual appliances Google Global Load Balancer (GFE) VPC topologies for isolation, Distributed firewall, IAM, Private access to Google services, Bastion hosts Built-in L3/L4 DDoS protection, Intelligent anycast for Global IPv6 and IPv4 LB, Autoscaling, Cross-region overflow, Cross-region failover, SSL (TLS) termination Google Network Direct Peering, Carrier Interconnect, VPN, Private Interconnect Next-gen firewall, Logging, Monitoring, Compliance Cloud Interconnect 3rd party virtual appliances Google Global Load Balancer (GFE) Google Network High capacity/High performance global network, Protection against UDP-based attacks 3rd party DDoS Defense 27
- 28. Leverage 3rd partyappliances and services Network Security (Next Gen Firewall/IPS/WAF) DDoS/WAF Endpoint/ Container Security Scanning, Logging & Compliance Encryption/ DLP TO DO: FIX ME Evernote users trust us with billions of their notes, so the security and privacy of those notes is top of mind as we move to the cloud. Google Cloud Platform security capabilities let us match the protections we provide for customers today, and improve on them. Rich Tener, Head of Security at Evernote “ ”https://blog.evernote.com/blog/2017/02/08/evernote-reaches-the-cloud/ 28
- 29. Under the Hood: AndromedaNetwork Virtualization Srinivas Krishnan Technical Lead, Google
- 30. Andromeda ● Scalable, Flexible,Secure, High Performance Infrastructure for Virtual Networking ● Powers Google Compute Engine Network
- 31. FASTER (US, JP,TW) 2016 Unity (US, JP) 2010 SJC (JP, HK, SG) 2013 Edge points of presence (>100 - all are not shown on this map) Leased and owned fiber # # Future regions and number of zones Current regions and number of zones 3 3 2 3 3 3 3 3 2 4 3 3 2 Frankfurt Singapore S Carolina N Virginia Belgium London Taiwan Mumbai Sydney Oregon Iowa São Paulo Finland Tokyo Montreal California Netherlands 3 3 33 GCP Network at a Glance https://peering.google.com https://cloud.google.com/compute/docs/regions-zones/regions-zones
- 32. Belgium Finland Frankfurt *SLA availabilitystatements are achievement targets and are subject to change and per Google terms and conditions https://peering.google.com https://cloud.google.com/compute/docs/regions-zones/regions-zones Singapore S Carolina London Taiwan Mumbai Oregon Iowa Tokyo São Paulo vnet-1: 10.1/16 vnet-2: 10.2/16 vnet-3: 10.2.1/24 Setting the Stage (Control Plane) ● Isolation across virtual networks ● Built for scale ○ 100k VMs in 184 ms1 1 median ● High Availability Targets* ○ 99.9% Single Zone ○ 99.99% Single Region ○ 99.999% Globally
- 33. Setting the Stage(Data Plane) Core network functions Guest VM Dataplane Host Mgmt Control Plane Guest VM Dataplane Host Mgmt Constantly Adding new functions ● Provide core network functions ● Constantly evolving new features ○ Supported by Live Migration and Dataplane Hitless Upgrades
- 34. Frankfurt https://peering.google.com https://cloud.google.com/compute/docs/regions-zones/regions-zones Singapore S Carolina N Virginia Belgium London Taiwan Mumbai Sydney Oregon Iowa SãoPaulo Finland Tokyo Overview Cluster Manager Regional Fabric Manager Google Fabric Manager API Google Fabric Manager API Google Fabric Manager API Regional Fabric Manager VM Controller VM Controller VM ControllerVM Controller OFE OFE OFE OFE Google Fabric Manager API VM Coprocessor Guest VM Open vswitch Host Mgmt Andromeda Dataplane Guest VMGuest VM VM Coprocessor Guest VM Open vswitch Host Mgmt Andromeda Dataplane Guest VMGuest VM VM Coprocessor Guest VM Open vswitch Host Mgmt Andromeda Dataplane Guest VMGuest VM VM Coprocessor Guest VM Open vswitch Host Mgmt Andromeda Dataplane Guest VMGuest VM
- 35. Providing High Availabilityand Scaling Fabric Manager VMC OFE VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM VMCVMC Replicated VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM Sharded VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM Horizontal scaling VMCVMCVMC OFE VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM VM Coprocessor Open vswitch Host Mgmt Andromeda Dataplane VMVM VM Sharded VMCVMCVMC OFE
- 36. Network Programming Models ●Pre-Programmed Model ○ Programs a full mesh of VM-VM connectivity ○ Works well for small virtual networks VM Control Plane VM VM Hoverboard VM ● On-Demand Model ○ Traditional OpenFlow learning packet model ○ First Packet always misses to controller ● Google Hybrid Model: Hoverboards ○ Uses a software gateway ○ Pre-Program small networks ○ Larger Networks ■ Packet goes through Hoverboards ■ Flows can be offloaded to the VM hosts
- 37. Guest OS Hypervisor Openvswitch Hardware NIC Andromeda1.0 Guest OS Userspace NIC Hardware NIC Hypervisor Andromeda 2.1 Bypass to Userspace NIC Guest OS Hypervisor Userspace NIC Hardware NIC Andromeda 2.0 Dataplane in Userspace Andromeda 1.5 Guest OS Hypervisor Openvswitch Hardware NIC Offloads + Live Migration Guest OS Userspace NIC Hardware NIC Hypervisor Hypervisor Bypass Future Andromeda Dataplane Quick History
- 38. Andromeda Dataplane Enabling NativeHw Performance in Software ● Fast Path ○ 2 million pps VM1 Guest VM vnic TX vnic RX Fastpath Egress Engine Ingress Engine VM2 Guest VM vnic TX vnic RX Coprocessor Coprocessor NIC vswitchd ctrl vswitchd ● Coprocessors ○ CPU Intensive functions ● On Host control plane
- 39. Unique Userspace Architecture ●Secure VM1 Guest VM vnic TX vnic RX VM2 Guest VM vnic TX vnic RX NIC ● Improved Robustness ● Rapid Releases vswitchd Andromeda Dataplane Fastpath Egress Engine Ingress Engine Coprocessor Coprocessor vswitchd ctrl Andromeda Dataplane Fastpath Egress Engine Ingress Engine Coprocessor Coprocessor vswitchd ctrl NO ROOTContinuous FuzzingASLR Andromeda Dataplane Fastpath Egress Engine Ingress Engine Coprocessor Coprocessor vswitchd ctrl
- 40. Fully Programmable Dataplane Openvswitch Google Extensions LoadBalancing Stats and Billing Policy Enforcement Table-0: Decap Table-0: Decap Table-50: Encap Flow Table Flow Key Action Coprocessors Traffic Shaper DoS and Abuse New Features vswitchd ctrl NIC In port=5, src ip=1.2.3.4 Encap, out port=3229 In port=470, Encap Decap, out port=5 ● Fully programmable from control plane VM Controller ● Programmable flow table ● Custom Google extension framework ● Easy Network function implementation
- 41. Case-Study: Internal LoadBalancing
- 42.
- 43. No Middle Box LoadBalancer Backend Backend Backend VIP Traditional Load Balancers Client VM Load Balancer Backend Backend Client VM Load Balancer Backend GCP Internal Load Balancing Client VMClient VM
- 44. Load Balancer inClient as Network Function VM Controllers OFEs Health Reporter Health CheckerClient VM Load Balancer Backend VMBackend VM Cluster Fabric Manager Control Health VM-VM OpenFlow
- 45.
- 46. Your toolkit ismuch bigger actually... Container Engine KubernetesCompute Engine App Engine Cloud Load Balancing Cloud CDN Cloud Interconnect Cloud Functions Global, Secure Network Cloud VPC Stackdriver And many many more ...
- 47. Cloud Networking: KeyTakeaways Google’s global, software-defined network: the underpinnings of high performance, flexibility, control, and security that Google Cloud provides Google Cloud’s secure VPC: Global VPC, choice of topologies (org, shared VPC, peered VPC…), distributed firewall, IAM, secure connectivity (Private Interconnect, Direct Peering, Carrier Interconnect, Cloud VPN) Google Cloud’s best-of-breed network services: High performance Global Load Balancing, Private Internal Load Balancing, High performance Cloud CDN, Cloud DNS. Google’s technical infrastructure is built for the cloud and employs a layered security model to secure the entire stack to address stringent regulatory and enterprise security needs. 1 2 3 4
- 48.