Uploaded byC4Media
Scaling Patterns for Netflix's Edge
The document discusses scaling patterns for Netflix's edge presented by senior software engineer Justin Ryan in November 2019. It highlights various scaling strategies, trade-offs, and specific use cases related to system design and performance optimization. The content also emphasizes the importance of debuggability, managing infrastructure, and leveraging existing resources while developing scalable systems.
More Related Content
Similar to Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
- 1. Justin Ryan, SeniorSoftware Engineer @ Netflix Twitter: quidryan, LinkedIn: www.linkedin.com/in/justin-charles-ryan Scaling Patterns for Netflix’s Edge November 2019
- 2. InfoQ.com: News &Community Site • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ netflix-edge-scalability-patterns/
- 3. Purpose of QCon -to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
- 4. Scaling Patterns forNetflix’s Edge Justin Ryan Laundry
- 5. Scaling Patterns forNetflix’s Edge Justin Ryan Laundry
- 6. Scaling Patterns forNetflix’s Edge Justin Ryan Laundry
- 7. Scaling Patterns forNetflix’s Edge Justin Ryan Laundry
- 8. Scaling Patterns forNetflix’s Edge Justin Ryan Space-Time Compromise
- 9. Scaling Patterns forNetflix’s Edge Justin Ryan Laundry
- 10. Scaling Patterns forNetflix’s Edge Justin Ryan Scaling Q32011 Q32019
- 11. Scaling Patterns forNetflix’s Edge Justin Ryan Scaling Edge
- 12. Scaling Patterns forNetflix’s Edge Justin Ryan Scaling Problems — Debuggability — Infrastructure — Managing
- 13. Scaling Patterns forNetflix’s Edge Justin Ryan Scalability
- 14. Scaling Patterns forNetflix’s Edge Justin Ryan Trade Offs
- 15. Scaling Patterns forNetflix’s Edge Justin Ryan Policies
- 16. Scaling Patterns forNetflix’s Edge Justin Ryan Accessible
- 17. Scaling Patterns forNetflix’s Edge Justin Ryan Five Use Cases — Melnitz — Mantis — Passport — Device Types — Sharding
- 18. Scaling Patterns forNetflix’s Edge Justin Ryan Logged In
- 19. Scaling Patterns forNetflix’s Edge Justin Ryan Cookies
- 20. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz Trust Cookie Cookie Expiration
- 21. Scaling Patterns forNetflix’s Edge Justin Ryan Cookies L7 Proxy Account Service API Expired Yes No
- 22. Scaling Patterns forNetflix’s Edge Justin Ryan Boom
- 23. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz Copyright Columbia Pictures
- 24. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz Set<CustomerId> Cookie Expiration
- 25. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz Bloom Filter: Compact Set which can answer the question is an object “Possibly in Set”?
- 26. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz
- 27. Scaling Patterns forNetflix’s Edge Justin Ryan Side Note: Bloom Filter — Bad Interview Question? Yes — Used in complex databases? Yes — Guava Library? Yes — Usable by you in your day-to-day jobs? Yes
- 28. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz L7 Proxy Account Service API Bloom Filter Expired Yes No Maybe No
- 29. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz Account Service Auth Event Melnitz A DB Melnitz B 1m L7 Proxy
- 30. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz
- 31. Scaling Patterns forNetflix’s Edge Justin Ryan Melnitz Trade Offs — Probabilistic Data Structure — Off the Shelf Components
- 32. Scaling Patterns forNetflix’s Edge Justin Ryan Side Note: Bloom Filter
- 33. Scaling Patterns forNetflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis — Passport — Device Types — Sharding
- 34. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis
- 35. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis 2 million request per second * 3 kilobytes per request * 10 micro-services = 57 GB/s =~ 4.6 PB / day
- 36. Scaling Patterns forNetflix’s Edge Justin Ryan Naive Approach Service Elastic Search Kibana
- 37. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis
- 38. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis Master Mantis Service Mantis Agent Mantis Source Job Elastic Search Kibana Kafka
- 39. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis Master Service Mantis Mantis Source Job Elastic Search Browser Mantis Agent Kafka
- 40. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis Master Mantis Service Mantis Agent Mantis Source Job Elastic Search Browser Metrics Javascript Transform
- 41. Scaling Patterns forNetflix’s Edge Justin Ryan Avoid Work
- 42. Scaling Patterns forNetflix’s Edge Justin Ryan Mantis Trade Offs — Mantis Query Language — Scale Solution to Zero
- 43. Scaling Patterns forNetflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport — Device Types — Sharding
- 44. Scaling Patterns forNetflix’s Edge Justin Ryan Passport
- 45. Scaling Patterns forNetflix’s Edge Justin Ryan Service Plan API L7Proxy Mid Tier A Mid Tier B Mid Tier C Passport
- 46. Scaling Patterns forNetflix’s Edge Justin Ryan Passport Service Plan API L7Proxy Mid Tier A Mid Tier B Mid Tier C
- 47. Scaling Patterns forNetflix’s Edge Justin Ryan Cache Service Plan API L7Proxy Mid Tier A Mid Tier B Mid Tier C
- 48. Scaling Patterns forNetflix’s Edge Justin Ryan Cache API L7Proxy Mid Tier A Mid Tier B Mid Tier C Service Plan
- 49. Scaling Patterns forNetflix’s Edge Justin Ryan Cache API L7Proxy Mid Tier A Mid Tier B Mid Tier C Service Plan
- 50. Scaling Patterns forNetflix’s Edge Justin Ryan Passport
- 51. Scaling Patterns forNetflix’s Edge Justin Ryan Passport Trade Offs — Data Passing — Heavy Data Structures
- 52. Scaling Patterns forNetflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport - Heavy Data Structure — Device Types — Sharding
- 53. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types
- 54. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types NFAPPL-01-IPAD7=5-09BEC57F64CA5F837C75F924B94D NFAPPL-01-IPAD3=4-15B2D9B33A994CC373017A9C6586 NFAPPL-02-IOS2=4-FT7WHG8H2UR2MCJ4YKR3P0DFH6 NFANDROID1-PRV-P-GOOGLPIXEL=2-7351-0F441DF98B8D99074 NFANDROID1-PRV-4339-0E50FC7582396F5C4496ED935F2F48ED 1395 1007 964
- 55. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types
- 56. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types
- 57. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types Mid-Tier F Mid-Tier E Mid-Tier D Mid-Tier C Mid-Tier BMid-Tier A DTS
- 58. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types Mid-Tier CMid-Tier BMid-Tier A DTS PubSub
- 59. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types Mid-Tier CMid-Tier BMid-Tier A DTS PubSub Device Types Device TypesDevice Types
- 60. Scaling Patterns forNetflix’s Edge Justin Ryan Device Types
- 61. Scaling Patterns forNetflix’s Edge Justin Ryan Device Type Trade Offs — Business Tuned Fallbacks — Leverage Existing Infrastructure
- 62. Scaling Patterns forNetflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport - Heavy Data Structure — Device Types - Device Grouping — Sharding
- 63. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding
- 64. Scaling Patterns forNetflix’s Edge Justin Ryan MSL — Message Security Layer == MSL — Secure Messaging Framework
- 65. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding MSL L7Proxy APIAPIAPIAPIAPIAPI
- 66. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding MSL L7Proxy MSL APIAPIAPI
- 67. Scaling Patterns forNetflix’s Edge Justin Ryan CPU Cost Per Request - CPU to RPS ratio - Load Average 31% decrease
- 68. Scaling Patterns forNetflix’s Edge Justin Ryan Latency 30% decrease 20% p99 decrease
- 69. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding MSL
- 70. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding MSL - % of Wall Clock in GC - GC Pause Time
- 71. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding MSL - Average Latency - 99% Latency
- 72. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding 3 = 1 + 1
- 73. Scaling Patterns forNetflix’s Edge Justin Ryan Sharding Trade Offs — Operational Overhead
- 74. Scaling Patterns forNetflix’s Edge Justin Ryan Five Use Cases — Melnitz - Distributed Bloom Filter — Mantis - Event Querying — Passport - Heavy Data Structure — Device Types - Device Grouping — Sharding - Runtime Refactoring
- 75. Scaling Patterns forNetflix’s Edge Justin Ryan Trough
- 76. Scaling Patterns forNetflix’s Edge Justin Ryan Trough - Used Instances - Reserved Instances
- 77. Scaling Patterns forNetflix’s Edge Justin Ryan Pattern #6 Trough — Leverage Unused Resources
- 78. Scaling Patterns forNetflix’s Edge Justin Ryan Trade Odds ● Probabilistic Data Structure ● Off the Shelf Components ● Avoid Work ● Scale Solution to Zero ● Data Passing ● Heavy Data Structures ● Leverage Existing Infrastructure ● Business Tuned Fallbacks ● Runtime Refactoring Melnitz Melnitz Mantis Mantis Passport Passport DTS DTS Sharding
- 79. Scaling Patterns forNetflix’s Edge Justin Ryan When to Use Debuggability ✗ ✓ ✓ ✓ ✓ ✗ Infrastructure ✓ ✗ ✗ ✓ ✗ ✗ Managing ✗ ✗ ✗ ✓ ✗ ✗ Costs ✓ ✓ ✗ ✗ ✓ ✓ Melnitz Mantis Passport DeviceTypesSharding Trough
- 80. Scaling Patterns forNetflix’s Edge Justin Ryan Bonus Trade Offs — Do the laundry
- 81.
- 82. Scaling Patterns forNetflix’s Edge Justin Ryan References — github.com/Netflix ◆ eureka ◆ zuul ◆ mantis ◆ msl — Spinnaker — Apache Pulsar — Google Guava (Bloom Filter)
- 83. Watch the videowith slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ netflix-edge-scalability-patterns/