BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE, PILANI
WORK INTEGRATED LEARNING PROGRAMMES
COURSE HANDOUT
Part A: Content Design
Course Title Scalable Services
Course No(s) SE ZG583
Credit Units 5
Course Author Akanksha Bharadwaj
Version No 2.0
Date May 2022
Course Objectives
No Course Objective
CO1 Build competence to design, develop, implement and manage scalable information
systems
CO2 Gain understanding of different techniques & tools for building and managing
scalable services
CO3 Gain understanding of challenges and best practices in creating and managing
scalable services
Text Book(s)
T1 Microservices patterns by Chris Richardson, Manning Publications 2018
T2 Microservices in action by Morgan Bruce & Paulo Pereira, Manning Publications 2018
T3 Building Microservices by Sam Newman, O'Reilly Media 2015
Reference Book(s) & other resources
R1 https://kubernetes.io/docs/concepts/
R2 The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the
Modern Enterprise, Second Edition by Michael T. Fisher; Martin L. Abbott Published by
Addison-Wesley Professional, 2015
R3 Scalability patterns by Microsoft Azure:
https://docs.microsoft.com/en-us/azure/architecture/patterns/category/performance-
scalability
R4 Scalability Patterns: Best Practices for Designing High Volume Websites by Chander
Dhall
Modular Content Structure
1. Getting to know Scalability:
1.1. Introduction to Performance, Consistency and availability
1.2. What is scalability?
1.3. Need for scalable architectures
1.4. Principles of Scalability
1.5. Guidelines for Building Highly Scalable Systems
1.6. Architecturally scalable requirements
1.7. Challenges for Scalability
1.8. YouTube case Study
2. Popular scaling approaches
2.1. Managing & processing high volumes of data
2.1.1. Partitioning and sharding
2.1.2. Distributed data (CAP theorem, NoSQL, HDFS)
2.1.3. Distributed Processing (Map reduce, Spark)
2.2. Managing high velocity data streams (Kafka)
2.2.1. Video streaming: Netflix, YouTube, use of CDN
2.2.2. Real time analytics: Credit card fraud detection
2.2.3. Web conferencing: WebEx, Zoom
2.2.4. Edge computing: IoT systems
2.3. Managing high volume transactions
2.3.1. Service Replicas & load balancing
2.3.2. Minimizing event processing: Command Query Responsibility Segregation (CQRS)
2.3.3. Asynchronous communication
2.3.4. Caching techniques: Distributed cache, global cache
2.4. Scalability features in the Cloud (AWS, Azure, Google)
2.4.1. Auto-scaling
2.4.2. Horizontal and vertical scaling
2.4.3. Use of Load balancers
2.4.4. Virtualization
2.4.5. Serverless computing
2.5. Best Practices for Achieving Scalability
3. Microservices - Introduction
3.1. Challenges with Monolith applications
3.2. Microservices architecture
3.3. Advantages and disadvantages of Microservices
3.4. Process & organization for microservices
4. Decomposition strategies
4.1. Decomposition by business capability
4.2. Decomposition by business domain
4.3. Decomposition guidelines
4.4. Obstacles to decomposing an application into services
4.5. Defining service APIs
5. Communication between Microservices
5.1. Inter-service communication
5.1.1. Synchronous communication (REST, gRPC)
5.1.2. Asynchronous communication
5.2. Application boundary
5.2.1. API gateway
5.2.2. API design
5.2.3. Creating and versioning APIs
5.2.4. API security
5.2.5. Backends for frontends
6. Transaction management
6.1. Distributed transactions
6.2. Implementation
6.3. Challenges
6.4. Solutions
6.5. Sagas
7. Building with pipelines
7.1. Continuous integration
7.2. Tooling
7.3. Repository patterns – Multi-repo, mono-repo
8. Designing reliable microservices
8.1. Sources of failure, cascading failures
8.2. Designing reliable communication: Retires, async. Comm., circuit breakers
8.3. Maximizing service reliability: Load balancing, Rate limiting (Queues, Throttling)
8.4. Service mesh
9. Securing and Testing scalable services
9.1. Securing code and repositories
9.2. Using Authentication and Authorization
9.3. Unit testing
9.4. Integration testing
9.5. Load testing
10. Deploying Microservices
10.1. Service startup
10.2. Running multiple instances
10.3. Adding load balancer
10.4. Service to host models
10.4.1. Single Service Instance to Host
10.4.2. Multiple static Service Per Host
10.4.3. Multiple scheduled services per host
10.5. Deploying services without downtime: Canaries, Blue-Green, & rolling deploys
10.6. Deploying microservices using Serverless deployment
11. Deployment with Containers
11.1. Introduction to containers
11.2. Containerizing a service
11.3. Deploying to a cluster
12. Monitoring
12.1. Golden signals
12.2. Types of metrics
12.3. Recommended practices
12.4. Collecting metrics
12.5. Instrumenting
12.6. Raising sensible & actionable alerts
12.7. Using logs & traces
12.8. Useful info in log entries
12.9. Tools for logging
12.10. Logging the right information
12.11. Tracing interaction between services
12.12. Visualizing traces
13. Kubernetes
13.1. Dockers for CaaS
13.2. What is Kubernetes
13.3. Deployment of Microservices using Kubernetes
13.4. Scalability in Kubernetes
13.5. Security in Kubernetes
13.6. CI/CD using Kubernetes
13.7. Kubernetes Dashboard
Learning Outcomes:
No Learning Outcomes
LO1 Understanding of different scenarios where scaling is needed
LO2 Understanding of different approaches to scaling
LO3 Understanding of microservices technology
LO4 Ability to design, develop and deploy microservices based applications
LO5 Understanding of ways to monitor and manage Microservices
LO6 Confident in using tools used in building scalable services
Part B: Contact Session Plan
Academic Term First Semester 2024-2025
Course Title Scalable Services
Course No SE ZG583
Lead Instructor SOMA SUNDARAM P
Course Contents
Contact List of Topic Title Topic # Text/Ref
Session (from content structure in Part A) (from content Book/external
structure in resource
Part A)
1 Getting to know Scalability 1 R2 and R4
Introduction to Performance, Consistency
and availability
What is scalability?
Need for scalable architectures
Principles of Scalability
Guidelines for Building Highly Scalable
Systems
Architecturally scalable requirements
Challenges for Scalability
YouTube case Study
2 Popular scaling approaches 2.1 and 2.2 R2
Managing & processing high volumes of
data
Partitioning and sharding
Distributed data (CAP theorem,
NoSQL, HDFS)
Distributed Processing (Map
reduce, Spark)
Managing high velocity data streams
(Kafka)
Video streaming: Netflix,
YouTube, use of CDN
Real time analytics: Credit card
fraud detection
Web conferencing: WebEx,
Zoom
Edge computing: IoT systems
3 Popular scaling approaches 2.3, 2.4 and R2
Managing high volume transactions 2.5
Service Replicas & load balancing
Minimizing event processing:
Command Query Responsibility
Segregation (CQRS)
Asynchronous communication
Caching techniques: Distributed
cache, global cache
Scalability features in the Cloud (AWS,
Azure, Google)
Auto-scaling
Horizontal and vertical scaling
Use of Load balancers
Virtualization
Serverless computing
Best Practices for Achieving Scalability
4 Microservices - Introduction 3.1 and 3.2 T1
Challenges with Monolith applications
Microservices architecture
Netflix case study
5 Microservices - Introduction 3.3, 3.4, 4.1 T1
Advantages and disadvantages of and 4.2
Microservices
Process & organization for Microservices
Decomposition strategies
Decomposition by business capability
Decomposition by business domain
6 Decomposition strategies 4.3, 4.4, 4.5 T1
Decomposition guidelines and 5.1
Obstacles to decomposing an application
into services
Defining service APIs
Communication between Microservices
Inter-service communication
o Synchronous communication
(REST, gRPC)
o Asynchronous communication
7 Communication between Microservices 5.2 and 6 T1
Application boundary
o API gateway
o API design
o Creating and versioning APIs
o API security
o Backends for frontends
Transaction management
Distributed transactions
Implementation
Challenges
Solutions
Sagas
8 Review and discussion
9 Building with pipelines 7 T3
Continuous integration
Tooling
Repository patterns – Multi-repo, mono-
repo
10 Designing reliable Microservices 8 T2
Sources of failure, cascading failures
Designing reliable communication: Retires,
async. Comm., circuit breakers
Maximizing service reliability: Load
balancing, Rate limiting (Queues,
Throttling)
Service mesh
11 Securing and Testing scalable services 9 T1 and T3
Securing code and repositories
Using Authentication and Authorization
Unit testing
Integration testing
Load testing
12 Deploying microservices 10.1, 10.2, T2
o Service startup 10.3 and 10.4
o Running multiple instances
o Adding load balancer
o Service to host models
o Single Service Instance to Host
o Multiple static Service Per Host
o Multiple scheduled services
per host
13 Deploying microservices 10.5, 10.6 and T2
o Deploying services without downtime: 11
Canaries, Blue-Green, & rolling deploys
o Deploying microservices using Serverless
deployment
Deployment with Containers
Introduction to containers
Containerizing a service
Deploying to a cluster
14 Monitoring 12 T2
Golden signals
Types of metrics
Recommended practices
Monitoring
Collecting metrics
Instrumenting
Raising sensible & actionable alerts
Using logs & traces
Useful info in log entries
Tools for logging
Logging the right information
Tracing interaction between services
Visualizing traces
15 Kubernetes 13 R1 and T3
Dockers for CaaS
What is Kubernetes
Deployment of Microservices using
Kubernetes
Scalability in Kubernetes
Security in Kubernetes
CI/CD using Kubernetes
Kubernetes Dashboard
16 Review and demonstration of Microservices based
application
Case studies:
1. YouTube: https://www.womenwhocode.com/blog/youtube-system-architecture
2. Netflix: https://netflixtechblog.com/tagged/cloud-architecture
3. Amazon Prime: https://aws.amazon.com/solutions/case-studies/amazon-prime-video/ ,
https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-
video-monitoring-service-and-reducing-costs-by-90
4. Google: https://highscalability.com/google-architecture/
5. Facebook: https://highscalability.com/facebook-an-example-canonical-architecture-for-
scaling-billi/ , https://medium.com/swlh/an-introduction-to-facebooks-system-
architecture-47cfcf597101
Labs:
1. Design and develop a Microservices based application
2. Attach a database to a service and perform basic CRUD operations
3. Exploring the communication between services by using shared database pattern
4. Exploring the communication using RabbitMQ
5. Deploying the application using Docker desktop
6. Configuring Minikube and running a local cluster
7. Deploying application on Minikube
Evaluation Scheme
Evaluation Name Type Weight Duration Day, Date, Session,
Component (Quiz, Lab, Project, (Open book, Time
Midterm exam, End Closed book,
semester exam, etc) Online, etc.)
EC – 1 Quiz 1 5% September 1-10,
2024
Quiz 2 5% October 10-20, 2024
Lab (exploring different 10% To be announced
tools)
Assignment (end to end 10% To be announced
app development)
EC – 2 Mid-term Exam Closed book 30% 2 hours Saturday,
21/09/2024 (FN)
EC – 3 End Semester Exam Open book 40% 2 ½ hours Saturday,
30/11/2024 (FN)
Note - Evaluation components can be tailored depending on the proposed model.
Important Information
Syllabus for Mid-Semester Test (Closed Book): SL No. 1 to 8
Syllabus for Comprehensive Exam (Open Book): All topics given in plan of study
Evaluation Guidelines:
1. EC-1 consists of either two Assignments or three Quizzes. Announcements regarding the
same will be made in a timely manner.
2. For Closed Book tests: No books or reference material of any kind will be permitted.
Laptops/Mobiles of any kind are not allowed. Exchange of any material is not allowed.
3. For Open Book exams: Use of prescribed and reference text books, in original (not
photocopies) is permitted. Class notes/slides as reference material in filed or bound form is
permitted. However, loose sheets of paper will not be allowed. Use of calculators is
permitted in all exams. Laptops/Mobiles of any kind are not allowed. Exchange of any
material is not allowed.
4. If a student is unable to appear for the Regular Test/Exam due to genuine exigencies, the
student should follow the procedure to apply for the Make-Up Test/Exam. The genuineness
of the reason for absence in the Regular Exam shall be assessed prior to giving permission to
appear for the Make-up Exam. Make-Up Test/Exam will be conducted only at selected exam
centers on the dates to be announced later.
It shall be the responsibility of the individual student to be regular in maintaining the self-study
schedule as given in the course handout, attend the lectures, and take all the prescribed evaluation
components such as Assignment/Quiz, Mid-Semester Test and Comprehensive Exam according to
the evaluation scheme provided in the handout.
