Build and
Deploy ML
Models
Module 10
Why Deploy?
• Make models accessible: Once
trained, models need to be deployed to be
1. used in real-world
applications. Deployment allows you to
Understandi integrate models into web
applications, APIs, or standalone systems.
ng Model • Automate predictions: Deployed
models can automate tasks that
Deployment previously required manual
evaluation. This can increase efficiency
and consistency.
• Monitor and improve
models: Deployment allows you to
monitor model performance in production
and gather data for further improvement.
Deployment Options
Cloud Platforms: Popular options include AWS
SageMaker, Google Cloud AI Platform, and Microsoft Azure
Machine Learning. These platforms provide managed services for
deploying, scaling, and managing models.
On-premise Servers: This approach offers more control over
the environment but requires dedicated infrastructure and
maintenance.
Containerization: Frameworks like Docker can package models
and dependencies into containers for easy deployment across
different environments.
Challenges and
Considerations
• Scalability: Deployed models should be able
to handle varying workloads and data volumes.
• Security: Secure model access and data
handling is crucial to prevent unauthorized use
or data breaches.
• Monitoring and Logging: Monitor model
performance and track errors to maintain
model effectiveness.
• Interpretability: If model decisions need to
be explained, consider deploying models with
built-in interpretability techniques.
• Versioning and Management: Track model
versions and deployments for easier rollbacks
and updates.
2. Flask for Deploying ML Models
Flask is a lightweight
web application
framework in Python. It
offers flexibility and
Flask Introduction:
control over application
structure, making it
suitable for deploying ML
models as web services.
Setting Up a Flask App:
Create a Python
Install Flask: Use pip Import Flask: from flask
file: This file will contain Define the App: app =
install Flask in your import Flask,
your Flask application Flask(__name__)
terminal. render_template, request
code.
Run the
Write Functions: Inside
App: Use app.run(debug=
Create Routes: Define routes, define functions to
True) to start the
routes for different URLs handle user requests
application on a local
using decorators (e.g., process input
server (usually
like @app.route('/path'). data, call the model, and
http://127.0.0.1:5000 by
generate responses).
default).
Integrating Models:
1 2 3 4
Load the Process Input Make Generate
Model: Use libraries Data: Within a route Predictions: Use the Output: Based on
like pickle or joblib to function, extract data loaded model to predictions, generate
load your trained from user requests predict on the appropriate
model into your Flask (e.g., form processed data. responses
app. submissions, API (e.g., display results
calls). in HTML
templates, return
JSON data for APIs).
Introduction to Docker
and Containerization
Docker is a containerization
platform that allows you to
package your application and
its dependencies into
lightweight, portable units
called containers. This makes
deployment and scaling much
easier.
Benefits of Containerization:
Isolation: Container Portability: Docker
s run in isolation containers can run
from each other and on any machine with
the host Docker
system, ensuring installed, regardless
consistent behavior of the underlying
across environments. operating system.
Reproducibility: De
fined container Scalability: Contain
images guarantee a ers can be easily
consistent scaled up or down
environment for your based on demand.
application.
Creating Docker Containers:
Dockerfile: This text file
Running the
defines the steps to build
Building the Container: Use docker
your container image. It
Image: Use the docker run to launch a container
includes instructions for
build command with the from your image. You can
installing
path to your Dockerfile to specify options like ports
libraries, copying
create the image. for exposing services
code, and setting
within the container.
environment variables.
Example
Dockerfile
for a Flask
App:
This Dockerfile:
Starts from a slim Python 3.8 base
image.
Sets the working directory within the
container.
Copies the requirements.txt file
(containing dependencies) and installs
them using pip.
Copies the entire application directory.
Runs the app.py script as the entry
point.
Deploying Dockerized Applications
Docker Hub: Docker Hub is a public registry for sharing and
storing Docker images. You can push your image to Docker Hub
for wider access and deployment across different environments.
Docker Compose: This tool allows you to define and manage
multi-container applications. It simplifies deploying services with
dependencies (e.g., databases) alongside your ML model
container.
Cloud Platforms: Major cloud providers offer Docker container
registry and deployment services. They integrate with Docker
Hub and allow you to manage containerized applications at scale.
3. FastAPI for Building APIs with
ML Models
FastAPI is a modern, high-performance web framework for
Python. It builds upon Flask and offers several advantages for
building RESTful APIs:
Automatic Data Validation: FastAPI automatically validates
user input against defined schemas, improving security and
robustness.
Type Hints: Leveraging type hints for request and response
data enhances code readability and maintainability.
Dependency Injection: FastAPI facilitates dependency
injection for managing dependencies more efficiently.
Building RESTful
APIs with FastAPI:
• Project Setup: Create a new Python project and
install FastAPI using pip install fastapi.
• Define Models: Define data models using Python
classes with type annotations to represent input and
output data for your API endpoints.
• Create Endpoints: Use decorators
like @app.post("/predict") to define API endpoints
for handling incoming requests (e.g., POST requests for
predictions).
• Process Data: Within the endpoint function, extract
data from the request body and perform necessary
preprocessing.
• Make Predictions: Use your loaded machine learning
model to generate predictions on the processed data.
• Return Response: Return a JSON response containing
the predictions or any error messages based on
validation or processing outcomes.
Cont...
• Project Setup: Create a new Python project and
install FastAPI using pip install fastapi.
• Define Models: Define data models using Python classes
with type annotations to represent input and output data for
your API endpoints.
• Create Endpoints: Use decorators
like @app.post("/predict") to define API endpoints for
handling incoming requests (e.g., POST requests
for predictions).
• Process Data: Within the endpoint function, extract data
from the request body and perform necessary
preprocessing.
• Make Predictions: Use your loaded machine learning
model to generate predictions on the processed data.
• Return Response: Return a JSON response containing the
predictions or any error messages based on validation or
processing outcomes.
Example
FastAPI
Endpoint
with ML
Model:
By combining FastAPI with This allows seamless
Docker, you can create a integration with other
robust and scalable solution applications and facilitates
for deploying machine access to your model's
learning models as RESTful predictions
APIs. programmatically
1. Overview of Major Cloud Providers:
Amazon Web
Services
Microsoft Google Cloud
(AWS): Offers a wide
Azure: Provides Platform
range of services for
Azure Machine (GCP): Features
deploying and
Learning, a cloud- Google AI Platform, a
managing ML
based service for suite of services for
models, including
end-to-end ML ML development and
Amazon
workflows. Azure deployment. GCP
SageMaker, a
integrates well with excels in large-scale
comprehensive
other Microsoft tools model training with
platform for
and offers advanced powerful
building, training, and
features like infrastructure like its
deploying
responsible AI and Tensor Processing
models. AWS boasts
explainability tools. Units (TPUs).
robust infrastructure
and scalability.
Choosing a Cloud Provider:
Consider your Existing Specific needs: Look for
budget: Pricing models infrastructure: If you services that cater to
vary between already use a cloud your project's
providers, so evaluate provider for other requirements (e.g., large-
their pay-as-you-go services, leveraging their scale training, specific ML
options and resource ML platform might offer frameworks, explainabilit
usage costs. integration benefits. y tools).
2. Deploying ML Models on the Cloud:
1 2 3 4
Cloud platforms offer Managed Containerization: Use Serverless
various tools and services Services: Utilize services Docker containers to Functions: Leverage
for deploying ML models: like SageMaker package your model and serverless functions
(AWS), Azure Machine dependencies for easier (e.g., AWS Lambda, Azure
Learning, or AI Platform deployment across Functions, GCP Cloud
(GCP) that provide pre-built different environments on Functions) for deploying
infrastructure for the cloud. lightweight models that
deploying, scaling, and respond to events or API
managing models. requests without managing
servers.
Auto-scaling: Cloud platforms enable
automatic scaling of resources
3. (compute, storage) based on model
workload, optimizing costs and ensuring model
Scalability availability.
and Cost Spot Instances: Utilize spot instances (AWS)
Considerati or preemptible VMs (GCP) for cost-effective
training on unused resources, but be aware of
ons: potential interruptions.
Cost Optimization Tools: Explore cloud
provider tools for cost optimization, such as
AWS Cost Explorer, Azure Cost
Management, or GCP Billing. These help
monitor resource usage and identify cost-
saving opportunities.
Student ML Application Project
1 2 3 4 5
Problem Definition: Data Acquisition Model Selection and Evaluation and Deployment: Select a
Clearly define the real- and Preprocessing: Training: Choose a Refinement: Evaluate cloud platform or local
world problem your Gather appropriate suitable ML model model performance deployment option and
application will data and (e.g., classification, using appropriate deploy your trained
address. clean/preprocess it for regression) and train it metrics and refine the model for real-world
model training. on the prepared data. model based on the use.
findings.
Presentation:
Prepare a clear and concise presentation showcasing your project,
including:
• Problem definition and motivation
• Data preparation and exploration
• Model architecture and training
• Evaluation results and discussion
• Deployment approach and considerations
• Future improvements and potential applications
Example Project Ideas:
Image Classification: Build an Sentiment Analysis: Develop a system Time Series Forecasting: Create a
application to classify images (e.g., pet for analyzing customer reviews or social model to predict stock prices or weather
breeds, fashion items) using a pre- media sentiment using an LSTM model patterns using recurrent neural networks
trained model like VGG16 or ResNet on deployed on AWS SageMaker. deployed on Microsoft Azure Machine
GCP AI Platform. Learning.
Additional Considerations:
VERSION CONTROL: USE GIT FOR DOCUMENTATION: DOCUMENT YOUR MONITORING AND FEEDBACK: MONITOR
VERSION CONTROL AND COLLABORATION PROJECT STEPS, CODE, AND DEPLOYMENT MODEL PERFORMANCE IN PRODUCTION
ON YOUR PROJECT CODE. DETAILS FOR FUTURE REFERENCE AND AND GATHER USER FEEDBACK FOR
SCALABILITY. CONTINUOUS IMPROVEMENT.
