Development challenges before Docker
Scenario: Arian is a full-stack developer working at a startup. He builds
and maintains a web application using React for the frontend, Django
for the backend, and PostgreSQL for the database.
Challenges Arian Faces
• "It works on my machine" Problem: Arian's app runs perfectly on his
laptop, but when his teammate Nahian runs it, there are errors due
to version mismatches in Python, Node.js, and PostgreSQL.
• Difficult Deployment Process: Deploying the app on a new server
requires installing dependencies manually, configuring environment
variables, and ensuring all services (React, Django, PostgreSQL) run
correctly.
• Resource Management Issues : Running the app locally requires
installing a PostgreSQL server, which conflicts with another project
that uses MySQL.
• Scaling the Application : As the startup grows, Aria wants to deploy
multiple instances of the backend and database for better
performance, but managing dependencies across different
environments is difficult.
How Docker solves these issues
• Containerization for Consistency: Arian uses Docker containers for
the frontend, backend, and database. Now, everyone on the team
runs the same environment, avoiding version conflicts.
• Simplified Deployment: With Docker Compose, Alex defines all
dependencies in a docker-compose.yml file, making deployment as
simple as running docker-compose up.
• Portability: The app runs the same way on Arian's laptop, Nahian's
machine, and production servers.
• Scalability: Alex uses Docker Swarm or Kubernetes to run multiple
instances of the backend and database for load balancing.
What is Docker?
Docker is an open-source platform that allows developers to package
applications and their dependencies into lightweight, portable
containers. These containers ensure that applications run consistently
across different environments, from development to production.
Why do we need Docker?
• Eliminates the "It Works on My Machine" Problem : Developers
often face issues where code runs fine on their machine but
breaks in production. Docker packages everything (code,
libraries, dependencies) into a container, ensuring it runs the
same everywhere.
• Cross-Platform & Portability: Docker containers work identically
on Windows, macOS, Linux, and cloud environments. Move
your app from local development to the cloud without
reconfiguring it.
• Faster Deployment & DevOps Efficiency: No need to manually
install software on different machines. Deploy applications in
seconds using pre-built Docker images.
• Lightweight & Resource-Efficient: Unlike virtual machines (VMs),
Docker containers share the OS kernel, making them faster and
more efficient. Uses fewer resources than running separate VMs.
• Simplifies Microservices & Scaling: Easily run and manage
multiple services (database, backend, frontend) in separate
containers. Scale applications up/down quickly with Docker
Swarm or Kubernetes.
• Consistent CI/CD & Testing: Docker ensures that developers,
testers, and production environments run identical
configurations. Works seamlessly with CI/CD pipelines (GitHub
Actions, Jenkins, GitLab CI/CD).
Core Components of Docker
1. Docker Engine:
• The core of Docker.
• Consists of daemon(dockerd) that listen for API requests.
• Manages container lifecycle (create,start,stop,delete)
2. Docker Image:
• A blueprint for a container
• Containing the OS, application code, and dependencies.
• Stored in Docker Hub or private registries.
• Created using a Dockerfile
3. Docker Containers:
• A running instance of a Docker image.
• Lightweight & isolated from the host system.
• Runs exactly the same across different machines.
• Can be started, stopped, restarted, or deleted.
4. Dockerfile : A script that defines how to build a Docker image.
5. Docker Compose : :
• A tool to manage multi-container applications using a docker-
compose.yml file.
• Defines multiple containers (frontend, backend, database).
6. Docker Hub: A public repository to store and share Docker images.
7. Docker CLI: Used to manage Docker via terminal commands.
What is Virtual Machine?
A Virtual Machine (VM) is a software-based simulation of a physical
computer. It allows multiple operating systems (OS) to run on a
single physical machine, each in its own isolated environment.
https://www.datacamp.com/blog/what-is-a-virtual-machine
VMs borrow resources from the physical computer to generate these
virtual computers. For instance, if a computer has 8 CPUs and 16 GBs of
RAM, we can virtualize an environment with 4 CPUs and 8 GBs of
RAM, split from the physical hardware.
Core Components of a VM
• Host Machine : The physical computer that runs VMs.
• Guest OS : The OS installed inside a VM (e.g., Ubuntu, Windows).
• Hypervisor : Software that creates and manages VMs.
Types of Hypervisors
• Type 1 (Bare Metal) : Runs directly on hardware (e.g., VMware
ESXi, Microsoft Hyper-V).
• Type 2 (Hosted) : Runs on a host OS (e.g., VirtualBox, VMware
Workstation).
The Underlying Technology ↑
&
Docker is written in the Go programming language and takes advantage
of several features of the Linux kernel to deliver its functionality. Docker
uses technology called namespace to provide the isolated workspace
called container. While running a container, Docker creates a set of
namespace for that container
Docker Architecture
↑
&
Docker Client:
• CLI or API that allows users to interect with Docker Daemon.
• It acts as a UI for Docker.
• Communicates with dockerd using a REST API, over UNIX sockets or
a network interface
Docker Daemon/ Dockerd:
• Listens for Docker API requests and manages Docker objects such as
images, containers, networks and volumes
• Suppose, Docker CLI sends the request docker run nginx docker
daemon. The Daemon pulls the nginx image if not available, creates
and starts a container from the image and manages networking and
storage.
• It's the core engine behind Docker.
Docker Registry:
• Docker registry stores Docker images
• During docker pull command, docker pulls the required images from
our configured registry.
• During push, docker pushes image to our configured registry
What is Docker Images
Docker Images are read-only templates used to create containers.
Image vs Container
What is Docker Container?
A Docker container is a lightweight, standalone, and executable
package that includes everything needed to run an application (Code,
System Tools, Libraries, Dependencies)
It runs consistently across different environments, from a developer's
laptop to production servers.
Features:
• A way to package applications with all necessary dependencies
and configuration.
• Portable artifact easily shared and moved around
• Makes development and deployment efficient
Normal System system with container
Container vs VM
When to Use a VM?
Running different OSes (e.g., Windows on Linux).
Hosting legacy applications.
Strong isolation needed (e.g., security-sensitive apps).
When to Use Containers?
Microservices architecture (lightweight, scalable apps).
CI/CD pipelines (fast deployments).
Running the same application across different environments.
🔹✔✔
Docker Commands
↓
1. Checking Docker Installation & Status
Verify if Docker is installed and running: docker --version, docker info
To check if Docker is properly installed and running
2. Managing Docker Containers
List running containers: docker ps
See which containers are currently running.
List all containers (including stopped ones): docker ps -a
To check if Docker is properly installed and running
Start a stopped container: docker start <container_id>
Restart a previously stopped container.
Stop a running container: docker stop <container_id>
Gracefully shut down a container.
Remove a container: docker rm <container_id>
Delete a stopped container to free up space.
3. Working with Docker Images
Search for an image in Docker Hub: docker search <image_name>
Find official images for software like Redis, PostgreSQL, or Nginx.
Download an image from Docker Hub: docker pull <image_name>:<tag>
Get an image before creating a container.
List all downloaded images: docker images
Check which images are available on your machine.
🔹✔
Remove an image: docker rmi <image_id>
Delete an unused image to free space.
4. Running a Container from an Image
Run a container in the background: docker run -d --name my_redis redis:4.0
Starts a container without attaching it to the terminal.
Run a container with a specific port: docker run -d -p
<host_port>:<container_port> --name <container_name> <image_name>
[docker run -d -p 5432:5432 --name my_postgres postgres:17]
Maps a port from the host machine to the container.
Run a container with environment variables: docker run -d -e
POSTGRES_PASSWORD=mysecretpassword --name db postgres:17
Some images require environment variables (e.g., database
credentials).
5. Interacting with Running Containers
View container logs: docker logs <container_id>
Check what’s happening inside a running container.
Access a running container’s terminal: docker exec -it <container_id> bash
Open a shell inside a running container (useful for debugging).
6. Managing Volumes (For Data Storage)
List all Docker volumes: docker volume ls
See what persistent storage is available.
Create a volume: docker volume create my_volume
Useful for keeping data even if containers stop.
✔
Run a container with a mounted volume: docker run -d -v my_volume:/data
--name my_container <image_name>
Some images require environment variables (e.g., database
credentials).
7. Stopping & Cleaning Up Docker
Stop all running containers: docker stop $(docker ps -q)
Quickly stop all running containers.
Remove all stopped containers: docker rm $(docker ps -aq)
Clean up unused containers.
Remove all unused images: docker rmi $(docker images -q)
Free up disk space.
Remove all unused resources: docker system prune -a
Perform a full cleanup of Docker(containers, images, volumes,
networks)
l)
What can it do?
Container vs Image
