Introduction to Docker and Monitoring with InfluxData

© 2017 InfluxData. All rights reserved.1 © 2017 InfluxData. All rights reserved.1
An introduction to Docker and how to handle time series data in container orchestration
25 SEP 2017
Introduction to Docker & Monitoring
with InfluxData

Our Speakers Today
Gary Forghetti
Technical Alliance
Engineer
Gunnar Aasen
Partner Engineering

Agenda
Ïntroduction to Docker, Images & Containers
¨Docker Compose & Swarm
¨What’s new in Docker 17.06
Ünderstanding the time series problem
Ïntroduction to InfluxData
¨Q & A

Deploying Applications with Docker
Gary Forghetti
Technical Alliance Engineer
Business Development
Docker, Inc.

● Overview of Docker
● Images and Containers
● docker-compose
● Docker Swarm
● What's new in Docker 17.06
Agenda

● Founded in 2013 as Linux developer tool.
● Docker solves the “works on my machine”
problem.
● Docker allows you to transform and modernize
your applications and infrastructure, and reduce
costs.
● Docker provides portability, agility and efficiency
and has built-in security.
● Docker can be run on physical hardware,
desktops, and on virtual machines in public or
private clouds.
Docker Overview
Docker is the world's leading software container platform.

● The Docker platform contains multiple products and
tools for developing, testing, deploying, and managing
applications packaged in containers.
● Docker technology focuses on convenience, ease of
use and enablement.
● Docker runs on Linux, macOS and Windows
● Docker Editions
○ Docker Community Edition (free, community
support)
○ Docker Enterprise Edition (subscription,
business day and business critical support)
● Docker Datacenter (container as a service platform)
○ Docker Universal Control Plane (management UI)
○ Docker Trusted Registry (image storage)
● Docker Datacenter is included with Docker EE
Standard and Advanced
Docker Overview
Docker is the world's leading software container platform.

● A Docker Image is a template which is used to build a running
Docker Container.
● It contains the application code, required binaries, libraries,
configuration files and metadata needed to run the application.
● Docker Images do not contain a full operating system.
○ No system kernel or kernel modules.
● Docker Images are stored and shared in registries.
○ Docker Hub (default), Docker Store, and private registries.
● Think of a Docker Image as a "container at rest".
Docker Images

Understanding Image Layers
● An image is a collection of files and some meta
data.
● Images are comprised of multiple layers.
● A layer is also just another image.
● Each image contains the software you want to
run.
● Every image contains a base layer.
● Docker uses a copy on write system.
● Layers are read only.
Docker Images

● Docker Containers are processes running
applications.
● Docker Containers are created from Docker
Images.
● Docker Containers are lightweight, standalone
and portable.
● Unlike virtual machines they do not contain a full
operating system.
● No system kernel or kernel modules.
● Docker Containers use the system kernel on the
host machine (Docker Node).
Docker Containers
Docker containers are based on open standards and run on all major Linux distributions, macOS,
Microsoft Windows, and on any infrastructure including VMs, bare-metal and in the cloud.

Comparing an application running on physical
hardware, on virtual machines and in Docker
Containers.

Limitations:
● Slow deployment times
● High costs
● Wasted resources
● Difficult to scale
● Difficult to migrate
● Vendor lock in
Application running on a Physical Server
One Application on One Physical Server

Benefits:
● Multiple containers can run on the same physical machine
or virtual machine.
● Containers run as isolated processes and share the system
resources CPU, Ram and Storage.
● Containers do not contain a full operating system, they all
share the same operating system kernel.
● Containers take up less space and start quicker.
● Smaller attack surface from a security perspective.
● Containers are easily scaled.
● Containers are portable.
Application running in a Docker Container
Docker Containers
Used together, Docker Containers and VMs provide a tremendous amount of flexibility to optimally
deploy and manage applications.

Benefits:
● Better resource pooling
● One physical machine divided into multiple virtual
machines
● Easier to scale
● VMs in the cloud
● Rapid elasticity
● Pay as you go model
Application running on a Virtual Machine
Hypervisor-Based Virtualization
Limitations:
● Each VM stills requires:
○ CPU allocation
○ RAM
○ Storage
○ An entire guest operating syst
● Full guest OS means wasted resources
● Application portability not guaranteed

Docker Terminology:
Stack
Service
Task

● Stack - a collection of Services that make up an application.
● Service - provides a function and is based on a docker image.
○ Some examples of Services:
■ Load Balancer
■ Web Frontend Application
■ Business Logic Backend Application
Database
● Task - an individual container running in a Service.
Service can have 1-n Tasks (replicas)
Terminology

Manual Application Stack deployment with
Docker.

Manual Application Stack deployment

Automate Application Stack deployments with
docker-compose.

docker-compose is a Docker tool which allows you to deploy multi-container applications
(Stacks) on a docker node in an automated fashion.
● You define your application Stack as one or more Services in a compose yaml file
Compose file documentation: https://docs.docker.com/compose/compose-file/
● The single command "docker-compose" is then used to create and start the Services in the
application Stack.
● The "docker-compose" command is also used to terminate and remove the Services in the
application Stack.
● docker-compose is intended for development and testing.
What is docker-compose?

Docker Application Stack Yaml file template

Clustering and container scheduling.
● Allows you to scale up/down
● Allows you to perform upgrades and roll backs (blue/green deployments)
● Has built-in service discovery and load balancing
● Ensures containers are restarted if they fail
● Allows you to control where containers run (which nodes)
● Has built-in End to End Security
○ Ensure only trusted servers are running your containers
○ Provides a mechanism to store and retrieve secure secrets, passwords and keys, and
restrict which containers can access them
● Supports Windows and Linux workloads in the same swarm
● Application Stack is defined as one or more services in a yaml file
● Single command deploys the Stack ("docker stack deploy") and destroys the Stack
("docker stack rm")
Docker Swarm is not enabled by default.
What is Docker Swarm?

● docker swarm (manage the swarm) https://docs.master.dockerproject.org/swarm/reference/
○ ca - Display and rotate the root CA
○ init - Initialize a swarm
○ join - Join a swarm as a node and/or manager
○ join-token - Manage join tokens
○ leave - Leave the swarm
○ unlock - Unlock swarm
○ unlock-key - Manage the unlock key
○ update - Update the swarm
● docker node (manage nodes in the swarm) https://docs.docker.com/engine/reference/commandline/node/
○ demote - Demote one or more nodes from manager in the swarm
○ inspect - Display detailed information on one or more nodes
○ ls - List nodes in the swarm
○ promote - Promote one or more nodes to manager in the swarm
○ ps - List tasks running on one or more nodes, defaults to current node
○ rm - Remove one or more nodes from the swarmu
○ update - Update a node
Docker Swarm commands

● docker stack (manage stacks) https://docs.docker.com/engine/reference/commandline/stack/
○ deploy - Deploy a new stack or update an existing stack
○ ls - List stacks
○ ps - List the tasks in the stack
○ rm - Remove one or more stacks
○ services - List the services in the stack
● docker service (manage services) https://docs.docker.com/engine/reference/commandline/service/
○ create - Create a new service
○ inspect - Display detailed information on one or more services
○ logs - Fetch the logs of a service or task
○ ls - List services
○ ps - List the tasks of one or more services
○ rm - Remove one or more services
○ scale - Scale one or multiple replicated services
○ update - Update a service
● docker secret (manage secrets) https://docs.docker.com/engine/reference/commandline/secret/
○ create - Create a secret from a file or STDIN as content
○ inspect - Display detailed information on one or more secrets
○ ls - List secrets
○ rm - Remove one or more secrets
Docker Swarm commands

Automate and manage Application Stacks with
Docker Swarm.

● Pets is a simple multi service dockerized application written by a Docker employee
○ Displays random pictures of pets and allows you to "vote" on your favorites.
● Pets is written in Python and uses the Flask Python Web Framework
● Pets consists of 2 Docker Services:
○ Pets Python Flask Frontend application
■ Custom Docker image: chrch/docker-pets:1.0
○ Database backend key value store used by the Pets application to store state data
■ Official Docker image: consul:0.7.2
● Pets is used a lot for demos and has also been used in labs at DockerCon
● Pets is available in a public GitHub repo -> https://github.com/mark-church/docker-pets
Pets - Example Docker Application

Docker Application Stack Yaml file

Display the Stack, Services, and Tasks

Display the Tasks for a Service

Docker Swarm Overlay Multi-Host Networking

Docker Swarm Ingress Network Routing

Update the pets_db Service with a new Docker Image

● Support for IBM Z and Windows 2016 Server
● Custom Roles
○ Now have the ability to create custom roles with very granular access (down to a specific API
function)
● Role Based Access Control for nodes
○ Restrict which users and teams can deploy to which nodes
● Mixed cluster support
○ Run Linux and Windows services in same cluster
○ Use placement constraints to run Windows containers on Windows nodes and Linux containers on
Linux nodes in the cluster
● Policy-Based Automation
○ Automate image promotion using pre-defined policies to move images from one repository to another
within the same registry
○ Create immutable repositories to prevent image tags from being modified or deleted, ensuring that
production application repositories are secure and locked down
● Multi-stage Builds
○ Create Dockerfile with multiple FROM statements and copy artifacts from 1 stage to another. Allows
you to build a docker image with less clutter and reduce it's size
● More info: https://blog.docker.com/2017/08/docker-enterprise-edition-17-06/
What's new in Docker 17.06

● What's new in Docker Enterprise Edition
○ https://www.youtube.com/watch?v=yGkoSsnKlQk
● Docker Modernizing Traditional Applications Program
○ https://blog.docker.com/2017/04/modernizing-traditional-apps-with-docker/
● DockerCon Europe 2017 – October 16-19
○ https://europe-2017.dockercon.com/
● Docker Documentation
○ https://docs.docker.com/
● Docker Community
○ https://www.docker.com/docker-community
● Docker Store
○ https://store.docker.com/
● Docker Technology Partner Guide
○ https://www.docker.com/sites/default/files/community/Docker-Technology-Partner-Program-Guide-082517.
pdf
● Docker Store Vendor Partner
○ https://store.docker.com/publisher/signup
● Docker Maintenance Lifecycle
○ https://success.docker.com/Policies/Maintenance_Lifecycle
● Docker Pricing
○ https://www.docker.com/pricing
Additional Information

Vagrantfile for a 3 node Docker Swarm

What makes time-series unique
¨Needs fast ingestion of data
¨High & low precision data have different retention policy
¨Software agent or DB must be able to compute
¨Unique & changing time query
Containers generating huge data load
¨Regular vs. irregular
¨Containers themselves + components within
Understanding the time series problem

Time to metrics… graphs… value…
¨Primary developers of InfluxDB
¨Created the TICK stack: platform for time series
¨Telegraf and Chronograf fully open source
ÏnfluxDB and Kapacitor open core
¨Single nodes scale to 1 million points ingested per second
Ënterprise clustered and high availability solution
ÏnfluxCloud managed platform
Introduction

Platform Components
Telegraf
¨Collection of metrics and events
¨Many input plugins, deployed to worker nodes
InfluxDB
¨High performance time-series database
Chronograf
¨User Interface for InfluxDB & Kapacitor
Kapacitor
¨Metrics processing and alerting engine

Demo

Learn More
URL or other notes on where to find further info.
URL or other notes on where to find further info.
https://hub.docker.com/_/influxdb/
http://webinars.devops.com/how-to-use-the-tick-stack-coreos-docker-to-make-your-sa
as-offering-better
https://www.influxdata.com/integration/docker-monitoring-tools/
https://github.com/influxdata/TICK-docker
https://github.com/influxdata/telegraf/tree/release-1.3/plugins/inputs/docker
https://github.com/influxdata/telegraf/issues/2054 (contributions welcome!)
https://docs.docker.com/compose/
https://docs.docker.com/engine/swarm/

Q & A?

Introduction to Docker and Monitoring with InfluxData

More Related Content

What's hot

Similar to Introduction to Docker and Monitoring with InfluxData

More from InfluxData

Recently uploaded

Introduction to Docker and Monitoring with InfluxData