Download to read offline
Uploaded byGiuseppe Piccolo
Containing the world with Docker
The document discusses the evolution of software deployment using Docker and containers, contrasting it with traditional monolithic environments that suffer from complexity and dependency issues. It highlights the advantages of using containers for scalability, portable applications, and efficient resource usage, alongside technical details about Docker's functionality and ecosystem. The text outlines practical use cases, deployment examples, and the integration of Docker with various development tools and platforms.
More Related Content
Similar to Containing the world with Docker
Containing the world with Docker
- 1. Containing the World with Docker Giuseppe Piccolo
- 2. Summary • The evolution of IT • The Solutions • Containers! • Docker • Use Cases • Examples – Deploy an application with Docker
- 3.
- 4. • Monolithic software •Monolithic environments – Specific OS – Specific Middleware – Specific Runtime
- 6.
- 7. • One application can run in more devices – PC – Smartphone – Tablets – … • One service for each needs
- 9.
- 10. More scalable than this!
- 12.
- 13. More testable components (services)
- 14. Software engineers are happy J
- 15.
- 16. Static website Web frontend UserDB Queue Analytics DB Background workers API endpoint nginx 1.5 +modsecurity +openssl+bootstrap 2 postgresql+pgv8 +v8 hadoop +hive +thrift +OpenJDK Ruby +Rails + sass + Unicorn Redis + redis-sentinel Python 3.0+celery + pyredis +libcurl+ ffmpeg + libopencv +nodejs +phantomjs Python 2.7+Flask + pyredis +celery + psycopg +postgresql- client DevelopmentVM QA server Public Cloud Disaster recovery Contributor’s laptop Production Servers Multiplicity of Stacks Production Cluster Customer Data Center Do services and apps interact appropriately? Can I migrate smoothly and quickly? Multiplicity of hardware environments
- 17. Static website Web frontend Backgroundworkers User DB Analytics DB Queue Developmen t VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’ s laptop Customer Servers ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Hell of configurations!
- 18. and the worst is yet to come…
- 19.
- 20. “Dependency hell is a colloquial term for the frustration of some software users who have installed software packages which have dependencies on specific versions of other software packages” Wikipedia
- 22.
- 24.
- 25. Development environment and Production environment are different!
- 26.
- 27. • How to avoid the Hell of configuration? • How to avoid the Dependency hell? • How to migrate and scale quickly ensuring compatibility? • How to replicate my services quickly?
- 28.
- 29.
- 30. • Software abstraction of physical machine • Virtualize anything, hardware included • Provide a complete isolation, thanks to Hypervisor
- 31. • Pro – Fully virtualized OS – Total isolation • Cons – Needs to take a snapshot entire VM in order to replicate them – Use a lot of space – Slow to move around – Poor performance
- 32. Hypervisors trend • 2011 –XEN : Default choice given Rackspace and Amazon use – KVM: Bleeding edge users • 2012 – KVM: Emerges as lead – XEN: Loses momentum
- 33. Hypervisors trend • 2013 –KVM: Maintains lead (around 90% for Mirantis) – Vmware: Emerges as surpising second choice – Containers (LXC, Parallels, Docker): Web Hosting and SAS focused – Xen and HyperV: infrequent requests (XenServer.org) • 2014 – 2015 – Stay tuned J
- 34.
- 35. • Open source VM manager • Released in 2010 • It allows you to script and package VM config and the provisioning setup via a Vagrant File • It is designed to run on top of almost VM tool: VirtualBox, VMWare, AWS, OpenStack • Automating operations with tools such as shell scripts, Chef and Puppet
- 36. Vagrant idea: A file to define a service!
- 37. In a Vagrant File you define: – An operating system – Required library and software – Programs and processes of your final application
- 38. Vagrant: Feature • Command-‐Line interface • Vagrant Share • Boxes • Provisioning • Networking • Synced Folders • Multi-‐Machine • Providers • Plugins
- 39. Vagrant: CLI • It allow us to interact with Vagrant • It offers the following commands: box,connect,destroy,halt,init,login,package a vm, rdp, … https://docs.vagrantup.com/v2/cli/index.html
- 40.
- 41. Install Vagrant and Virtual Box then init Vagrant 1. Install a box (in this example ubuntu) 2. Edit Vagrant previously created by init command Other box can be found at https://vagrantcloud.net
- 42. Start a Vagrant Machine
- 43. 1. Start the box 2. Login into the vm 3. You can destroy the vm by
- 44.
- 45. • By default, it shares your project directory to the /vagrant directory in the guest machine • If you create, in your host os the file will be on the vagrant vm
- 46.
- 47. • Let’s install Apache via a bootstrap.sh file • If you create a file on your host os the file will be on the vagrant vm (use vagrant reload -‐-‐provision)
- 48.
- 49. • Port forwarding : allows you to specify port on the guest system to share via a port on the host machine • By running vagrant reload or vagrant up we can see on http://127.0.0.1:8080 our apache • It supports also bridge configurations and other configurations (https://docs.vagrantup.com/v2/networking)
- 50.
- 51. Providers • Vagrant ships out of the box with support for VirtualBox • If you want to change this behavior (e.g. use VMWare or AWS) you can change the provider • How?
- 52. AWS Vagrant file example
- 53. Share • It is possible to share a Vagrant box with anyone in the world in many ways: – HTTP – SSH – Others (AWS, …)
- 54.
- 55. How many people have heard of Containers before this Talk?
- 56. You've always used him but did not know
- 57. • Google services runs into a container (Gmail, Search, …) • Spotify runs into containers • Ebay runs into container • Baidu runs into containers • ...
- 58.
- 59. I try to explain with an analogy J
- 60. Cargo Transport –Pre 1960 Multiplicity of Goods Do I worry about how goods interact (e.g. coffee beans next to spices) Multipilicityof methods for transporting/storing Can I transport quickly and smoothly (e.g. from boat to train to truck)
- 61. Also a matrix from hell ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
- 62. As you can see not only computer scientists have problems to solve
- 63. How they solved them?
- 64. Standardized shipment with containers! Multiplicity of Goods Do I worry about how goods interact (e.g. coffee beans next to spices) Can I transport quickly and smoothly (e.g. from boat to train to truck) …in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.
- 65. Farewell, Matrix from hell!
- 66. Farewell, Matrix from hell! ? ? ? ? ? ?
- 67. Farewell, Matrix from hell!
- 68.
- 69. Why don’t use the same principle in Computer Science?
- 70. Static website WebfrontendUser DB Queue Analytics DB Development VM QA server Public Cloud Contributor’s laptop Multiplicity of Stacks Multiplicity of hardware environments Production Cluster Customer Data Center Do services and apps interact appropriately? Can I migrate smoothly and quickly …that can be manipulated using standard operations and run consistently on virtually any hardware platform An engine that enables any payload to be encapsulated as a lightweight, portable, self- sufficient container…
- 71. Static website Web frontend Backgroundworkers User DB Analytics DB Queue Developmen t VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’ s laptop Customer Servers Problem Solved!
- 72. This Solution already exists…
- 73.
- 74. Linux Containers (LXC) •Units of software delivery • Run everywhere – Don’t worry about kernel version – Don’t worry about host distro – (but container and host architecture must match*) • Run anything – If it can run on Linux Kernel, it can run J *Unless you emulate CPU with quemu and binfmt
- 75. What is a Container? • It’s a lightweight VM – Own his process space – Own his network interface – Can run stuff as root – Can have it’s own /sbin/init (different from the host)
- 76. What is a Container? • At low level – Container run his processes isolated from the other containers – Share the kernel with host – No device emulation
- 77. Separation of Concerns at environment level!
- 78. Developer Point of View
- 79. Inside my container •My code • My libraries • My package manager • My app • My data
- 80. Ops Point of View
- 81. Outside my container •Logging • Remote access • Network configuration • Monitoring
- 82. All containers start, stop, copy, attach, migrate, etc. the same way!
- 83.
- 84. Isolation with namespaces •pid • mnt • net • uts • ipc • user
- 85. Isolation with cgroups •memory • cpu • blkio • devices
- 86. • AppArmor and SELinux Capability • Seccomp (very specific use case); seccomp-‐bpf • Beware of full-‐scale kernel exploits! • Least but not last: think twice before granting root! And security?
- 87. • Unioning file system (AUFS, overlayfs) • Snapshotting filesystems (BTRFS, ZFS) Copy-‐on-‐write storage
- 88.
- 89. Almost no overhead! •Processes are isolated but run straight on the host • Native CPU performance (no emulation) • Good memory performance • A little overhead in network performance
- 90.
- 91.
- 92. Quick Survey • How many people have heard of Docker before this Talk? • How many people have tried Docker? • How many people are using Docker in production?
- 93.
- 94. “With Docker, developers can build any app in any language using any toolchain. “Dockerized” apps are completely portable and can run anywhere – collegue’s OSX and Windows Laptops, QA servers running Ubuntu in the cloud, and production data center VMs running Red Hat.” Docker.io
- 95. In a nutshell … • Rewrite of dotCloud internal container engine – Original version: Python, closed source, internal stuff only – Release version: Go, legacy-‐free • Docker is an open source engine to commoditize LXC • Allows to create and share standard images, using copy-‐on-‐write mechanism (e.g. AUFS)
- 96. Better than VMs App A Hypervisor (Type 2) Host OS Server Guest OS Bins/ Libs App A’ Guest OS Bins/ Libs App B Guest OS Bins/ Libs App A’ Docker Host OS Server Bins/Libs App A Bins/Libs App B App B’ App B’ App B’ VM Container Containers are isolated, but share OS and, where appropriate, bins/libraries Guest OS Guest OS …result is significantly faster deployment, much less overhead, easier migration, faster restart
- 97.
- 98. • VE (Virtual Environments) based on LXC • Portable deploy across machines • Versioning: docker include git-‐like capabilities for tracking versions of a container • Component reuse: it allows building or stacking already created packages. You can create “base images” and then running more machine based on the image • Shared: libraries there is a public repository with several images (http://registry.hub.docker.com)
- 99. Why Docker and not only LXC? • Fast delivery of your applications • Deploy and scale more easily • Get higher density and run more workload • Fast deployment makes for easier management • Provide standard operation to handle containers • Because is a commodity to easily handle containers
- 100.
- 101. Source Code Repository Dockerfile For A Docker Engine Docker Container Image Registry Build Docker Host 2 OS (Linux) Container A Container B Container C Container A Push Search Pull Run Host 1 OS (Linux)
- 102.
- 103. Docker Engine Docker Container Image Registry Docker Engine Push Update Bins/ Libs App A App Δ Bins/ Base Container Image Host is now running A’’ Container Mod A’’ App Δ Bins/ Bins/ Libs App A Bins/ Bins/ Libs App A’’ Host running A wants to upgrade to A’’. Requests update. Gets only diffs Container Mod A’
- 104.
- 105. • Operating systems –Virtually any distribution with a 2.6.32+ kernel – Red Hat/Docker collaboration to make work across RHEL 6.4+, Fedora, and other members of the family (2.6.32 +) – CoreOS—Small core OS purpose built with Docker • OpenStack – Docker integration into NOVA (& compatibility with Glance, Horizon, etc.) accepted for Havana release • Private PaaS – OpenShift – Solum (Rackspace, OpenStack) – Other TBA • Public PaaS – Deis, Voxoz, Cocaine (Yandex), Baidu PaaS • DevOps Tools – Integrations with Chef, Puppet, Jenkins, Travis, Salt, Ansible +++ • Applications – 1000’s of Dockerized applications available at index.docker.io
- 106.
- 107. Run a simple container
- 108. • Using the run command • if the image does not exist in the local repository, you get it from the remote repository
- 109. • After the download we run into container • If we want to exit from container just press CTRL + P + Q • If we want to see the status of running container just use the ‘ps’ command • If we want to execute a specific command on a running container just use the ‘exec’ command
- 110. Start and Stop containers
- 111. • To run a container from an existing image just use the ‘run’ command • To stop a container just use the ‘stop’ command pass the container’s name or ID Tip: If you don’t remember container’s ID or name get it by a ‘docker ps’ command
- 112. Remove a container If you want to remove a container you can use the ‘rm’ command
- 113.
- 114. • You can specify one or more port on the host to map with one or more port on the container newtwork interface • If you see the status of container, you can see the network interfaces of container
- 115.
- 116. In Docker we can specify one or more folder to share between container and host, while run a container
- 117. • If we create a file into host • we see the same file into container and vice-‐ versa
- 118.
- 119. • A Dockerfile is a file that tell to docker how to build custom images • It starts from a base image • All the added data from this point on are added as an additional layer starting from base image https://docs.docker.com/reference/builder/
- 120.
- 121. Build a Dockerfile Docker assume that a file named Dockerfile exists in specified folder! After build we can see the image into images list We can run the created image with docker run comand
- 122. Deploy a web application
- 123. Deploy a web application -‐ Wordpress • Wordpress Application (1 Container) – Apache webserver / PHP 5.6 – Wordpress4 • MySQL (1 Container) – MySQL server Community 5.6
- 124.
- 125. • Run MySQL container Tips: -‐e flag permit send environment variable to container • And the wordpress container
- 127. References • http://www.slideshare.net/dotCloud/docker-‐intro-‐ november?related=2 • http://www.slideshare.net/fabiofumarola1/develop-‐ with-‐linux-‐containers-‐and-‐docker •http://www.slideshare.net/jpetazzo/introduction-‐ docker-‐linux-‐containers-‐lxc • https://www.docker.com/ • https://docs.docker.com • https://github.com/docker/docker • https://docs.docker.com/examples/mongodb/