Downloaded 278 times
Uploaded byCarlos Sanchez
Scaling Jenkins with Docker and Kubernetes
The document discusses scaling Jenkins using Docker and Kubernetes, highlighting key concepts such as container orchestration, replication controllers, and service configurations. It provides examples of Kubernetes resources, including pods and services, specifically tailored for Jenkins deployments. Additionally, it shares details on setting up Jenkins clusters and utilizing the Kubernetes Jenkins plugin for improved integration.
More Related Content
Similar to Scaling Jenkins with Docker and Kubernetes
Scaling Jenkins with Docker and Kubernetes
- 1. Scaling Jenkins withDocker and Kubernetes Carlos Sanchez @csanchez
- 2.
- 3.
- 4. But it isnot trivial
- 5. @YourTwitterHandle #jenkinsconf Docker Linux containers Union File System FileSystem Users Processes Network
- 6.
- 7.
- 8.
- 9.
- 10.
- 11. #jenkinsconf Footer How would youdesign your infrastructure if you couldn't login? Ever. Kelsey Hightower CoreOS
- 12. #jenkinsconf Kubernetes Container cluster orchestration Dockercontainers across multiple hosts (nodes or minions) Higher level API Enforced state Monitoring of endpoints
- 13.
- 14.
- 15. #jenkinsconf Master Kubernetes API Server schedulingand synchronization etcd Kubernetes Controller Manager Server implements replication algorithm watching etcd
- 16. #jenkinsconf Node Docker Kubelet ensures state ofPods Kubernetes Proxy simple network proxy etcd SkyDNS ElasticSearch + Kibana
- 17.
- 18.
- 19.
- 20.
- 21. #jenkinsconf Google Container Engine gcloudbeta container --project my-project clusters create cluster-1 --machine-type g1-small --num-nodes 2
- 22.
- 23. Node --- kind: "Node" apiVersion: "v1" metadata: name:"127.0.0.1" selfLink: "/api/v1/nodes/127.0.0.1" uid: "8c3192d2-48ef-11e5-8d27-bae1092286ff" resourceVersion: "55118" creationTimestamp: "2015-08-22T17:02:19Z" labels: kubernetes.io/hostname: "127.0.0.1" spec: externalID: "127.0.0.1" status: capacity: cpu: "0" memory: "0" pods: "40" conditions: - type: "Ready" status: "True" lastHeartbeatTime: "2015-08-26T19:38:20Z" lastTransitionTime: "2015-08-26T12:18:48Z" reason: "kubelet is posting ready status" addresses: - type: "LegacyHostIP" address: "127.0.0.1" nodeInfo: ...
- 24. #jenkinsconf Pod Group of colocatedcontainers Same network namespace/IP Environment variables Shared volumes host mounted empty volumes GCE data disks AWS EBS volumes
- 25.
- 26. Pod --- kind: "Pod" apiVersion: "v1" metadata: name:"jenkins" labels: name: "jenkins" spec: containers: - name: "jenkins" image: "csanchez/jenkins-swarm:1.609.2" ports: - containerPort: 8080 hostPort: 8090 - containerPort: 50000 hostPort: 50000 volumeMounts: - name: "jenkins-data" mountPath: "/var/jenkins_home" volumes: - name: "jenkins-data" hostPath: path: "/home/docker/jenkins"
- 27. #jenkinsconf Replication controller Ensure anumber of pods are running Pod templates Rolling update podpod pod container 1 container 2 container 3
- 28.
- 29.
- 30. #jenkinsconf Replication controller --- apiVersion: "v1" kind:"ReplicationController" metadata: name: "jenkins-slave" labels: name: "jenkins-slave" spec: replicas: 1 template: metadata: name: "jenkins-slave" labels: name: "jenkins-slave" spec: containers: - name: "jenkins-slave" image: "csanchez/jenkins-swarm-slave:2.0" env: command:
- 31. #jenkinsconf Replication controller command: - "/usr/local/bin/jenkins-slave.sh" -"-master" - "http://$(JENKINS_SERVICE_HOST):$(JENKINS_SERVICE_PORT)" - "-tunnel" - "$(JENKINS_SLAVE_SERVICE_HOST):$(JENKINS_SLAVE_SERVICE_PORT)" - "-username" - "jenkins" - "-password" - "jenkins" - "-executors" - "1"
- 32. #jenkinsconf Services Pod discovery IP perservice Route to pods selected with labels Can create a load balancer in GCE and AWS
- 33.
- 34. Services --- apiVersion: "v1" kind: "Service" metadata: name:"jenkins" spec: type: "NodePort" selector: name: "jenkins" ports: - name: "http" port: 8090 nodePort: 32080 protocol: "TCP" --- apiVersion: "v1" kind: "Service" metadata: name: "jenkins-slave" spec: type: "NodePort" selector: name: "jenkins" ports: - name: "http" port: 50000 nodePort: 32050 protocol: "TCP"
- 35. #jenkinsconf Networking all containers cancommunicate with all other containers without NAT all nodes can communicate with all containers (and vice-versa) without NAT the IP that a container sees itself as is the same IP that others see it as Containers in a Pod can talk using localhost
- 36. #jenkinsconf Networking Every machine inthe cluster is assigned a full subnet ie. node A 10.0.1.0/24 and node B 10.0.2.0/24 Simpler port mapping Only supported by GCE CoreOS flannel Creates an overlay network in other providers
- 37.
- 38. #jenkinsconf Docker Machine Provision Dockerengines VirtualBox, replaces boot2docker ! Amazon EC2 Microsoft Azure Google Compute Engine OpenStack Rackspace VMware …
- 39. #jenkinsconf Docker Compose Orchestration ofmulti-container apps Based on Fig Defined by: containers configuration links volumes
- 40.
- 41. #jenkinsconf Kubernetes cluster withdocker-compose # Docker Compose definition for a one node Kubernetes cluster # Based on Docker Cookbook example # https://github.com/how2dock/docbook/ch05/docker etcd: image: kubernetes/etcd:2.0.5.1 net: "host" command: /usr/local/bin/etcd --addr=127.0.0.1:4001 --bind-addr=0.0.0.0:4001 --data-dir=/var/etcd/data master: image: gcr.io/google_containers/hyperkube:v1.0.1 net: "host" volumes: - /var/run/docker.sock:/var/run/docker.sock command: /hyperkube kubelet --api_servers=http://localhost:8080 --v=2 --address=0.0.0.0 --enable_server --hostname_override=127.0.0.1 --config=/etc/kubernetes/manifests proxy: image: gcr.io/google_containers/hyperkube:v1.0.1 net: "host" privileged: true command: /hyperkube proxy --master=http://127.0.0.1:8080 --v=2 https://github.com/carlossg/kubernetes-jenkins
- 42. #jenkinsconf Jenkins master pod kind:"Pod" apiVersion: "v1" metadata: name: "jenkins" labels: name: "jenkins" spec: containers: - name: "jenkins" image: "csanchez/jenkins-swarm:1.609.2" ports: - containerPort: 8080 hostPort: 8090 - containerPort: 50000 hostPort: 50000 https://github.com/carlossg/kubernetes-jenkins
- 43. #jenkinsconf storage options volumeMounts: - name: "jenkins-data" mountPath:"/var/jenkins_home" volumes: - name: "jenkins-data" hostPath: path: "/home/docker/jenkins" # gcePersistentDisk: # pdName: my-data-disk # fsType: ext4 # awsElasticBlockStore: # volumeID: aws://<availability-zone>/<volume-id> # fsType: ext4 https://github.com/carlossg/kubernetes-jenkins
- 44. #jenkinsconf Jenkins master services --- apiVersion:“v1" kind: "Service" metadata: name: "jenkins" spec: type: "NodePort" selector: name: "jenkins" ports: - name: "http" port: 8090 nodePort: 32080 protocol: “TCP" --- apiVersion: "v1" kind: "Service" metadata: name: "jenkins" spec: type: "NodePort" selector: name: "jenkins" ports: - name: "http" port: 8090 nodePort: 32080 protocol: "TCP" https://github.com/carlossg/kubernetes-jenkins
- 45. #jenkinsconf Jenkins slaves replicationpool --- apiVersion: "v1" kind: "ReplicationController" metadata: name: "jenkins-slave" labels: name: "jenkins-slave" spec: replicas: 1 template: metadata: name: "jenkins-slave" labels: name: "jenkins-slave" spec: containers: https://github.com/carlossg/kubernetes-jenkins
- 46. #jenkinsconf Jenkins slaves replicationpool - name: "jenkins-slave" image: "csanchez/jenkins-swarm-slave:2.0" command: - "/usr/local/bin/jenkins-slave.sh" - "-master" - "http://$(JENKINS_SERVICE_HOST):$(JENKINS_SERVICE_PORT)" - "-tunnel" - "$(JENKINS_SLAVE_SERVICE_HOST):$(JENKINS_SLAVE_SERVICE_PORT)" - "-username" - "jenkins" - "-password" - "jenkins" - "-executors" - "1" https://github.com/carlossg/kubernetes-jenkins
- 47. #jenkinsconf Jenkins cluster inKubernetes kubectl get nodes kubectl create --validate -f pod.yml kubectl get pods kubectl create --validate -f service-http.yml kubectl create --validate -f service-slave.yml kubectl get services kubectl create --validate -f replication.yml kubectl get pods kubectl scale replicationcontrollers --replicas=20 jenkins-slave https://github.com/carlossg/kubernetes-jenkins
- 48.
- 49. #jenkinsconf Kubernetes Jenkins plugin Asa plugin on demand slaves https://github.com/jenkinsci/kubernetes-plugin
- 50.
- 51. #jenkinsconf Kubernetes Jenkins plugin pods,not replication controllers Jenkins Cloud API Fabric8 Java API Workflow support
- 52.
- 53.
- 54. #jenkinsconf Roadmap Stable API When KubernetesJava lib is stable Using new Jenkins Cloud/Containers APIs
- 55. #jenkinsconf Example code andslides Available at http://slideshare.csanchez.org https://github.com/carlossg/kubernetes-jenkins http://blog.csanchez.org
- 56.
- 57. #jenkinsconf Footer Thanks to ourSponsors! 78