Download as PDF, PPTX
Uploaded byHungWei Chiu
Integration kubernetes with docker private registry
The document outlines the setup and usage of a private Docker registry in a Kubernetes environment, detailing the registry's purpose as a stateless and scalable application for distributing Docker images. It discusses the architecture options for deploying the registry, including pros and cons of running it inside or outside the Kubernetes cluster, and offers solutions for enabling secure connections. Additionally, it suggests using an Nginx proxy on each Kubernetes node to simplify access to the registry and mitigate certificate issues.
More Related Content
Similar to Integration kubernetes with docker private registry
Integration kubernetes with docker private registry
- 1.
- 2. WHOAMI Hung-Wei Chiu () hwchiu.com Experience ○ Software Engineer at Linker Networks (now) ○ Co-organizer of SDNDS-TW ○ Co-organizer of CUTNG Fields ○ Linux Kernel, Network Stack ○ Networking/Kubernetes
- 3. outline Introduction to DockerRegistry K8S & Docker Registry ○ Scenario ○ What’s problem ○ How to solve
- 4. DockerRegistry A stateless, highlyscalable server side application that stores and lets you distribute Docker images.
- 5. Thebasicdockerusage docker pull nginx:yyy ○Official Repositories + tag docker pull hwchiu/xxxx:yyy ○ Username + Image name + tag
- 6. Thebasicdockerusage docker pull nginx:yyy ○Official Repositories + tag docker pull hwchiu/xxxx:yyy ○ Username + Image name + tag That’s for Docker Hub. You don’t need to worry about the location of the registry
- 7. Docker Hub Docker pullnginx Host1 Docker pull hwchiu/aaa Host2 Docker knows that you want to pull image from the docker hub (Implicitly)
- 8. Privateregistry Setup the registryfor yourself For externally-accessible ○ You need to handle the network by yourself ○ IP address or hostname ○ Use HTTPS by default For localhost ○ Only accessed from localhost ○ Always trust (HTTP/HTTPS)
- 9. My Registry Docker pull 192.168.2.3/image1 Host1 Dockerpull 192.168.2.3/image2 Host2 Docker knows that you want to pull image from the private registry 192.168.2.3
- 10. Kubernetes Pull the dockerimages and run
- 11. ImageAscenario I want touse the kubernetes to do the CI/CD of my application. I want to build the docker image based on my application in the kubernetes pod. I want to run the docker image I build before
- 12. workloads Run a kubernetespod(DockerHelper) ○ Build the docker image ○ Push the docker image to private registry Run another kubernetes pod ○ Based on your own application
- 13. In the networkview Make sure hosts can connect to registry via IP/Hostname
- 14. My Registry Host1 Host2 a.b.c.d Kubernetes cluster DockerHelper Build/Push my own docker image We must make sure the host1 can connect to a.b.c.d
- 15. My Registry Host1 Host2 a.b.c.d Kubernetes cluster DockerHelper Build/Push my own docker image Run my own docker image We must make sure the host2 can connect to a.b.c.d
- 16. In the (in)securityview We need to handle the HTTPS
- 17. If you’re rich,buy the certificate and skip this.
- 18. My Registry Host1 Host2 IP:$REGISTRY_SERVER Kubernetes cluster Runmy own docker image Start the docker daemon with –insecure-registry=$REGISTRY_SERVER option For insecure solution Docker Helper Build/Push my own docker image
- 19. My Registry Host1 Host2 IP:$REGISTRY_SERVER Kubernetes cluster Runmy own docker image Put the cert into /etc/docker/certs.d/$REGISTER_SERVER/ca.crt For secure solution Docker Helper Build/Push my own docker image
- 20. Let’s start todesign the architecture
- 21. Privateregistrylocation Where do wesetup the docker registry? In the cluster/Out of the cluster ?
- 22. Inthecluster Run as aPod in the cluster Pros: ○ DH pod can access it by hostname ■ Kubernetes service ○ K8S guarantee the running instance of registry container Cons: ○ Need to handle the data sync within all nodes ■ If the new registry runs on different node. ○ The k8s node can’t access it via hostname.
- 23. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network Registry Server 1. Deploy the Register Server with k8s service.
- 24. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network Registry Server 1. Deploy the Register Server with k8s service. 2. Docker-Helper (Pod) can use registry.default to talk (easy) 3. K8s node can’t use registry.default (not easy) Docker Helper Build/Push my own docker image Registry.default
- 25. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network Registry Server 1. Deploy the Register Server with k8s service. 2. Docker-Helper (Pod) can use registry.default to talk (easy) 3. K8s node can’t use registry.default (not easy) Run own docker image (can’t use the registry.default)
- 26. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network Registry Server 1. Sync the data between all K8S node Ø Mount external volume Ø Sync the filesystem
- 27. Outofthecluster Run as astandalone server out of the cluster Pros: ○ Node can access it by ip/hostname(DNS) ■ Should consider the SSL Cons: ○ DH Pod should knows the location of registry server ■ Use DNS, need dns server (not easy) ■ Use IP, DH Pod show know the IP once server changes IP.
- 28. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network 1. The admin should know the IP/DNS of the Register Server K8S masterRegistry Server
- 29. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network 1. The admin should know the IP/DNS of the Register Server 2. DH push the image to Registry Server K8S masterRegistry Server Docker Helper Build/Push my own docker image
- 30. Kubernetes cluster K8S master K8SnodeK8S node K8S node Magic Network 1. The admin should know the IP/DNS of the Register Server 2. DH push the image to Registry Server 3. Pull the image from the Registry Server and run as Pod. K8S masterRegistry Server Run own docker image
- 31. Problems Certificated (we’re notrich) ○ We need to modify the docker config for each node ○ We also need to modify the docker config for DH pod Network ○ Only the out of cluster + DNS seems good ■ I don’t want to handle any DNS by myself
- 32. How to solvethose problems ????
- 34. Finally, I thoughta solution
- 35. Fornetworking We run theregistry as Pod on K8S We also run nginx (daemonSet) on each k8s node. ○ Act as a proxy ○ Forward the packet to registry.default Assume the docker image is ○ localhost:5566/myapp:master
- 36. Registry Server Nginx-ProxyNginx-Proxy Dockerdaemon localhost:5566 iptables Pod_IP:80 Registry.default:5566 Overlay network K8s node IP: 5566 iptables Pod_IP:5566 K8S cluster NODE view
- 37. FortheDockerHelperPod We runs thepod with hostnetwork=true The DH Pod will use the host network stack. The DH must build the docker image name as the form localhost:5566/…
- 38. Forsecurity The docker trustthe localhost We don’t need to handle all certificated problems now.
- 39.