Download as PDF, PPTX
Uploaded bySIGHUP
Kubernetes extensibility: CRDs & Operators
The document discusses the use of Kubernetes Custom Resource Definitions (CRDs) and operators to automate application lifecycle management in a cloud-native environment. It highlights Kubernetes extensibility features, operator patterns, and the properties and capabilities of controllers for managing resources efficiently. The presentation aims to address the challenges of repetitive manual operations and proposes the use of operators as a solution for scaling, management, and stateful applications.
More Related Content
Similar to Kubernetes extensibility: CRDs & Operators
Kubernetes extensibility: CRDs & Operators
- 1. Building within Kubernetes CRDsand Operators 9 / 3 / 2018 - Incontro Devops Italia
- 2. whoami Giacomo Tirabassi Devops Engineer@ Sighup Srl twitter.com/gitirabassi github.com/gitirabassi I like to automate, cook and git together
- 3. Sighup Srl All thingsKubernetes sighup.io twitter.com/sighup_
- 4. Why are wehere? - Operations are boring - Repeating work over and over - Distributed Systems are hard to manage by humans - Application Lifecycle Management as Code
- 6.
- 7.
- 8. K8S Extensibility --Infrastructure ● Container Runtime (CRI) ● Network Plugin (CNI) ● Volumes (CSI) → alpha in 1.9 ● Cloud Controller Manager (Aws, GCP, Azure, Openstack, oVirt) → alpha in 1.10 ● Device Plugins (GPU, FPGAs, QRNGs) → alpha in 1.8
- 9. K8S Extensibility --Infrastructure ● Secrets Backend (Vault, Azure AWS GCP KMS ) → alpha in 1.9 ● Replace Scheduler ● Multiple Scheduler ● Scheduler Extender ● Custom Controller
- 10. K8S Extensibility --API ● API(-server) Aggregation ● Custom Resource Definition
- 11.
- 14. K8s Resource Types ●API-Server is a REST Application with CRUD interface ● API-Server doesn’t know anything about infrastructure ● API-Server manages Pods/Deployments/Services like apples and pears (no special meaning) ● Object Properties: ○ Apiversion ○ Kind ○ Metadata ○ Specs
- 15. K8s Resource Types https://MASTER_NODE_URL/apis/extensions/v1beta1/namespaces/default/deployments https://MASTER_NODE_URL/api/v1 /nodes https://MASTER_NODE_URL/api/ v1 /namespaces/default/services Master url GROUP VERSION NAMESPACE KIND
- 16. Custom Resource Definition ●Previously known as TPR ( 1.2 -- 1.7 ) ● Stable from kubernetes 1.8 ● C-R-U-D Object ● RBAC ● State definition API: declarative ● Versioning ● Automatic `kubectl` compatibility demo
- 17. K8s Resources https://MASTER_NODE_URL/apis/sighup.io/v1alpha1/namespaces/default/apples Master urlGROUP VERSION NAMESPACE KIND
- 18.
- 21. Controllers - Reconciliation loops:Observe + Analyze + Act - Converge desidered state with real state - Attached to infrastructure events (getters & listers & informers) - Simple Interface to implement: ADD, UPDATE, DELETE - Cluster aware - Actions can be internal to the cluster or external (beginning of service catalog)
- 22. Controllers Properties - Fullytyped - Cache - Workqueues - Chaos - Leader election - Code generation - Clientsets
- 23. Custom controllers examples -Automatic notification on resources allocation, role bindings, errors - Automatic service creation for every new deployment - Automatic Secret distribution - Automatic new namespace configurations (conservatives defaults) - Automatic Custom horizontal pod scaler - Automatic ingress controller configuration demo
- 24.
- 25. Patterns - CRD +Volume Plugin + Controller ⇒ Rook - CRD + Network Plugin ⇒ Calico Canal - CRD + Controller ⇒ Operator - CRD + Controller ⇒ core features prototyping
- 26. Operators - Not akubernetes Object: just a pattern - Introduced by COREOS → 3/11/2016 - Go program running in the cluster as deployment (other language supported)
- 27. Operators available - Etcd/ kafka / kubedb / prometheus / influx / elasticsearch / mongodb / memcached / postgres - Vitess / redis - Rook - Cert-manager - Api gateway: kanali / kong - All operations infrastructure facing: backup, dr, snapshots, vpn
- 28. Why Operators? - CustomBootstrap, Backup, Restore, DR, Monitoring, Telemetry - Manual recurrent cleanups - Ad-hoc commands for scaling/join/leave - Clustering - Stateful (writeOnceReadMany, writeManyReadMany) - Custom Management (dedicated hardware or people)
- 29. Why Operators? - Howto automate application lifecycle without operators? - Many Wheels: - Accessibility - Availability (HA) - Authorization - Events - Reconciliation Loop ( real state → desired state )
- 30. Operator’s alternatives? - YAMLmanifest → not templatable, very verbose - Helm Charts → only deploy, no cluster aware - Third party automation → reinvent wheels and polling on api-server for changes - Operator → complex business logic to deal with, simple for multiple instances of object demo
- 31. Problems ● Bring togetherpeople who knows about go, kubernetes and the specific application ● Convince people to trust automation ● Like kernel modules → very powerful, but it can crash the whole system ● Kubernetes Go SDK is a mess
- 32. Recap - Manual operationsare expensive, boring and error prone - Abstracting apps livecycle management from underlying infrastructure (k8s feature) - Non-cloud application can become simple objects - Stateful is hard but not blocking
- 33. Tools ● github.com/golang/dep ● github.com/kubernetes/code-generator ●github.com/giantswarm/operatorkit ● github.com/rool/operator-kit ● github.com/spotahome/kooper
- 34.