Download to read offline
Uploaded byIsmaeel Enjreny
Deploy Elasticsearch Cluster on Kubernetes
Kubernetes can be used to deploy an Elasticsearch cluster. Kubernetes runs workloads by placing containers into pods to run on nodes. Pods are the smallest deployable units that contain one or more containers with shared resources. For stateful applications like Elasticsearch, a StatefulSet should be used instead of a Deployment to ensure ordered startup and termination of pods with persistent storage. The Elasticsearch cluster can be deployed on Kubernetes using StatefulSets, ConfigMaps to store configurations, and PersistentVolumes to provide storage for data shards.
More Related Content
Similar to Deploy Elasticsearch Cluster on Kubernetes
More from Ismaeel Enjreny
Deploy Elasticsearch Cluster on Kubernetes
- 1. Deploy Elasticsearch Clusteron Kubernetes Presenter: Elastic Saudi Arabia - Meetup Date: 23 June - 2021 - 09:00 PM
- 2. Chief Architect Elastic CommunityLead in the Middle East Staff engineer | CKA and CKAD Mohammad Abusaa Ismail Anjrini
- 3.
- 4.
- 5. What is Kubernetes? Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. Kubernetes runs your workload by placing containers into Pods to run on Nodes.
- 6. Pods Pods are thesmallest deployable units of computing that you can create and manage in Kubernetes. A Pod is a group of one or more containers, with shared storage and network resources. Container vs initContainer
- 7. StatefulSets vs Deployment ItemDeployment Statefulset Identity of pods Identical, it will get random Id Sticky unique Identifier for each pod Order of creation/deletion Random order Ordered, cannot be created or deleted at the same time Pods endpoints Random name, we cannot rely on it. Fixed per pod Persistent storage By nature , it does not need it, but you can use it By nature, a StatefulSet needs persistent storage so that the application saves its state and does not lose data across restarts.
- 8. Persistent volume A PersistentVolume(PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes. It have a lifecycle independent of any individual Pod that uses the PV.
- 9.
- 10.
- 11.
Editor's Notes
- #2 How To Set Up an Elasticsearch, Fluentd and Kibana (EFK) Logging Stack on Kubernetes: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-on-kubernetes Persistent Volume Claim for StatefulSet: https://zhimin-wen.medium.com/persistent-volume-claim-for-statefulset-8050e396cc51 Install Elasticsearch with Docker: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html https://www.elastic.co/downloads/elastic-cloud-kubernetes
- #5 https://kubernetes.io/ CNCF graduated project
- #10 ELK Docker files https://github.com/elastic/dockerfiles For UserID: 1000 -> check the file https://github.com/elastic/elasticsearch/blob/master/distribution/docker/src/docker/Dockerfile -> adduser -D -u 1000 -G elasticsearch Based on the following file, you can pass elasticsearch.yml settings as Docker env variables, check after “if [[ "$ES_DISTRIBUTION_TYPE" == "docker" ]]; then” https://github.com/elastic/elasticsearch/blob/master/distribution/src/bin/elasticsearch-env
- #11 Kibana Env. variables: https://github.com/elastic/dockerfiles/blob/7.13/kibana/bin/kibana-docker