KEMBAR78
Secrets in Kubernetes | PDF
Senior System Architect, Google
Developer Expert, Authorised Trainer
SECRETS IN KUBERNETES
JERRY JALAVA
JERRY@QVIK.FI | @W_I
WHAT SECRETS DO
APPLICATIONS HAVE?
QUESTION
@QVIK
COMMON SECRETS
‣ Database credentials
‣ API credentials & endpoints (Twitter, FB,
etc.)
‣ Infrastructure API credentials (Google,
AWS, Azure)
‣ Private (TLS) Keys (careful here)
‣ Etc.
@QVIKPhotos: CC0 License, https://www.pexels.com
I’LL JUST INCLUDE THEM
WITH MY CODE…
WELL, EASY ENOUGH…
@QVIK
DON’T!
@QVIK
JUST AS AN EXAMPLE
Dev put AWS keys on Github.
Bots are crawling all over GitHub seeking secret keys…
"When I woke up the next morning, I had emails and a
missed phone call from Amazon AWS - ~140 servers
running on my AWS account"
- a developer served with a $2,375 Bitcoin mining bill
http://www.theregister.co.uk/
@QVIKPhotos: CC0 License, https://www.pexels.com
I’LL JUST INCLUDE THEM
IN MY CONTAINER…
WELL, EASY ENOUGH…
@QVIK
AGAIN, DON’T!
@QVIK
DOCKER
is about transportable and executable code.
Containers can be inspected, exported and published
publicly.
Putting sensitive material inside Docker Container is
NOT a good idea.
@QVIK
THE TWELVE-FACTOR APP
‣ http://12factor.net/
‣ Methodology for building software-as-a-service apps, collection of
best practises
‣ Suggests following:
‣ Store config in the environment
‣ config is everything that is likely to vary between deploys
@QVIK
KUBERNETES (K8s)
‣ Ancient Greek for ā€œpilotā€ or ā€œhelmsmanā€;
root of the English word ā€œgovernorā€
‣ Orchestrator for containers
‣ Supports multi-cloud environments
‣ Started by Google
‣ Open source
‣ Manage applications, not machines
@QVIK
MAIN COMPONENTS
PODS
Ephemeral units
used to manage 1-n
tightly coupled
containers
LABELS
Metadata attached
to objects such as
Pods. Enable
organization and
selection of objects
REPLICATION
CONTROLLERS
Manages requested
number of Pod
ā€œreplicasā€ from
defined template
1 2 3
SERVICES
Low overhead load-
balancing of
requests to set of
Pods based on
Labels
4
@QVIK
KUBERNETES SECRETS
INTRODUCING
@QVIK
http://kubernetes.io/docs/user-guide/secrets/
WHAT ARE THEY?
‣ K8s Secret-objects are first-class citizens in the ecosystem
‣ Designed to hold all kinds of sensitive information in safe
and flexible way.
‣ They can be used by Pods (FS & Env) and the underlying
kubelet when pulling images
@QVIK
PROS
‣ Secrets can be mounted as data volumes or be exposed
as environment variables to be used by a container in a
pod
‣ A secret is only sent to a node if a pod on that node
requires it
‣ Secret data on nodes is stored in tmpfs volumes and thus
does not come to rest on the node
‣ Communication between user to the api-server, and from
api-server to the kubelets, is protected by SSL/TLS
@QVIK
CONS
‣ In the API server secret data is stored as plaintext in etcd,
therefore:
‣ Administrators should limit access to etcd to admin
users
‣ Secret data in the API server is at rest on the disk that
etcd uses (wipe/shred disks when not used)
‣ It is not possible currently to control which users of a K8s
cluster can access a secret (Support planned)
‣ It is still possible to accidentally push the Secrets
definition to version control
@QVIK
GOTCHAS
‣ A Secret needs to be created before any pods that
depend on it
‣ Individual secrets are limited to 1MB in size
‣ They can only be referenced by pods in same namespace
‣ Once a pod is started, its secret volumes will not change,
even if the secret resource is modified
‣ It is not possible currently to check what resource version
of a secret object was used when a pod was created
(planned feature)
‣ The key must be formatted as DNS subdomain (leading
dots allowed), with max length of 253 characters
@QVIK
EXAMPLES
‣ echo "admin" > ./username.txt && echo "1f2d1e2e67df" > ./
password.txt
‣ kubectl create secret generic db-user-pass --from-file=./
username.txt --from-file=./password.txt
‣ echo "admin" | base64
‣ kubectl create -f ./mysecret.yaml
@QVIK
DEMO TIME
@QVIK
https://github.com/jerryjj/devsec_050416/blob/master/demo/secrets.md
KUBERNETES CONFIGMAP
INTRODUCING
@QVIK
http://kubernetes.io/docs/user-guide/configmap/
WHAT ARE THEY?
‣ key-value pairs of configuration data
‣ Similar to Secrets, but designed to more conveniently
support working with strings that do not contain sensitive
information
‣ Can be used to store fine-grained information like
individual properties or coarse-grained information like
entire config files or JSON blobs
@QVIK
DEMO TIME
@QVIK
https://github.com/jerryjj/devsec_050416/blob/master/demo/configmaps.md
RESOURCES
‣ https://github.com/jerryjj/devsec_050416
‣ http://kubernetes.io/docs/
‣ http://12factor.net/
@QVIK
THANK YOU
www.qvik.fi
Secrets in Kubernetes

Secrets in Kubernetes

  • 1.
    Senior System Architect,Google Developer Expert, Authorised Trainer SECRETS IN KUBERNETES JERRY JALAVA JERRY@QVIK.FI | @W_I
  • 2.
    WHAT SECRETS DO APPLICATIONSHAVE? QUESTION @QVIK
  • 3.
    COMMON SECRETS ‣ Databasecredentials ‣ API credentials & endpoints (Twitter, FB, etc.) ‣ Infrastructure API credentials (Google, AWS, Azure) ‣ Private (TLS) Keys (careful here) ‣ Etc. @QVIKPhotos: CC0 License, https://www.pexels.com
  • 4.
    I’LL JUST INCLUDETHEM WITH MY CODE… WELL, EASY ENOUGH… @QVIK
  • 5.
  • 6.
    JUST AS ANEXAMPLE Dev put AWS keys on Github. Bots are crawling all over GitHub seeking secret keys… "When I woke up the next morning, I had emails and a missed phone call from Amazon AWS - ~140 servers running on my AWS account" - a developer served with a $2,375 Bitcoin mining bill http://www.theregister.co.uk/ @QVIKPhotos: CC0 License, https://www.pexels.com
  • 7.
    I’LL JUST INCLUDETHEM IN MY CONTAINER… WELL, EASY ENOUGH… @QVIK
  • 8.
  • 9.
    DOCKER is about transportableand executable code. Containers can be inspected, exported and published publicly. Putting sensitive material inside Docker Container is NOT a good idea. @QVIK
  • 10.
    THE TWELVE-FACTOR APP ‣http://12factor.net/ ‣ Methodology for building software-as-a-service apps, collection of best practises ‣ Suggests following: ‣ Store config in the environment ‣ config is everything that is likely to vary between deploys @QVIK
  • 11.
    KUBERNETES (K8s) ‣ AncientGreek for ā€œpilotā€ or ā€œhelmsmanā€; root of the English word ā€œgovernorā€ ‣ Orchestrator for containers ‣ Supports multi-cloud environments ‣ Started by Google ‣ Open source ‣ Manage applications, not machines @QVIK
  • 12.
    MAIN COMPONENTS PODS Ephemeral units usedto manage 1-n tightly coupled containers LABELS Metadata attached to objects such as Pods. Enable organization and selection of objects REPLICATION CONTROLLERS Manages requested number of Pod ā€œreplicasā€ from defined template 1 2 3 SERVICES Low overhead load- balancing of requests to set of Pods based on Labels 4 @QVIK
  • 13.
  • 14.
    WHAT ARE THEY? ‣K8s Secret-objects are first-class citizens in the ecosystem ‣ Designed to hold all kinds of sensitive information in safe and flexible way. ‣ They can be used by Pods (FS & Env) and the underlying kubelet when pulling images @QVIK
  • 15.
    PROS ‣ Secrets canbe mounted as data volumes or be exposed as environment variables to be used by a container in a pod ‣ A secret is only sent to a node if a pod on that node requires it ‣ Secret data on nodes is stored in tmpfs volumes and thus does not come to rest on the node ‣ Communication between user to the api-server, and from api-server to the kubelets, is protected by SSL/TLS @QVIK
  • 16.
    CONS ‣ In theAPI server secret data is stored as plaintext in etcd, therefore: ‣ Administrators should limit access to etcd to admin users ‣ Secret data in the API server is at rest on the disk that etcd uses (wipe/shred disks when not used) ‣ It is not possible currently to control which users of a K8s cluster can access a secret (Support planned) ‣ It is still possible to accidentally push the Secrets definition to version control @QVIK
  • 17.
    GOTCHAS ‣ A Secretneeds to be created before any pods that depend on it ‣ Individual secrets are limited to 1MB in size ‣ They can only be referenced by pods in same namespace ‣ Once a pod is started, its secret volumes will not change, even if the secret resource is modified ‣ It is not possible currently to check what resource version of a secret object was used when a pod was created (planned feature) ‣ The key must be formatted as DNS subdomain (leading dots allowed), with max length of 253 characters @QVIK
  • 18.
    EXAMPLES ‣ echo "admin"> ./username.txt && echo "1f2d1e2e67df" > ./ password.txt ‣ kubectl create secret generic db-user-pass --from-file=./ username.txt --from-file=./password.txt ‣ echo "admin" | base64 ‣ kubectl create -f ./mysecret.yaml @QVIK
  • 19.
  • 20.
  • 21.
    WHAT ARE THEY? ‣key-value pairs of configuration data ‣ Similar to Secrets, but designed to more conveniently support working with strings that do not contain sensitive information ‣ Can be used to store fine-grained information like individual properties or coarse-grained information like entire config files or JSON blobs @QVIK
  • 22.
  • 23.
  • 24.