New Perl module Container::Buildah - SVPerl presentation

New Perl module Container::Buildah - SVPerl presentation

• 1.
  presented by IanKluft Silicon Valley Perl monthly meeting (online) October 1, 2020 San Jose/Silicon Valley, California
• 2.
  ● Wrapper module Iwrote around Open Source buildah tool – Buildah itself is from Red Hat, available in most Linux distros – GitHub: containers/buildah repo – web site: buildah.io ● Started as my scripting tool to build containers for Podman – Still at early stage, now supports all buildah subcommands ● Provides a front-end to and a layer above buildah – Perl code can run inside a container namespace to set up the image – Makes multi-stage builds more convenient
• 3.
  Intro to LinuxContainers ● Linux containers are a process-level system virtualization – Contrast with Virtual Machines which run separate OS kernel ● Originally Unix systems provided chroot() to change the filesystem root of a process and its subprocesses – Limits access to system in case a process is breached – chroot wasn’t sufficient isolation – root user in a chroot jail has full device access to break out
• 4.
  Linux Kernel supportfor Containerization cgroups ● kernel tracks groups of processes descended from a common parent ● cgroups v1 in Linux 2.6.24 (2007) – resource limitations: memory, fs cache – prioritization in CPU & I/O scheduling – accounting: logging or billing – control: singal/stop processes in a cgroup, OOM killer ● cgroups v2 in Linux 4.5 (2016) – namespace isolation
• 5.
  Linux Kernel supportfor Containerization namespaces ● namespaces compartmentalize resources like a separate system ● processes inherit namespaces from parent ● mount – since 2.4.19 (2002) ● IPC – since 3.0 (2011) ● network – since 3.0 (2011) ● UTS/node name – since 3.0 (2011) ● process ID – since 3.8 (2013) ● user ID – since 3.8 (2013) ● cgroups – since 4.6 (2016) ● time – since 5.4 (2020) ● syslog - proposed
• 6.
  Docker pioneered containerimages ● Dockerfile defines how to build a container image – layers of filesystem & config info – all can run as containers on the same Linux kernel – file format standardized under OCI Open Container Initiative Example container image stacks dependency APK pkgs base distro: Alpine scripts & config app APK packages dependency DEB pkgs base distro: Debian scripts & config app DEB packages dependency RPM pkgs base distro: Fedora scripts & config app RPM packages Linux kernel
• 7.
  Docker model ofcontainers ● Docker popularized Linux containers ● server daemon runs as root ● clients access server through API ● containers launched from server Docker daemon local image storage containers CLI image registry API
• 8.
  Docker model ofcontainers recurring security concerns ● root daemon is a big target – source of many security bugs ● API adds to attack surface Docker daemon local image storage containers CLI attacker image registry API container ship fire photo by Royal Netherlands Navy
• 9.
  Podman model ofcontainers ● Podman was made by Red Hat ● CLI or systemd run podman as a subprocess ● no daemon: root not required – what everyone wanted from Docker ● containers are “rootless” if run by non-root user ● user namespaces isolate root in container from root on host podman user image storage containers CLI image registry systemd
• 10.
  Rootless containers withPodman Podman run as root (UID 0) Podman run as non-root (UID 1000) processes in container run as root (UID 0) UID 0 inside container UID 0 outside container UID 0 inside container UID 1000 outside container processes in container run as non-root (UID 8) UID 8 inside container UID 8 outside container UID 8 inside container UID 2131624 outside container namespaced UID depends on host /etc/subuid range for UID 1000 UID of a container process on the host depends on UIDs of Podman & inside container
• 11.
  What is Buildah? ● OpenSource tool by Red Hat to build OCI-compatible containers – OCI containers are compatible with Docker and Kubernetes ● Replaces the builder portion of Docker – Named by developers for project leader Dan Walsh’s southern-accent pronunciation of “builder” ● Works alongside Podman – podman runs containers as child processes, not in a root daemon – “podman build” uses buildah to build container images – buildah runs containers during build with podman
• 12.
  Buildah command lineinterface ● buildah CLI uses subcommands similar to git – buildah command args ● most buildah subcommands look like lines from a Dockerfile – each perform a step in building a container image ● additional subcommands: – build-using-dockerfile or “bud” - does a build from a Dockerfile – unshare – “user-namespace share”, run a command in a namespace ● Container::Buildah re-runs itself to continue inside a container namespace – other container management: containers, info, inspect, manifest, etc
• 13.
  Buildah CLI subcommands 1of 2 add Add content to the container build-using-dockerfile Build an image using instructions in a Dockerfile commit Create an image from a working container config Update image configuration settings containers List working containers and their base images copy Copy content into the container from Create a working container based on an image images List images in local storage info Display Buildah system information inspect Inspect the configuration of a container or image login Login to a container registry logout Logout of a container registry
• 14.
  Buildah CLI subcommands 2of 2 manifest Manipulate manifest lists and image indexes mount Mount a working container's root filesystem pull Pull an image from the specified location push Push an image to a specified destination rename Rename a container rm Remove one or more working containers rmi Remove one or more images from local storage run Run a command inside of the container tag Add an additional name to a local image umount Unmount the root file system of the specified working containers unshare Run a command in a modified user namespace version Display the Buildah version information
• 15.
  Perl module Container::Buildah ● wrapperfunctions for each buildah CLI subcommand ● seamless entry into build container namespace – avoids the hurdle of separate scripts outside & inside container ● support for multi-stage container builds – one or more container stages to build prerequisites – only keep the build product, such as a library or app, not the compilers/tools – final stage builds container image from prerequisites – configuration specifies “produces” directory or “consumes” stage-name – Container::Buildah runs stages in order of dependencies ● it adds a missing layer by jumping through the hoops for you – regains Dockerfile advantage of easing entry to container namespace
• 16.
  Container::Buildah code overview ● version0.3.0 is current – released today ● 3 classes (line counts include POD, comments & blanks) Container::Buildah 700+ lines Container::Buildah::Subcommand 1100+ lines Container::Buildah::Stage 700+ lines ● 200+ unit test cases in 8 scripts – 1600+ lines under t/ directory
• 17.
  Container::Buildah class ● top-level frontend of the package ● singleton design pattern – one instance ● Container::Buildah::init_config(key => value, …) – sets configuration of container build ● values may include other config items with Perl’s Template Toolkit macros – sets callback functions for each build stage ● Container::Buildah::main() – process command line and run container build ● imports wrapper methods from Container::Buildah::Subcommand
• 18.
  Container::Buildah::Subcommand ● methods for executingexternal commands ● wrapper methods for buildah subcommand – except those that need a container id ● those are methods of Container::Buildah::Stage – should be called as methods of Container::Buildah ● CLI flags in 1st argument as a hash reference – boolean flags require a 1 or 0 value for true/false ● arguments otherwise are same order as CLI
• 19.
  Container::Buildah::Stage ● contents – container IDof a stage – accessors to config tree for the stage ● Callback functions for each build stage are passed an object of this type – methods do not have a container ID parameter – comes from the object ● implements subcommands which require container ID – add, commit, config, copy, run
• 20.
  Example: Hello Worldin C initialization and configuration use Container::Buildah; use YAML::XS; # set paths as constants # directory for build stage to make its binaries my $build_dir = "/opt/hello-build"; # directory for build stage to save product files my $bin_dir = "/opt/hello-bin"; # input directory my $hello_src = "hello.c"; # YAML config file my $hello_bin = "hello"; # container parameters Container::Buildah::init_config( basename => "hello", base_image => 'docker://docker.io/alpine:[% alpine_version %]', required_config => [qw(alpine_version)], stages => { build => { from => "[% base_image %]", func_exec => &stage_build, produces => [$bin_dir], }, runtime => { from => "[% base_image %]", consumes => [qw(build)], func_exec => &stage_runtime, commit => ["[% basename %]:latest"], }, }, );
• 21.
  Example: Hello Worldin C callbacks for build and runtime stage containers # container-namespace code for build stage sub stage_build { my $stage = shift; $stage->debug({level => 1}, "start"); my $input_dir = $cb->get_config('opts', 'inputs'); $stage->run( # install dependencies [qw(/sbin/apk add --no-cache binutils gcc musl-dev)], # create build and product directories ["mkdir", $build_dir, $bin_dir], ); $stage->config({workingdir => $build_dir}); $stage->copy({dest => $build_dir}, $input_dir."/" .$hello_src); $stage->run( ["gcc", "--std=c17", $hello_src, "-o", "$bin_dir/$hello_bin"], ); } # container-namespace code for runtime stage sub stage_runtime { my $stage = shift; $stage->debug({level => 1}, "start"); my $cb = Container::Buildah->instance(); # container environment $stage->config({ entrypoint => $bin_dir.'/'.$hello_bin, }); } # main Container::Buildah::main();
• 22.
  Example: Hello Worldin C building and running the example ● build it $ perl hello_build.pl ● after it builds, use podman to run it $ podman run -it hello:latest Hello world!
• 23.
  Questions? CPAN: https://metacpan.org/pod/Container::Buildah Github: https://github.com/ikluft/Container-Buildah