Visit the HAProxy booth to receive the complete eBook
SAHAMPPLES KubeCon + CloudNativeCon 2020
C TER
Exclusive eBook!
HAProxy in
Kubernetes
Supercharge Your Ingress Routing
www.haproxy.com
HAProxy in
Kubernetes
Supercharge Your Ingress Routing
© 2020 HAProxy Technologies
Table of Contents
Introduction 3
The Ingress Pattern 5
Install the HAProxy Kubernetes Ingress Controller with Helm* 13
Routing Traffic* 22
TLS with Let's Encrypt* 28
Multi-tenant Kubernetes Clusters* 39
Kubernetes Deployment Patterns* 56
Where Next* 75
GET THE COMPLETE EBOOK
* Visit the HAProxy booth at
KubeCon 2020 to receive the
complete eBook, or download it
at: h
aproxy.com
Introduction
By now, it's clear that Kubernetes is a force within the
technology landscape, enabling sophisticated
orchestration of containerized services across a cluster of
servers. It's also clear that organizations need simple, yet
powerful, components to solve problems like routing
traffic into their Kubernetes clusters. From its inception,
the platform was built for this kind of extensibility, with the
foresight that users would desire ways to integrate their
favorite tools.
In this book, you will learn how to combine the power of
the HAProxy load balancer with Kubernetes. HAProxy has
been recast as a Kubernetes Ingress Controller, which is a
Kubernetes-native construct for traffic routing. The
controller is maintained as a distinct project, with a regular
release cycle, a growing community of developers, and an
increasing number of converts who favor it over other
ingress controllers.
As you read this book, consider your use case. Do you
require enhanced security? The HAProxy Kubernetes
Ingress Controller lets you enforce rate limits and you can
whitelist client IP addresses to better control access. Do you
want to supercharge your Kubernetes Ingress routing?
HAProxy is known as the world's fastest software load
balancer and has been benchmarked against alternatives
like NGINX, Envoy, and Traefik. Do you need detailed
observability in order to graph trends and make informed
decisions?
HAProxy in Kubernetes 3
HAProxy comes with a built-in Stats page that provides
many counters and gauges for monitoring traffic, and these
are also exposed by a Prometheus endpoint.
With 20 years of ongoing development under its belt, the
HAProxy project is stronger than ever, and it's no surprise
that it's found a niche within the Kubernetes platform. We
hope that this book gives you all of the knowledge you
need to start using it. Community members like you drive
us to make it even better, so we welcome you to join us on
Slack, follow us on Twitter, and send us your questions!
slack.haproxy.org/
twitter.com/haproxy
github.com/haproxytech/kubernetes-ingress/
HAProxy in Kubernetes 4
The Ingress Pattern
Containers allow cross-functional teams to share a
consistent view of an application as it flows through
engineering, quality assurance, deployment and support
phases. The Docker CLI gives you a common interface
through which you can package, version, and run your
applications. What’s more, with orchestration platforms
like Kubernetes, the same tools can be leveraged to
manage services across isolated Dev, Staging and
Production environments.
This technology is especially convenient for teams working
with a microservices architecture, which produces many
small, but specialized, services. However, the challenge:
How do you expose your containerized services outside of
the container network?
In this chapter, you’ll learn about the HAProxy Kubernetes
Ingress Controller, which is built upon HAProxy, the world’s
fastest and most widely used load balancer. As you’ll see,
using an ingress controller solves several tricky problems
and provides an efficient, cost-effective way to route
requests to your containers. Having HAProxy as the engine
gives you access to many of the advanced features you
know and love.
HAProxy in Kubernetes 5
Kubernetes Basics
First, let’s review the common approaches to routing
external traffic to a pod in Kubernetes. Since there are
many, thorough explanations available online that
describe Kubernetes in general, we will skip the basics. In
essence, Kubernetes consists of physical or virtual
machines—called nodes—that together form a cluster.
Within the cluster, Kubernetes deploys pods. Each pod
wraps a container (or more than one container) and
represents a service that runs in Kubernetes. Pods can be
created and destroyed as needed.
To maintain the desired state of the cluster, such as which
containers, and how many, should be deployed at a given
time, we have additional objects in Kubernetes. These
include ReplicaSets, StatefulSets, Deployments,
DaemonSets, and more. For our current discussion, let’s
skip ahead and get to the meat of the topic: accessing pods
via Services and Controllers.
HAProxy in Kubernetes 6
Services
A service is an abstraction that allows you to connect to
pods in a container network without needing to know a
pod’s location (i.e. which node is it running on?) or to be
concerned about a pod’s lifecycle. A service also allows you
to direct external traffic to pods. Essentially, it’s a primitive
sort of reverse proxy. However, the mechanics that
determine how traffic is routed depend on the service’s
type, of which there are four options:
● ClusterIP
● ExternalName
● NodePort
● LoadBalancer
When using Kubernetes services, each type has its pros
and cons. We won’t discuss ClusterIP because it doesn’t
allow for external traffic to reach the service—only traffic
that originates within the cluster. ExternalName is used to
route to services running outside of Kubernetes, so we
won’t cover it either. That leaves the NodePort and
LoadBalancer types.
NodePort
When you set a service’s type to NodePort, that service
begins listening on a static port on every node in the
cluster. So, you’ll be able to reach the service via any
node’s IP and the assigned port. Internally, Kubernetes
does this by using L4 routing rules and Linux IPTables.
HAProxy in Kubernetes 7
While this is the simplest solution, it can be inefficient and
also doesn’t provide the benefits of L7 routing. It also
requires downstream clients to have awareness of your
nodes’ IP addresses, since they will need to connect to
those addresses directly. In other words, they won’t be
able to connect to a single, proxied IP address.
LoadBalancer
When you set a service’s type to LoadBalancer, it exposes
the service externally. However, to use it, you need to have
an external load balancer. The external load balancer
needs to be connected to the internal Kubernetes network
on one end and opened to public-facing traffic on the
other in order to route incoming requests. Due to the
dynamic nature of pod lifecycles, keeping an external load
balancer configuration valid is a complex task, but this
does allow L7 routing.
HAProxy in Kubernetes 8
Oftentimes, when using Kubernetes with a
platform-as-a-service, such as with AWS’s EKS, Google’s
GKE, or Azure’s AKS, the load balancer you get is
automatic. It’s the cloud provider’s load balancer solution.
If you create multiple Service objects, which is common,
you’ll be creating a hosted load balancer for each one. This
can be expensive in terms of resources. You also lose the
ability to choose your own preferred load balancer
technology.
There needed to be a better way. The limited, and
potentially costly, methods for exposing Kubernetes
services to external traffic led to the invention of Ingress
objects and Ingress Controllers.
HAProxy in Kubernetes 9
Controllers
The official definition of a controller, not specific to ingress
controllers, is:
a control loop that watches the shared state of the cluster
through the apiserver and makes changes attempting to
move the current state towards the desired state.
For example, a Deployment is a type of controller used to
manage a set of pods. It is responsible for replicating and
scaling of applications. It watches the state of the cluster in
a continuous loop. If you manually kill a pod, the
Deployment object will take notice and immediately spin
up a new one so that it keeps the configured number of
pods active and stable.
Other types of controllers manage functions related to
persistent storage, service accounts, resource quotas, and
cronjobs. So, in general, controllers are the watchers,
ensuring that the system remains consistent. An Ingress
Controller fits right in. It watches for new services within
the cluster and is able to dynamically create routing rules
for them.
Ingress
An Ingress object is an independent resource, apart from
Service objects, that configures external access to a
service’s pods. This means you can define the Ingress later,
after the Service has been deployed, to hook it up to
external traffic. That is convenient because you can isolate
HAProxy in Kubernetes 10
service definitions from the logic of how clients connect to
them. This approach gives you the most flexibility.
L7 routing is one of the core features of Ingress, allowing
incoming requests to be routed to the exact pods that can
serve them based on HTTP characteristics such as the
requested URL path. Other features include terminating
TLS, using multiple domains, and, most importantly, load
balancing traffic.
In order for Ingress objects to be usable, you must have an
Ingress Controller deployed within your cluster that
implements the Ingress rules as they are detected. An
Ingress Controller, like other types of controllers,
continuously watches for changes. Since pods in
Kubernetes have arbitrary IPs and ports, it is the
responsibility of an Ingress Controller to hide all internal
networking from you, the operator. You only need to
define which route is designated to a service and the
system will handle making the changes happen.
HAProxy in Kubernetes 11
It’s important to note that Ingress Controllers still need a
way to receive external traffic. This can be done by
exposing the Ingress Controller as a Kubernetes service
with either type NodePort or LoadBalancer. However, this
time, when you add an external load balancer, it will be for
the one service only and the external load balancer’s
configuration can be more static.
Conclusion
In this chapter, you learned about Ingress objects and
Ingress Controllers and how they solve the problem of
routing external traffic into your cluster in a more flexible
and cost-effective way than exposing services with
NodePorts and LoadBalancers alone.
In the next chapter, you'll learn how to install the HAProxy
Kubernetes Ingress Controller so that you can benefit from
its blazing fast performance and mature set of features.
HAProxy in Kubernetes 12
Where Next
Now that you've had an introduction to using the HAProxy
Kubernetes Ingress Controller, what should you do next?
First, know that the HAProxy Kubernetes Ingress
Controller is continuously evolving and new features are
released at a regular cadence. Check out its official GitHub
repository to see the latest version.
The next thing is to appreciate that we've only scratched
the surface. On the whole, Kubernetes covers a lot of
ground, including authentication, resilience, deployments,
and job scheduling. HAProxy fills a niche for routing and
load balancing, but because the platform is extensible,
you'll likely find new use cases and potential integrations
between HAProxy and other components.
File an issue on GitHub or consider trying to solve an
existing issue. If you have questions, you'll find people
quick to help on the HAProxy Slack, where there's a
channel dedicated to Kubernetes. Perhaps you will be the
next mentor to someone else who is picking up
Kubernetes for the first time. Whatever the case, we're
glad you've joined us.
GET THE COMPLETE EBOOK
Visit the HAProxy booth at KubeCon 2020 to
receive the complete eBook, or download it at
haproxy.com
© 2020 HAProxy Technologies
