Top Ten Challenges - Part 5: Multi-tenancy

(This is part 5 of a series about challenges and considerations that I have come across with customers when deploying software into Kubernetes. You can find the intro post here.)

While this topic seems somewhat trivial ("oh, by the way, let's set this up for multi-tenancy"), I find it to be one of the trickier ones. It starts out with agreeing on the definition of what a "tenant" is, how it is represented in the deployed software application and then how it can be implemented and configured accordingly.

To me, multi-tenancy is really about isolation, in other words, the level at which software supports multi-tenancy is related to the degree of isolation (and sharing!) it supports. Because it is expected to give a user, or rather, a group of users (i.e. the "tenant") the impression of having an instance of the application all to themselves. They can log in, they can customize what they see and how they interact with the application, and they can be assured that no other tenant can access their data. Easy, right? As an example, cloud environments are supposed to behave that way. You create an "instance" of a service, say, a database service and you get access to what looks like a private database server instance. But it isn't. To make it work (and cost effective!) at cloud scale, this database service instance will most likely be set up on shared infrastructure and use other shared services to do its job.

And that is where it becomes interesting: how do I support multi-tenancy in Kubernetes? In the most extreme case, I can give each tenant their own cluster, with the application on it. After all, that way I get maximum isolation and separation of tenants from each other. But in most cases, it's obviously not efficient. You want a Kubernetes cluster to be shared. So how do you achieve isolation and separation on a cluster? By using namespaces. You could, for example, deploy multiple copies of an application into separate namespaces, one namespace per tenant. As mentioned in the previous blog post, though, there are some resources that are cluster scoped and therefore will be shared, no matter what. But that is not necessarily a problem in this context, of course. 

Keep in mind that a Kubernetes cluster is not designed for true multi-tenancy today. It comes with a single control plane, API server, and etcd data store, and these are all shared, regardless of whatever desire you may have to isolate cluster tenant users from each other. This Kubernetes blog offers a nice summary of some of the options I mentioned above and includes pointers to related projects currently happening in the community. It mentions the ability to have separate, per-tenant control planes and shared workers in a single cluster, but I have to admit that I have never seen a cluster like that, and I doubt that OpenShift supports it to begin with.

Besides creating a multi-tenant cluster, multi-tenancy can also be completely defined at the application level. That is, an application can have a tenancy model for users (for example, by introducing the notion of an "account", basically mimicking what you see in a public cloud), it can expose "instances" via service broker (or use the Service Binding Operator) and provide for a degree of separation in the data layer (and, for example, support tenant-based backup/restore or encryption of the data), all while running in a single Kubernetes namespace! It has to be architected and implemented accordingly, of course, and if it hasn't been, I have found it next to impossible to turn a single-tenant application into a multi-tenant one without major refactoring or even rewrite.

My takeaway in all this is that multi-tenancy in Kubernetes is easily achievable for replicating, that is, by replicating clusters, or by replicating deployments in different namespaces. But that rarely results in efficient use of the underlying resources and is not cost effective, especially when running applications for a large number of tenants. Introducing multi-tenancy at the application level is a more efficient alternative, but it has to be architected from the ground up.

(Photo by NeONBRAND on Unsplash.)