Kubernetes is a symptom, not a solution

by gskyon 7/4/2025, 12:42 AMwith 30 comments

by refactor_masteron 7/4/2025, 1:35 AM

> Docker is essentially a sandwich of disk images where you can shove absolutely anything, and then these images get executed by running whatever legacy software you’ve crammed in there, regardless of how horrific or inconsistent it might be, with zero behavioral controls.

Is this a problem with Docker, or a problem with people who use Docker? Without much controversy I feel this argument could be made about literally any abstract entity in society from `get_foo` to national institutions. Abstractions seem to be a necessary evil in corporative societies.

Also, the article is way too short and meatless to provoke any real thoughts, and comes across as "If you don't already know what Kubernetes is, just know that it's bad". I don't see how that's going to sway anyone's opinion on anything.

by nisaon 7/4/2025, 1:25 AM

This (ragebaity/ai?) post kind of mixes things up. Kubernetes is fine I think but almost everything around it and the whole consulting business is the problem. You can build a single binary, use a single layer oci container and run it with a single config map with memory quota on 30 machines just fine.

Take a look at the early borg papers what problem does it solves. Helm is just insane but you can use jsonnet that is modelled after Google's internal system.

Only use the minimal subset and have an application that is actually build to work fine in that subset.

by lixtraon 7/4/2025, 1:26 AM

Maybe thought provoking. But sad to read ai garbage. It’s easy to imagine a better world. But you also need to provide a way to reach it. For example a lot of things are wrong with docker. But it enables us to run yesterday’s software in the cloud. Tomorrow’s software is not written yet.

by kkfxon 7/4/2025, 5:07 AM

The cloud is someone else computer and the deploy solution alternatives are declarative systems like NixOS or Guix, finally adopted by a large community instead of wasting resources and enlarging attack surfaces by using container tech in general.

The underneath core problem is that OSes are not a single user-programmable application where any "program" is just a set of functions like original Smalltalk workstations or LispM. They are hard to evolve, but much quicker to experiment and gives ground for much quick innovation instead of wasting resources with giant boilerplate code and walled gardens.

Another underneath core problem is hardware where vendors in 2025 still do not care about light-out management on anything. These days an IP-KVM should and must be built-in in any computer.

by lambdasquirrelon 7/4/2025, 1:45 AM

The problem with this thesis is that there’s no other easy way to make microservices composed of different language stacks work well together, even if they are distributed-first. What happens when you jump from Elixir to Scala actors? It can’t easily be said that k8s is just an assembler of garbage.

Ultimately, the AI content is much like so much of content in this internet age: really good at convincing people who lack the background to call BS, but easily disproved by anyone with basic background.

by Uehrekaon 7/4/2025, 1:39 AM

The “Unpopular Opinion framing” is such a wonderful cheat code. You get to seem like an independent thinker for free, your argument gets bonus moral superiority points, and if anyone points out that your argument is actually very popular you get to claim it’s all a joke and mock them for taking you seriously (even though you actually mean what you’re saying).

by amaion 7/4/2025, 6:07 PM

There are alternatives to Kubernetes like Nomad: https://www.hashicorp.com/de/products/nomad

by turtlebitson 7/4/2025, 2:26 AM

The OP's stated ideal world is Erlang/Unison, WASI and OpenServerless. Why not champion those instead of being a contrarian? I'm always open to seeing whats better.

by cwilluon 7/4/2025, 2:25 AM

Can we start marking predominately ai submissions with an [AI] when it's not already obvious from the title?

by remramon 7/4/2025, 2:16 AM

At least the article is honest about being a clickbaity AI-generated attack...

by charles_fon 7/4/2025, 5:18 AM

I remember a world where you had to manually connect to a physical machine then copy and paste DLLs around tp deploy stuff. A batch script (that occasionally worked) was great innovation when it came to deployment. Not that long ago I worked with a company where it took persuasion to stop the madness of manually deploying to "the cloud" from an IDE at night when the traffic was slower.

Kubernetes is borderline painful, because it's specific, you need to learn it, you'll get burnt a bunch of times and unlike a VM you can't connect and reboot. And it has this tendency like Javascript to have breaking changes every 9 months. But if you're doing anything at some scale, it's also pretty good once it's set. If you don't, and what you're looking for is simplicity, go for one of the functions / lambda / heroku alternatives. Yes, lock-in, but also not that much if you abstain from leaning into the proprietary too much. Which brings me to that

> Kubernetes has also created a powerful lock-in effect

What? Isn't lock-in to kubernetes the same lock-in you have with your programming language, databases and largely architecture decisions? I mean if you're using one of these cloud resources sure, but generally when just relying on the orchestration, it actually provides the opposite.

by jauntywundrkindon 7/4/2025, 2:24 AM

What a diseased miserable propaganda hit job, "by AI" and also they confess sort of not.

I actually like a lot do the hope & wants expressed. Yes, WASI has huge promise to be a much better CORBA that I too am incredibly excited to watch develop, with hope. But it's not here, wasi preview2 is much better, but is still incredibly sync only with an even more bare bones than Rust (!) "poll" (do all of the actual work now, not poll.for completion) system.

I'm trying to cool my jets. So much feels not just biased, but outright idiotic & willing to lie like crazy. I can see this hate being so popular, such a dark side fan base lapping up the disdain that drips off such a lopsided ungraceful hatchet job.

> whatever random stuff you decided to put inside it!" The only reason this approach gained traction is because it saves upload space compared to creating full virtual machines. That’s it.

Generously, this is written by an ignorant idiot: best case. Its hard to credit the AI/person even ignorance. Seems everyone in the last 5 years at least oughtta be damned well aware that containers have pretty real advantages versus VMs.

Just talking purely about how your machine runs VMs vs containers, there's so many wins. Considerably less CPU & latency overhead because you only need to run one kernel, rather than dozens or hundreds which are all trying to schedule but which don't actually have scheduling authority. And that one kernel can allocate memory more efficiently and potentially share page cache across images, radically improving both memory and cache utilization.

Manageability is so much better. There are probably some attempts out there for base VMs, but a full OS tends to have a lot of corp config and twiddling. Where-as you can grab a node or python or whatever base image, built on a nice slim OS or distroless (no starter required but https://github.com/GoogleContainerTools/distroless is a resource), and add just a little more of your stuff on top.

The profile of what needs to be managed with containers is much much much less; the host and your container runtime provide much of what a VM would need configured on it.

It's stunning to find this variety of simple uninformed rage bait, that didn't pass muster 5 years ago! Yes, the convenience of layers pulling in what you want is high. But that's 1. Good and 2. Also backed by many significant technical wins vs VMs.

> From this fundamentally broken concept of componentization, we’ve constructed the entire Kubernetes ecosystem, which exists primarily to solve the problem of managing these semantically meaningless horrors by aggregating all possible kinds of digital garbage into something that resembles a functioning system.

Like, alas, yes maybe? Actually running modern services requires an amalgamation of more than just code.

Maybe we are just talking about the container itself, in which case this is absurd. You can fill a container up with whatever. I think VM's are much worse bucket of bit problems, usually without simple Dockerfiles to explain themselves. The container can be extremely minimal, again, as Distroless shows. Containers feel far less of an offense than what happens with modern package management, which companies typically just don't do a good job of at all.

But more generously I think we take this snipe as a bigger view, about Kubernetes breadth and scope and all the other pieces.

And there, it is a ridiculously preposterously one sized critique, that 100% is locked on to the thing developers are closest to (code) and which ignores everything else that's cloud.

Perhaps yes there are better ways to send code to systems. Perhaps this is far more complex than what you'd want to send up code.

But what makes Kubernetes popular and different from the many "ways to run code" whose corpses decay along the path of history is that none of them provided much other than code. I talked about what containers provide versus needing to configure VMs, and that goes 5x for Kubernetes. You can pass in secrets, pass in your various aws resources with ACK, pass in databases from psotgres-operator, pass in storage, pass in fancy networking with multus, now with new Kube you can pass on GPUs.

Kube provides. It's like dependency injection for computing (rather than code) at large. Your manifest asks for some stuff it needs, and the platform makes it available. This is such a powerful distinguishing advancement over where we were. And it's so extensible, so readily able to model and provide such an endless variety of manageability across the whole breadth of cloud stuff. Cloud stuff that is in no way unique to Kubernetes. This criticism isn't against Kubernetes, it's against system design that depends on anything, it's criticism that hates CloudFormation and Terraform at least as bitterly for daring to let your service have dependencies.

> Deploying a system on Kubernetes is always a nightmare of managing a collection of unhinged components that can explode at any moment for any reason.

So is the real world. So is running things.

Having a consistent place where everything is managed via similar tools is a great relief, versus every system blowing up in its own way. Versus every dependency having its own bespoke operational paradigm.

And it heals so much better! Kubernetes's biggest problem is that it's so damned autonomic, it self heals so well, that we never even have to get the intimate gory bloody experience we had to claw through. Teams aren't setting up their own postgres HA and backup and replication by hand, over multiple sprints. They pull down a very professional very tested operator, and the controller works, it operates the cluster across a huge range of failures incredibly well. We are bad at debugging and understanding Kubernetes because it mostly works extremely fantastically robustly and well without us.

> How do we create software components for distributed computing that can be easily deployed in the cloud?

And,

> And here’s the kicker: universal components that work across all languages already exist. Look at WASI (WebAssembly System Interface), components built on WebAssembly that provide a platform-independent, efficient format that any language can compile to.

I talked to this some. I too have huge hopes in WASI. I want that future so so bad. But it's not here yet (in any meanginfully usable async fashion), and it's speculative that it's all going to work well someday (fingers very crossed, I want it). Watch a Luke Wagner talk and get hype, get involved, find out! https://youtu.be/W3f8AAte0LM

But even when WASI is here, i 100% am going to be using wasmCloud (or something else) to manage WebAssembly runtimes, doing more and more, on Kubernetes, using the storage, config, networking, and secret primitives provided by the base Kubernetes platform, using other services like postgres and nats and redis, which are managed by very good very professional Kubernetes controllers.

The paradigm of Kubernetes brings incredible clarity to help us manage the sea of things. In an miraculously-extensible cross type-of-thing way we have never enjoyed the luxury of before. With a paradigm of reliability we've rarely enjoyed.

Wasm will do a lot, but it's just code. We still ought want & be working on management layers. There's so much value to the powerful platform providing world Kubernetes advanced us towards, that containers advanced us beyond VMs with. Looking forward to more. Loving seeing the managed stuff.

The symptom is that the world is complex. Eliminating or never going to settled agriculture would have resulted in none of these problems happening. But that's not the sort of solution we really want, that lets us do amazing things, that creates a mature capable world. We have to straddle and cope with complexity, and simply retreating from it and calling it bad like this article does again and again is such a retro-naivety that so darkly refuses to acknowledge that while complexity sometimes hurts, while we say we want simplicity, we have to also balance that we are an advanced civilization & we want the nice things that advanced civilizations come with, and that simply pop poo-ing and scouring the complexity, while hip and edgy, isn't an sufficient practical or coherent line of thought that merits giving up like that. On we go.