Kubernetes + LetsEncrypt: it seemed worth investigating …

Mark McWiggins
3 min readAug 8, 2021

I am working on contract for a small division of a large company. There are just a very few of us techies working at this place.

One of my first tasks was to get a “real” SSL certificate set up to point to the web address that would be pointing to a GKE load balancer. How hard could it be?

A complex systems diagram of a Kubernetes installation
Hidden costs of Kubernetes

I spent probably 8 full work days at this; here’s my report.

I don’t have a ton of experience with Kubernetes; I’ve been working intermittently with it for several months. I have been using LetsEncrypt happily on my own websites for several years. I’ve found it very easy to set up and … essentially to set and forget!

Let’s Encrypt logo
Let’s Encrypt: The SSL Cert authority of choice, at least for small sites

But I had no idea that LetsEncrypt had a whole different infrastructure for use with Kubernetes, and it’s complex enough that there’s a set of Kubernetes pods called cert-manager that’s arisen to manage it. I thought I would try this …

The first thing I do in any circumstance like this is to google for examples. I did this and found several. But! Most were from 2019, and Kubernetes’ powers that be never got the “don’t break existing APIs” mantra down. Almost all of the 2019 examples needed at least one change to even run against current clusters.

Also (a) a lot of the more recent examples like this one seem incredibly complex for what they are illustrating (huge rawgithubcontent and a flag ‘— validate=false’ — why?) and (b) I could not get any of them to work!

I finally got to the bottom of things this afternoon with this one … once again, that required syntax changes (this one from April 2021!) But this one finally had some suggestions for debugging things that weren’t working, and I found it:

Waiting for HTTP-01 challenge preparation: wrong status code ‘404’, expected ‘200’.

OK, but that was 3 levels deep in the kubectl command list … it finally struck me that this was going to just be too difficult to manage for this little organization, and I found a simpler solution for them, I think.

Clouflare supports SSL connections allowing users to see a real CA-blessed certificate on the front end, but connects to back end servers with self-signed certificates. This is much easier … just add the certificate to the Docker image we’re using and we’re good to go.

What if I were interested in Kubernetes’ greater adoption?

  • Stop breaking APIs … they may have a sort of way of doing this, the flag ‘— validate=false’ , but I didn’t try this as I wasn’t sure enough what I was doing without confusing the issue with this flag
  • Support a cookbook of common use cases, kept updated for every common cloud platform. Regression test with every change to make sure it still works on all platforms.

I doubt this will happen, though … Kubernetes is now at the top of its “hype cycle” even though there are skeptics … I guess after this experience I have to add my voice to the skeptical chorus!

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

No responses yet

Write a response