Update Kubernetes deps to 1.16 #995
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
OK, this was a bit of a saga. In short: Kubernetes dependency management is broken, subprojects don't talk to each other, nobody versions things the same way, and I'm not sure how this all works. In short, Kubernetes 1.14 to 1.16 have made massive a large series of breaking changes between code generation and client interfaces. This is all fine, because we can update using Go modules, right? It's a nice thought, but totally wrong. Because there is no coordinated release strategy for SIG tooling, you cannot depend on SIGs to keep their dependencies up to date with Kubernetes. In this instance, controller-runtime up until recently was pinned to client-go v11. This would be fine, except for a breaking change. Go Modules also uses the property of minimum version selection to solve modules, and KUDO, core Kubernetes projects, and controller-runtime (and another project we'll discuss in a moment!) all had a diamond dependency. In short, as soon as you update client-go, controller-runtime is broken, but as soon as you update controller-runtime, client-go is broken, unless you downgrade _everything_ to Kubernetes 1.14. Ok, so this is now fixed on master with some review and help. So we go to update to 1.16, and lo and behold, it all looks like it's working - until we go to compile the controller and CLI. Suddenly, client-go is forced _back_ down to 1.14, breaking everything again. Who's the culprit this time? According to `go mod graph`, it's Kustomize, which despite being officially included in the Kubernetes CLI, is still using client-go v11. This is so comical this point, I forced client-go to 1.16, which had _more_ breaking changes in the codegen signatures! I re-generated client-gen off of a 1.16 compatible version, and this all builds now and tests fail. Really, this makes me want to throw out our dependencies and use a discovery based interface. The situation in Kubernetes is so bad for external author developers, that I'm not sure how someone sanely keeps an operator up to date. I guess that's the point of KUDO, but we should set it all on fire and just use Clojure or something. Codegen is an abomination.
|
Also, so far, it seems like just overriding client-go works fine. It's the lynchpin that causes the cascading nightmare. |
nfnt
reviewed
Oct 24, 2019
nfnt
approved these changes
Oct 24, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
OK, this was a bit of a saga. In short: Kubernetes dependency management is broken, subprojects don't talk to each other, nobody versions things the same way, and I'm not sure how this all works.
In short, Kubernetes 1.14 to 1.16 have made a series of breaking changes between code generation and client interfaces. This is all fine, because we can update using Go modules, right?
It's a nice thought, but totally wrong. Because there is no coordinated release strategy for SIG tooling, you cannot depend on SIGs to keep their dependencies up to date with Kubernetes. In this instance, controller-runtime up until recently was pinned to client-go v11. This would be fine, except for a breaking change. Go Modules also uses the property of minimum version selection to solve modules, and KUDO, core Kubernetes projects, and controller-runtime (and another project we'll discuss in a moment!) all had a diamond dependency. In short, as soon as you update client-go, controller-runtime is broken, but as soon as you update controller-runtime, client-go is broken, unless you downgrade everything to Kubernetes 1.14.
Ok, so this is now fixed on master with some review and help. So we go to update to 1.16, and lo and behold, it all looks like it's working - until we go to compile the controller and CLI. Suddenly, client-go is forced back down to 1.14, breaking everything again. Who's the culprit this time? According to
go mod graph, it's Kustomize, which despite being officially included in the Kubernetes CLI, is still using client-go v11.This is so comical this point, I forced client-go to 1.16, which had more breaking changes in the codegen signatures! I re-generated client-gen off of a 1.16 compatible version, and this all builds now and tests pass.
Really, this makes me want to throw out our dependencies and use a discovery based interface. The situation in Kubernetes is so frustrating for external developers that I'm not sure how someone sanely keeps an operator up to date. I guess that's the point of KUDO, but we should set it all on fire and just use Clojure or Elixir. Codegen is an abomination.