From ab819f3a2ddc3a5d76cc56c517d42cbb5a283f98 Mon Sep 17 00:00:00 2001 From: Michael Ruoss Date: Thu, 9 Feb 2023 22:05:29 +0100 Subject: [PATCH 1/6] formatting --- guides/the_operator.livemd | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/guides/the_operator.livemd b/guides/the_operator.livemd index 6e25e25..27ad0b3 100644 --- a/guides/the_operator.livemd +++ b/guides/the_operator.livemd @@ -32,13 +32,13 @@ The operator defines custom resources, watch queries and their controllers and s Overall, an operator has the following responsibilities: -* to provide a wrapper for starting and stopping the +- to provide a wrapper for starting and stopping the operator as part of a supervision tree -* To define the resources to be watched together with the +- To define the resources to be watched together with the controllers which handle action events on those resources. -* to define an initial pluggable pipeline containing the step `:delegate_to_controller` for all action events +- to define an initial pluggable pipeline containing the step `:delegate_to_controller` for all action events to pass through -* To define any custom resources ending up in the manifest +- To define any custom resources ending up in the manifest generated by `mix bonny.gen.manifest` ```elixir @@ -104,8 +104,8 @@ crds In `controllers/2` we define the queries and their event handlers, i.e. controllers. function should return a list where each element of the list is a map with these 2 keys: -* `:query` - A list operation of type `K8s.Operation.t()`. Bonny will watch the cluster with this operation and forward all events to the `:controller`. -* `:controller` - A controller (See the [controller guide](controllers.livemd)) or any other `Pluggable` step. Accepts a module or a `{controller :: module(), init_opts :: keyword()}` tuple. If a tuple is given, the init_opts are passed to the controller's `init/1` function. +- `:query` - A list operation of type `K8s.Operation.t()`. Bonny will watch the cluster with this operation and forward all events to the `:controller`. +- `:controller` - A controller (See the [controller guide](controllers.livemd)) or any other `Pluggable` step. Accepts a module or a `{controller :: module(), init_opts :: keyword()}` tuple. If a tuple is given, the init_opts are passed to the controller's `init/1` function. If you managed to define a valid `conn` above, you can now run the operator defined above in this livebook. The code below starts the operator and shows the supervision tree. Note how the operator starts an `EventRecorder` and two proceses for each controller defined in `controllers/2`. These two processes are the `Watcher` and the `Reconciler`. The `Watcher` watches for `ADD`, `MODIFY` and `DELETE` events in the cluster. The `Reconciler` regularly creates `:reconcile` events for each resource found in the cluster. @@ -139,8 +139,8 @@ The operator implements a [Pluggable](https://hex.pm/packages/pluggable) pipelin Bonny comes with a few steps to your convenience. In most caes it makes sense to add at least `Bonny.Pluggable.ApplyDescendants` and `Bonny.Pluggable.ApplyStatus` to the end of your operator pipeline. -* `Bonny.Pluggable.AddManagedByLabelToDescendants` - Adds the `app.kubernetes.io/managed-by` label to all descendants registered within the pipeline. -* `Bonny.Pluggable.ApplyDescendants` - applies all the descendants added to the `%Bonny.Axn{}` struct. -* `Bonny.Pluggable.ApplyStatus` - applies the status of the given `%Bonny.Axn{}` struct to the status subresource. -* `Bonny.Pluggable.Logger`- logs an action event and when status, descendants and events are applied to the cluster. If desired, it makes sense to be placed as first step in your operator pipeline but can also be added to a controller pipeline. -* `Bonny.Pluggable.SkipObservedGenerations` - halts the pipelines for a defined list of actions if the observed generation equals the resource's generation. You'll find further documentation on this module in the [Controller](controllers.livemd) guide. +- `Bonny.Pluggable.AddManagedByLabelToDescendants` - Adds the `app.kubernetes.io/managed-by` label to all descendants registered within the pipeline. +- `Bonny.Pluggable.ApplyDescendants` - applies all the descendants added to the `%Bonny.Axn{}` struct. +- `Bonny.Pluggable.ApplyStatus` - applies the status of the given `%Bonny.Axn{}` struct to the status subresource. +- `Bonny.Pluggable.Logger`- logs an action event and when status, descendants and events are applied to the cluster. If desired, it makes sense to be placed as first step in your operator pipeline but can also be added to a controller pipeline. +- `Bonny.Pluggable.SkipObservedGenerations` - halts the pipelines for a defined list of actions if the observed generation equals the resource's generation. You'll find further documentation on this module in the [Controller](controllers.livemd) guide. From 9d91f1a96da01fd09b09a6818b68f61a0304c652 Mon Sep 17 00:00:00 2001 From: Michael Ruoss Date: Thu, 9 Feb 2023 22:06:05 +0100 Subject: [PATCH 2/6] Add test for Bonny.Pluggable.AddMissingGVK Step --- test/bonny/pluggable/add_missing_gvk_test.exs | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 test/bonny/pluggable/add_missing_gvk_test.exs diff --git a/test/bonny/pluggable/add_missing_gvk_test.exs b/test/bonny/pluggable/add_missing_gvk_test.exs new file mode 100644 index 0000000..b4a106e --- /dev/null +++ b/test/bonny/pluggable/add_missing_gvk_test.exs @@ -0,0 +1,26 @@ +defmodule Bonny.Pluggable.AddMissingGVKTest do + use ExUnit.Case, async: true + use Bonny.Axn.Test + + import YamlElixir.Sigil + + alias Bonny.Pluggable.AddMissingGVK, as: MUT + + test "adds apiVersion and kind fields to the resource" do + opts = MUT.init(apiVersion: "v1", kind: "ConfigMap") + + resource = ~y""" + metadata: + name: foo + namespace: default + data: + key: value + """ + + result = MUT.call(axn(:add, resource: resource), opts) + + # generation already observed + assert "v1" == result.resource["apiVersion"] + assert "ConfigMap" == result.resource["kind"] + end +end From 757cd53a85ebf88e5b8aab6edf413f10d868e436 Mon Sep 17 00:00:00 2001 From: Michael Ruoss Date: Thu, 9 Feb 2023 22:07:27 +0100 Subject: [PATCH 3/6] add implementation of Bonny.Pluggable.AddMissingGVK --- lib/bonny/pluggable/add_missing_gvk.ex | 37 ++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 lib/bonny/pluggable/add_missing_gvk.ex diff --git a/lib/bonny/pluggable/add_missing_gvk.ex b/lib/bonny/pluggable/add_missing_gvk.ex new file mode 100644 index 0000000..62a8e23 --- /dev/null +++ b/lib/bonny/pluggable/add_missing_gvk.ex @@ -0,0 +1,37 @@ +defmodule Bonny.Pluggable.AddMissingGVK do + @moduledoc """ + The Kubernetes API sometimes doesn't set the fields `apiVersion` and `kind` on + items of a list operation. E.g. if you use `K8s.Client` to get a list of + deployments, the deployments won't contains those two fields. This is being + discussed on https://github.com/kubernetes/kubernetes/issues/3030. + + Bonny sometimes depends on `apiVersion` and `kind` to be defined on the + resource being handled. This is the case for setting the status, e.g. when + using `Bonny.Pluggable.SkipObservedGenerations` or `Bonny.Axn.update_status/2`. + + Add this step to your controller in order to set those values on all resources + being handled. + + + ## Examples + + step Bonny.Pluggable.AddMissingGVK, + apiVersion: "apps/v1", + kind: "Deployment" + """ + + @behaviour Pluggable + + @impl true + def init(opts \\ []), + do: %{ + "apiVersion" => Keyword.fetch!(opts, :apiVersion), + "kind" => Keyword.fetch!(opts, :kind) + } + + @impl true + def call(axn, config) do + resource = Map.merge(config, axn.resource) + struct!(axn, resource: resource) + end +end From 93f4d0cb9ad11027b24c11b7f526fb95688c5f46 Mon Sep 17 00:00:00 2001 From: Michael Ruoss Date: Thu, 9 Feb 2023 22:07:34 +0100 Subject: [PATCH 4/6] Add docs --- guides/the_operator.livemd | 1 + test/bonny/pluggable/skip_observed_generations_test.exs | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/guides/the_operator.livemd b/guides/the_operator.livemd index 27ad0b3..3c86b69 100644 --- a/guides/the_operator.livemd +++ b/guides/the_operator.livemd @@ -140,6 +140,7 @@ The operator implements a [Pluggable](https://hex.pm/packages/pluggable) pipelin Bonny comes with a few steps to your convenience. In most caes it makes sense to add at least `Bonny.Pluggable.ApplyDescendants` and `Bonny.Pluggable.ApplyStatus` to the end of your operator pipeline. - `Bonny.Pluggable.AddManagedByLabelToDescendants` - Adds the `app.kubernetes.io/managed-by` label to all descendants registered within the pipeline. +- `Bonny.Pluggable.AddMissingGVK` - Add fields `apiVersion` and `kind` to the resource if they are missing. - `Bonny.Pluggable.ApplyDescendants` - applies all the descendants added to the `%Bonny.Axn{}` struct. - `Bonny.Pluggable.ApplyStatus` - applies the status of the given `%Bonny.Axn{}` struct to the status subresource. - `Bonny.Pluggable.Logger`- logs an action event and when status, descendants and events are applied to the cluster. If desired, it makes sense to be placed as first step in your operator pipeline but can also be added to a controller pipeline. diff --git a/test/bonny/pluggable/skip_observed_generations_test.exs b/test/bonny/pluggable/skip_observed_generations_test.exs index f877686..f6678b6 100644 --- a/test/bonny/pluggable/skip_observed_generations_test.exs +++ b/test/bonny/pluggable/skip_observed_generations_test.exs @@ -1,5 +1,5 @@ defmodule Bonny.Pluggable.SkipObservedGenerationsTest do - use ExUnit.Case + use ExUnit.Case, async: true use Bonny.Axn.Test alias Bonny.Pluggable.SkipObservedGenerations, as: MUT From 1e28aa3975fd866933c9d563a4f5aa8c0434cddc Mon Sep 17 00:00:00 2001 From: Michael Ruoss Date: Thu, 9 Feb 2023 22:10:50 +0100 Subject: [PATCH 5/6] add changelog --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 40ee9e0..e8d3e4f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 +### Added + +- A pluggable step for adding missing GVK information to resources - [#184](https://github.com/coryodaniel/bonny/pull/184), [#183](https://github.com/coryodaniel/bonny/issues/183) + ## [1.0.1] - 2023-02-04 From a7408b0667d4e08fd650bb803072cae2c334625e Mon Sep 17 00:00:00 2001 From: Michael Ruoss Date: Thu, 9 Feb 2023 22:11:49 +0100 Subject: [PATCH 6/6] empty opts not allowed --- lib/bonny/pluggable/add_missing_gvk.ex | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bonny/pluggable/add_missing_gvk.ex b/lib/bonny/pluggable/add_missing_gvk.ex index 62a8e23..3ec50f2 100644 --- a/lib/bonny/pluggable/add_missing_gvk.ex +++ b/lib/bonny/pluggable/add_missing_gvk.ex @@ -23,7 +23,7 @@ defmodule Bonny.Pluggable.AddMissingGVK do @behaviour Pluggable @impl true - def init(opts \\ []), + def init(opts), do: %{ "apiVersion" => Keyword.fetch!(opts, :apiVersion), "kind" => Keyword.fetch!(opts, :kind)