Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Allow control plane provider to set endpoint #10667

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/book/src/developer/architecture/controllers/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ provisions EC2 instances that will become a Kubernetes cluster through some boot

The cluster controller will set an OwnerReference on the infrastructureCluster. This controller should normally take no action during reconciliation until it sees the OwnerReference.

An infrastructureCluster controller is expected to eventually have its `spec.controlPlaneEndpoint` set by the user/controller.
An infrastructureCluster controller is expected to either supply a controlPlaneEndpoint (via its own `spec.controlPlaneEndpoint` field),
or rely on `spec.controlPlaneEndpoint` in its parent [Cluster](./cluster.md) object.

If an endpoint is not provided, the implementer should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated.

The Cluster controller bubbles up `spec.controlPlaneEndpoint` and `status.ready` into `status.infrastructureReady` from the infrastructureCluster.

Expand Down Expand Up @@ -50,7 +53,7 @@ is a map, defined as `map[string]FailureDomainSpec`. A unique key must be used f
- `controlPlane` (bool): indicates if failure domain is appropriate for running control plane instances.
- `attributes` (`map[string]string`): arbitrary attributes for users to apply to a failure domain.

Note: once any of `failureReason` or `failureMessage` surface on the cluster who is referencing the infrastructureCluster object,
Note: once any of `failureReason` or `failureMessage` surface on the cluster who is referencing the infrastructureCluster object,
they cannot be restored anymore (it is considered a terminal error; the only way to recover is to delete and recreate the cluster).

Example:
Expand Down
39 changes: 35 additions & 4 deletions docs/book/src/developer/architecture/controllers/control-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,14 @@ Kubernetes control plane consisting of the following services:

The Cluster controller will set an OwnerReference on the Control Plane. The Control Plane controller should normally take no action during reconciliation until it sees the ownerReference.
vincepri marked this conversation as resolved.
Show resolved Hide resolved

A Control Plane controller implementation should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated.
A Control Plane controller implementation must either supply a controlPlaneEndpoint (via its own `spec.controlPlaneEndpoint` field),
or rely on `spec.controlPlaneEndpoint` in its parent [Cluster](./cluster.md) object.

The Cluster controller bubbles up `status.ready` into `status.controlPlaneReady` and `status.initialized` into a `controlPlaneInitialized` condition from the Control Plane CR.
If an endpoint is not provided, the implementer should exit reconciliation until it sees `cluster.spec.controlPlaneEndpoint` populated.

A Control Plane controller can optionally provide a `controlPlaneEndpoint`

The `ImplementationControlPlane` *must* rely on the existence of
`status.controlplaneEndpoint` in its parent [Cluster](./cluster.md) object.
The Cluster controller bubbles up `status.ready` into `status.controlPlaneReady` and `status.initialized` into a `controlPlaneInitialized` condition from the Control Plane CR.

### CRD contracts

Expand Down Expand Up @@ -110,6 +112,35 @@ documentation][scale].
deletion. A duration of 0 will retry deletion indefinitely. It defaults to 10 seconds on the
Machine.

#### Optional `spec` fields for implementations providing endpoints

The `ImplementationControlPlane` object may provide a `spec.controlPlaneEndpoint` field to inform the Cluster
controller where the endpoint is located.

Implementers might opt to choose the `APIEndpoint` struct exposed by Cluster API types, or the following:

<table>
<tr>
<th> Field </th>
<th> Type </th>
<th> Description </th>
</tr>
<tr>
<td><code>host</code></td>
<td>String</td>
<td>
The hostname on which the API server is serving.
</td>
</tr>
<tr>
<td><code>port</code></td>
<td>Integer</td>
<td>
The port on which the API server is serving.
</td>
</tr>
</table>

#### Required `status` fields

The `ImplementationControlPlane` object **must** have a `status` object.
Expand Down
1 change: 1 addition & 0 deletions docs/proposals/20230407-flexible-managed-k8s-endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ More specifically we would like to introduce first class support for two scenari

- Permit omitting the `<Infra>Cluster` entirely, thus making it simpler to use with Cluster API all the Managed Kubernetes implementations which do not require any additional Kubernetes Cluster Infrastructure (network settings, security groups, etc) on top of what is provided out of the box by the managed Kubernetes primitive offered by a Cloud provider.
- Allow the `ControlPlane Provider` component to take ownership of the responsibility of creating the control plane endpoint, thus making it simpler to use with Cluster API all the Managed Kubernetes implementations which are taking care out of the box of this piece of Cluster Infrastructure.
- Note: In May 2024 [this pull request](https://github.com/kubernetes-sigs/cluster-api/pull/10667) added the ability for the control plane provider to provide the endpoint the same way the infrastructure cluster would.

The above capabilities can be used alone or in combination depending on the requirements of a specific Managed Kubernetes or on the specific architecture/set of Cloud components being implemented.

Expand Down
17 changes: 16 additions & 1 deletion internal/controllers/cluster/cluster_controller_phases.go
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ func (r *Reconciler) reconcileInfrastructure(ctx context.Context, cluster *clust

// Get and parse Spec.ControlPlaneEndpoint field from the infrastructure provider.
if !cluster.Spec.ControlPlaneEndpoint.IsValid() {
if err := util.UnstructuredUnmarshalField(infraConfig, &cluster.Spec.ControlPlaneEndpoint, "spec", "controlPlaneEndpoint"); err != nil {
if err := util.UnstructuredUnmarshalField(infraConfig, &cluster.Spec.ControlPlaneEndpoint, "spec", "controlPlaneEndpoint"); err != nil && err != util.ErrUnstructuredFieldNotFound {
return ctrl.Result{}, errors.Wrapf(err, "failed to retrieve Spec.ControlPlaneEndpoint from infrastructure provider for Cluster %q in namespace %q",
cluster.Name, cluster.Namespace)
}
Expand All @@ -218,6 +218,8 @@ func (r *Reconciler) reconcileInfrastructure(ctx context.Context, cluster *clust

// reconcileControlPlane reconciles the Spec.ControlPlaneRef object on a Cluster.
func (r *Reconciler) reconcileControlPlane(ctx context.Context, cluster *clusterv1.Cluster) (ctrl.Result, error) {
vincepri marked this conversation as resolved.
Show resolved Hide resolved
log := ctrl.LoggerFrom(ctx)

if cluster.Spec.ControlPlaneRef == nil {
return ctrl.Result{}, nil
}
Expand Down Expand Up @@ -274,6 +276,19 @@ func (r *Reconciler) reconcileControlPlane(ctx context.Context, cluster *cluster
}
}

vincepri marked this conversation as resolved.
Show resolved Hide resolved
if !ready {
log.V(3).Info("Control Plane provider is not ready yet")
return ctrl.Result{}, nil
}

// Get and parse Spec.ControlPlaneEndpoint field from the control plane provider.
if !cluster.Spec.ControlPlaneEndpoint.IsValid() {
if err := util.UnstructuredUnmarshalField(controlPlaneConfig, &cluster.Spec.ControlPlaneEndpoint, "spec", "controlPlaneEndpoint"); err != nil && err != util.ErrUnstructuredFieldNotFound {
return ctrl.Result{}, errors.Wrapf(err, "failed to retrieve Spec.ControlPlaneEndpoint from control plane provider for Cluster %q in namespace %q",
cluster.Name, cluster.Namespace)
}
}

return ctrl.Result{}, nil
}

Expand Down
Loading
Loading