machineAutoscaler CRD

[enxebre]

⚠️ Cluster API maintainers can ask to turn an issue-proposal into a CAEP when necessary, this is to be expected for large changes that impact multiple components, breaking changes, or new large features.

Goals

1. Provide operators with an opt in mechanism for scalable resources i.e machineSet/machineDeployment to be considered for autoscaling.

Non-Goals/Future Work

1. Discuss cluster-api autoscaler provider logic.

User Story

1. As a user I want my cluster to be cost effective so I need my node groups to grow and shrink automatically according to the existing workload at anytime.

2. As an operator I want an opt in mechanism so I can choose which of the scalable resources i.e machineSet/machineDeployment in my cluster I want to be considered for autoscaling.

Detailed Description

If we add support for a cluster-api provider for the autoscaler we'll need an automated mechanism to let the autoscaler know which scalable resources it should take into consideration for autoscaling purposes.

This is to propose a new CRD machineAutoscaler along with a controller which targets individual scalable resources and annotates them appropriately so they can be recognised by a cluster-api autoscaler provider and taken into consideration for autoscaling.

nodeGroupMinSizeAnnotationKey = "cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size"

nodeGroupMaxSizeAnnotationKey = "cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size"

Aside from this concise RFE but dependent we'd like to find consensus to revive the PRs below, merge a cluster-api autoscaler provider which would support the >= v1alpha2 types and commit to its maintenance.
openshift/kubernetes-autoscaler#118
kubernetes/autoscaler#1866
kubernetes/enhancements#609

Contract changes [optional]

[Describe contract changes between Cluster API controllers, if applicable.]

Data model changes [optional]

apiVersion: "cluster.x-k8s.io"
kind: "MachineAutoscaler"
metadata:
  name: "worker-us-east-1a" 
  namespace: "cluster-api"
spec:
  minReplicas: 1 
  maxReplicas: 12 
  scaleTargetRef: 
    apiVersion: cluster.x-k8s.io
    kind: MachineSet 
    name: worker-us-east-1a

/kind proposal

[enxebre]

cc @elmiko @ncdc @detiber

[randomvariable]

Subscribing to this

[vincepri]

/milestone Next

[enxebre]

/assign @enxebre

[mooperd]

+1

[michaelgugino]

It's unclear to me what the annotations are for in the description.

I think I like the idea of having an opaque object for the cluster-autoscaler to interact with instead of having to code it to handle a variety of types (which may expand in the future).

Perhaps, though, this new CRD is better targeted at the cluster-autoscaler directly, rather than the cluster-api. This way, there could be other external reconcilers for this new CRD rather than just cluster-api ones. We would then write an implementation for said CRD.

[elmiko]

Perhaps, though, this new CRD is better targeted at the cluster-autoscaler directly, rather than the cluster-api. This way, there could be other external reconcilers for this new CRD rather than just cluster-api ones. We would then write an implementation for said CRD.

just so i follow, this is suggesting that the MachineAutoscaler resource would be owned by the cluster-autoscaler project but cluster-api (and perhaps others) could consume these resources to perform action?

[michaelgugino]

Perhaps, though, this new CRD is better targeted at the cluster-autoscaler directly, rather than the cluster-api. This way, there could be other external reconcilers for this new CRD rather than just cluster-api ones. We would then write an implementation for said CRD.

just so i follow, this is suggesting that the MachineAutoscaler resource would be owned by the cluster-autoscaler project but cluster-api (and perhaps others) could consume these resources to perform action?

@elmiko yes, that's what I'm suggesting.

Perhaps this resource should be scalable itself. The cluster-autoscaler just scales this resource. We write a controller that syncs the resource info to the status here, we write a controller that syncs replica counts from this to desired machineset/deployment/whatever. On scale down, autoscaler annotates some nodes it wants to go away, our new controller annotates machines appropriately, scales down machineset.

This would be a large deviation from where we are now, just an idea.

[vincepri]

As per April 27th 2020 community guidelines, this project follows the process outlined in https://github.com/kubernetes-sigs/cluster-api/blob/master/CONTRIBUTING.md#proposal-process-caep for large features or changes.

Following those guidelines, I'll go ahead and close this issue for now and defer to contributors interested in pushing the proposal forward to open a collaborative document proposal instead, and follow the process as described.

/close

[k8s-ci-robot]

@vincepri: Closing this issue.

In response to this:

As per April 27th 2020 community guidelines, this project follows the process outlined in https://github.com/kubernetes-sigs/cluster-api/blob/master/CONTRIBUTING.md#proposal-process-caep for large features or changes.

Following those guidelines, I'll go ahead and close this issue for now and defer to contributors interested in pushing the proposal forward to open a collaborative document proposal instead, and follow the process as described.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.