Add AIBrix Custom Autoscaling Algorithm APA #223
Conversation
| // | ||
| // Huo, Qizheng, et al. "High Concurrency Response Strategy based on Kubernetes Horizontal Pod Autoscaler." | ||
| // Journal of Physics: Conference Series. Vol. 2451. No. 1. IOP Publishing, 2023. | ||
| func (k *KpaAutoscaler) APA_Scale(currentPodCount float64, currentUsePerPod float64, spec *DeciderKpaSpec) int32 { |
There was a problem hiding this comment.
Should we create a separate autoscaler for APA?
| return r.reconcileKPA(ctx, pa) | ||
| case autoscalingv1alpha1.APA: |
There was a problem hiding this comment.
Let's have some discussion here. I think we should create a new autoscaler. I understand some common parts have been implemented. We just need to figure out the long term plan. It looks good to me for short term
| @@ -53,6 +53,54 @@ spec: | |||
| targetPort: 8000 | |||
| nodePort: 30081 | |||
| type: NodePort | |||
| --- | |||
There was a problem hiding this comment.
Let's create a separate service account this time
There was a problem hiding this comment.
I added a commit to address this change. 2a5b389
506a803 to
d72a9c0
Compare
|
two remaining issues make a lot of sense.
Let's add it back. The reason I suggest to remove it in the last PR is we need a separate service account/role/rolebinding for it. Let's do not attach the role to the
Let me take a look as well. Let's better abstract the algorithm and autoscaler part. it indeed has a lots of code can be reused |
b58a21c to
2a5b389
Compare
* add apa algorithm * Add HPCA algorithm as a implementation of APA * fix make lint * Add separate pod-autoscaler service account --------- Co-authored-by: Jiaxin Shan <seedjeffwan@gmail.com>
Pull Request Description
Besides the existing KPA (KNative) and HPA (Kubernetes native) autoscalers, this pull request introduces a preliminary implementation of our custom autoscaling solution, APA. This implementation is inspired by
HCPAdiscussed in the paper HCPA.Remaining Issues
RBAC in e2e Tests: The end-to-end tests for both KPA and APA depend on the
/metricendpoint of a mocked deployment llama2, which requires sufficient RBAC permissions to access k8s replica information inside the container. As discussed in this conversation, these permissions were removed in a previous PR. Should we consider reverting that change?Code Duplication Between KPA and APA: There is significant overlap in the implementation for KPA and APA, particularly in areas like Spec definition, MetricClient, and the reconciling pipeline. For the time being, this PR introduces APA algorithm as a if-branch within the KPA scaling function. Please refer to the detailed discussion in issue [RFC] Design of Custom Autoscaling Algorithms for KPA and AIBrix #224 .
This PR is a work in progress, any feedback is highly appreciated.
Related Issues
Resolves: #224
Important: Before submitting, please complete the description above and review the checklist below.
Contribution Guidelines (Expand for Details)
We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:
Pull Request Title Format
Your PR title should start with one of these prefixes to indicate the nature of the change:
[Bug]: Corrections to existing functionality[CI]: Changes to build process or CI pipeline[Docs]: Updates or additions to documentation[API]: Modifications to aibrix's API or interface[CLI]: Changes or additions to the Command Line Interface[Misc]: For changes not covered above (use sparingly)Note: For changes spanning multiple categories, use multiple prefixes in order of importance.
Submission Checklist
By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.