feat: on-demand capacity reservation support #7726

jmdeal · 2025-02-11T13:26:41Z

Fixes #N/A

Description
Implements the AWS provider component for ODCR support.

How was this change tested?

Does this change impact docs?

Yes, PR includes docs updates
Yes, issue opened: #
No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

netlify · 2025-02-11T13:26:57Z

✅ Deploy Preview for karpenter-docs-prod canceled.

Name	Link
🔨 Latest commit	`199602b`
🔍 Latest deploy log	https://app.netlify.com/sites/karpenter-docs-prod/deploys/67bf0011ecbd3100085a772b

jonathan-innis

First pass :) I didn't get through all of the files, though yet. I just looked over the provider and controller ones

charts/karpenter-crd/templates/karpenter.sh_nodeclaims.yaml

go.mod

pkg/apis/v1/ec2nodeclass_hash_test.go

pkg/apis/v1/ec2nodeclass_status.go

pkg/controllers/nodeclass/capacityreservation.go

pkg/providers/capacityreservation/provider.go

pkg/providers/capacityreservation/types.go

pkg/providers/launchtemplate/launchtemplate.go

jonathan-innis

Nice work! Most of the comments were small -- overall, looks reasonable to me

pkg/apis/v1/ec2nodeclass_validation_cel_test.go

pkg/cache/unavailableofferings.go

pkg/providers/amifamily/resolver.go

pkg/providers/instancetype/offering/provider.go

pkg/providers/instancetype/types.go

coveralls · 2025-02-21T11:49:32Z

Pull Request Test Coverage Report for Build 13543211365

Details

938 of 1088 (86.21%) changed or added relevant lines in 27 files are covered.
5 unchanged lines in 4 files lost coverage.
Overall coverage increased (+1.5%) to 66.771%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/nodeclass/controller.go	18	19	94.74%
pkg/apis/v1/ec2nodeclass_status.go	4	6	66.67%
pkg/providers/instancetype/offering/provider.go	142	144	98.61%
pkg/providers/instancetype/types.go	47	49	95.92%
pkg/controllers/controllers.go	0	3	0.0%
pkg/errors/errors.go	0	3	0.0%
pkg/providers/amifamily/resolver.go	47	50	94.0%
pkg/fake/ec2api.go	43	47	91.49%
pkg/providers/capacityreservation/provider.go	52	57	91.23%
pkg/providers/instance/instance.go	117	122	95.9%

Files with Coverage Reduction	New Missed Lines	%
pkg/controllers/nodeclass/controller.go	1	69.16%
pkg/operator/operator.go	1	7.25%
pkg/providers/amifamily/resolver.go	1	87.5%
pkg/fake/utils.go	2	95.12%

Totals
Change from base Build 13509475622:	1.5%
Covered Lines:	6639
Relevant Lines:	9943

💛 - Coveralls

jmdeal

/karpenter snapshot

github-actions · 2025-02-21T12:49:51Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-cc6c3ad5942fc69e00c1e8d2890288086347145e.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-cc6c3ad5942fc69e00c1e8d2890288086347145e" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

pkg/providers/amifamily/resolver.go

pkg/apis/v1/ec2nodeclass_status.go

pkg/apis/v1/ec2nodeclass_validation_cel_test.go

pkg/apis/v1/ec2nodeclass_status.go

pkg/providers/instancetype/instancetype.go

pkg/providers/instancetype/offering/provider.go

pkg/controllers/nodeclass/capacityreservation.go

pkg/providers/instancetype/instancetype.go

pkg/controllers/nodeclaim/capacityreservation/controller.go

pkg/providers/instancetype/suite_test.go

pkg/controllers/nodeclass/controller.go

pkg/controllers/nodeclaim/capacityreservation/controller.go

jonathan-innis

Really awesome work! We're so close!

charts/karpenter-crd/templates/karpenter.k8s.aws_ec2nodeclasses.yaml

pkg/controllers/nodeclaim/capacityreservation/controller.go

pkg/controllers/nodeclass/controller.go

pkg/cloudprovider/drift.go

pkg/fake/ec2api.go

pkg/fake/utils.go

pkg/apis/v1/ec2nodeclass.go

pkg/fake/ec2api.go

pkg/test/utils.go

pkg/controllers/nodeclaim/capacityreservation/controller.go

pkg/providers/launchtemplate/suite_test.go

needed to cache based on subnet zones, since NodeClaims don't get zonal requirements injected from the nodeclass

jmdeal

/karpenter snapshot

github-actions · 2025-02-25T22:53:04Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-9d6c1b00848c0e18c0a009b93686d788452801ca.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-9d6c1b00848c0e18c0a009b93686d788452801ca" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

jonathan-innis

No comments -- though I know we are still waiting on a few more updates and some E2Es

test/pkg/environment/aws/expectations.go

test/suites/scheduling/suite_test.go

jmdeal

/karpenter snapshot

github-actions · 2025-02-26T10:20:09Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-18d6949912e7212a2c68f340d3f03f81ac73ad86.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-18d6949912e7212a2c68f340d3f03f81ac73ad86" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

jmdeal

/karpenter snapshot

github-actions · 2025-02-26T10:54:15Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-61aa5f5f4bffce7908f6bc6f54ff4bb26cd4a633.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-61aa5f5f4bffce7908f6bc6f54ff4bb26cd4a633" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

pkg/providers/capacityreservation/provider.go

pkg/providers/capacityreservation/types.go

test/pkg/environment/aws/expectations.go

test/suites/drift/suite_test.go

test/suites/scheduling/suite_test.go

jonathan-innis

LGTM 🚀

jmdeal · 2025-02-26T11:51:42Z

Unused import 😢

jonathan-innis

Spoke too soon 😄

LGTM 🚀

jmdeal requested a review from a team as a code owner February 11, 2025 13:26

jmdeal requested a review from bwagner5 February 11, 2025 13:26

jonathan-innis reviewed Feb 13, 2025

View reviewed changes

jmdeal force-pushed the feat/reserved-capacity branch 3 times, most recently from c0a96bd to cc6c3ad Compare February 21, 2025 11:44

jmdeal commented Feb 21, 2025

View reviewed changes

pkg/providers/amifamily/resolver.go Outdated Show resolved Hide resolved

jmdeal force-pushed the feat/reserved-capacity branch from 5095850 to 2e27014 Compare February 21, 2025 17:03

jonathan-innis reviewed Feb 21, 2025

View reviewed changes

jmdeal mentioned this pull request Feb 22, 2025

karpenter_cloudprovider_instance_type_offering_available is returning the wrong data #7758

Open

jmdeal force-pushed the feat/reserved-capacity branch from 573be55 to a244119 Compare February 23, 2025 23:56

jmdeal mentioned this pull request Feb 24, 2025

feat: RFC Implementation Supporting ODCR #6198

Closed

10 tasks

jmdeal force-pushed the feat/reserved-capacity branch from a244119 to 26eb9de Compare February 24, 2025 02:43

jonathan-innis reviewed Feb 24, 2025

View reviewed changes

jmdeal force-pushed the feat/reserved-capacity branch 4 times, most recently from 0d396ca to 010638f Compare February 25, 2025 03:17

jonathan-innis reviewed Feb 25, 2025

View reviewed changes

jmdeal added 5 commits February 25, 2025 14:10

feat: on-demand capacity reservation support

aeb14a1

checkpoint

7773397

checkpoint review feedback

112b9b3

checkpointing

bcdee30

offering cache key fix

a9e83b6

needed to cache based on subnet zones, since NodeClaims don't get zonal requirements injected from the nodeclass

jmdeal added 5 commits February 25, 2025 14:11

remaining feedback

a504833

test: capacity reservation status tests

9996026

instance tests

f566425

checkpoint feedback + lt tests

423b3bb

test: cloudprovider functional tests

e3d1cdd

jmdeal force-pushed the feat/reserved-capacity branch from 010638f to e3d1cdd Compare February 25, 2025 19:11

remaining functional + feedback

9d6c1b0

jmdeal force-pushed the feat/reserved-capacity branch from 30da57c to 9d6c1b0 Compare February 25, 2025 22:41

jmdeal commented Feb 25, 2025

View reviewed changes

jonathan-innis reviewed Feb 26, 2025

View reviewed changes

e2e tests

1f35782

jonathan-innis reviewed Feb 26, 2025

View reviewed changes

test/pkg/environment/aws/expectations.go Outdated Show resolved Hide resolved

test/suites/scheduling/suite_test.go Show resolved Hide resolved

jmdeal added 2 commits February 25, 2025 21:46

checkpoint

f6198e2

remaining e2es + feedback

18d6949

jmdeal commented Feb 26, 2025

View reviewed changes

not found error fix

61aa5f5

jmdeal force-pushed the feat/reserved-capacity branch from 4d4cb55 to 61aa5f5 Compare February 26, 2025 10:52

jmdeal commented Feb 26, 2025

View reviewed changes

jonathan-innis reviewed Feb 26, 2025

View reviewed changes

jonathan-innis previously approved these changes Feb 26, 2025

View reviewed changes

checkpoint feedback

199602b

jmdeal dismissed jonathan-innis’s stale review via 199602b February 26, 2025 11:50

jmdeal force-pushed the feat/reserved-capacity branch from 1e6bc49 to 199602b Compare February 26, 2025 11:50

jonathan-innis approved these changes Feb 26, 2025

View reviewed changes

jmdeal merged commit eff767c into aws:main Feb 26, 2025
19 checks passed

jmdeal mentioned this pull request Mar 6, 2025

Karpenter support for prioritizing AWS Capacity Reservations #3042

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: on-demand capacity reservation support #7726

feat: on-demand capacity reservation support #7726

jmdeal commented Feb 11, 2025

netlify bot commented Feb 11, 2025 •

edited

Loading

jonathan-innis left a comment •

edited

Loading

jonathan-innis left a comment

coveralls commented Feb 21, 2025 •

edited

Loading

jmdeal left a comment

github-actions bot commented Feb 21, 2025

jonathan-innis left a comment

jmdeal left a comment

github-actions bot commented Feb 25, 2025

jonathan-innis left a comment

jmdeal left a comment

github-actions bot commented Feb 26, 2025

jmdeal left a comment

github-actions bot commented Feb 26, 2025

jonathan-innis left a comment •

edited

Loading

jmdeal commented Feb 26, 2025

jonathan-innis left a comment

feat: on-demand capacity reservation support #7726

feat: on-demand capacity reservation support #7726

Conversation

jmdeal commented Feb 11, 2025

netlify bot commented Feb 11, 2025 • edited Loading

✅ Deploy Preview for karpenter-docs-prod canceled.

jonathan-innis left a comment • edited Loading

Choose a reason for hiding this comment

jonathan-innis left a comment

Choose a reason for hiding this comment

coveralls commented Feb 21, 2025 • edited Loading

Pull Request Test Coverage Report for Build 13543211365

Details

💛 - Coveralls

jmdeal left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 21, 2025

jonathan-innis left a comment

Choose a reason for hiding this comment

jmdeal left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 25, 2025

jonathan-innis left a comment

Choose a reason for hiding this comment

jmdeal left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 26, 2025

jmdeal left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 26, 2025

jonathan-innis left a comment • edited Loading

Choose a reason for hiding this comment

jmdeal commented Feb 26, 2025

jonathan-innis left a comment

Choose a reason for hiding this comment

netlify bot commented Feb 11, 2025 •

edited

Loading

jonathan-innis left a comment •

edited

Loading

coveralls commented Feb 21, 2025 •

edited

Loading

jonathan-innis left a comment •

edited

Loading