Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eks-agent: allow endpoint override when querying pre-existing cluster info #860

Merged
merged 2 commits into from
Sep 8, 2023

Conversation

etungsten
Copy link
Contributor

@etungsten etungsten commented Sep 8, 2023

Issue number:
Partially addresses #565. Only for EKS service endpoint.

Description of changes:

    agents: add endpoint override support in eks provider
    
    We add a configuration in EKS cluster resource agent to allow EKS
    service endpoint override when querying cluster information for a
    pre-existing cluster.
    eks-resource-agent: use AWS API calls to gather created cluster info
    
    Instead of calling eksctl for certain cluster info, use AWS API calls
    and AWS-CLI calls to gather cluster metadata.

Testing done:

Normal pre-existing cluster created using eksctl:
Deployed a manifest for the cluster, EC2 instances, quick sonobuoy test.

---
apiVersion: testsys.system/v1
kind: Resource
metadata:
  name: ipv6-bottlerocket
  namespace: testsys
spec:
  agent:
    timeout: 70d
    name: eks-provider
    image: eks-resource-agent:latest
    keepRunning: true
    configuration:
      creationPolicy: never
      clusterName: ipv6-bottlerocket
      region: us-west-2
  dependsOn: []
  destructionPolicy: never
---
---
apiVersion: testsys.system/v1
kind: Resource
metadata:
  name: ipv6-bottlerocket-instances
  namespace: testsys
spec:
  agent:
    timeout: 70d
    name: ec2-provider
    image: ec2-resource-agent:latest
    keepRunning: true
    configuration:
      clusterName: ${ipv6-bottlerocket.clusterName}
      clusterType: eks
      instanceCount: 2
      instanceProfileArn: ${ipv6-bottlerocket.iamInstanceProfileArn}
      nodeAmi: ami-
      region: us-west-2
      subnetIds: ${ipv6-bottlerocket.publicSubnetIds}
      instanceTypes: ["m5.large"]
      endpoint: ${ipv6-bottlerocket.endpoint}
      certificate: ${ipv6-bottlerocket.certificate}
      clusterDnsIp: ${ipv6-bottlerocket.clusterDnsIp}
      securityGroups: ${ipv6-bottlerocket.securityGroups}
  dependsOn: [ipv6-bottlerocket]
  destructionPolicy: onDeletion
---
apiVersion: testsys.system/v1
kind: Test
metadata:
  name: ipv6-bottlerocket-test
  namespace: testsys
spec:
  agent:
    timeout: 70d
    name: sonobuoy-test-agent
    image: sonobuoy-test-agent:latest
    keepRunning: true
    configuration:
      region: ${ipv6-bottlerocket.region}
      kubeconfigBase64: ${ipv6-bottlerocket.encodedKubeconfig}
      plugin: "e2e"
      mode: quick
  dependsOn: []
  resources: [ipv6-bottlerocket-instances, ipv6-bottlerocket]

Resource gets created, test runs to completion:

$ cli --kubeconfig etung-testsys.kubeconfig status
 NAME                                            TYPE                         STATE                                           PASSED                     FAILED                    SKIPPED 
 ipv6-bottlerocket-test                          Test                         passed                                               5                          0                       7206 
 ipv6-bottlerocket                               Resource                     completed                                                                                                    
 ipv6-bottlerocket-instances                     Resource                     completed     

For a pre-existing cluster in a different EKS service endpoint:

Deployed the following manifest:

---
apiVersion: testsys.system/v1
kind: Resource
metadata:
  name: beta-x86-64-aws-k8s-128
  namespace: testsys
spec:
  agent:
    timeout: 7d
    name: eks-provider
    image: eks-resource-agent:what
    keepRunning: true
    configuration:
      eksServiceEndpoint: https://api.beta.us-west-2.wesley.amazonaws.com
      creationPolicy: never
      clusterName: beta-x86-64-aws-k8s-128
      region: us-west-2
  dependsOn: []
  destructionPolicy: never
---
apiVersion: testsys.system/v1
kind: Resource
metadata:
  name: beta-x86-64-aws-k8s-128-instances
  namespace: testsys
spec:
  agent:
    timeout: 7d
    name: ec2-provider
    image: ec2-resource-agent:latest
    keepRunning: true
    configuration:
      clusterName: ${beta-x86-64-aws-k8s-128.clusterName}
      clusterType: eks
      instanceCount: 2
      instanceProfileArn: ${beta-x86-64-aws-k8s-128.iamInstanceProfileArn}
      nodeAmi: ami-
      region: us-west-2
      subnetIds: ${beta-x86-64-aws-k8s-128.publicSubnetIds}
      instanceTypes: ["m5.large"]
      endpoint: ${beta-x86-64-aws-k8s-128.endpoint}
      certificate: ${beta-x86-64-aws-k8s-128.certificate}
      clusterDnsIp: ${beta-x86-64-aws-k8s-128.clusterDnsIp}
      securityGroups: ${beta-x86-64-aws-k8s-128.securityGroups}
  dependsOn: [beta-x86-64-aws-k8s-128]
  destructionPolicy: onDeletion
---
apiVersion: testsys.system/v1
kind: Test
metadata:
  name: beta-x86-64-aws-k8s-128-test
  namespace: testsys
spec:
  agent:
    timeout: 7d
    name: sonobuoy-test-agent
    image: sonobuoy-test-agent:latest
    keepRunning: true
    configuration:
      region: ${beta-x86-64-aws-k8s-128.region}
      kubeconfigBase64: ${beta-x86-64-aws-k8s-128.encodedKubeconfig}
      plugin: "e2e"
      mode: quick
  dependsOn: []
  resources: [beta-x86-64-aws-k8s-128-instances, beta-x86-64-aws-k8s-128]

Resource gets created, tests run to completion:

$ cli --kubeconfig etung-testsys.kubeconfig status
 NAME                                                 TYPE                        STATE                                         PASSED                    FAILED                   SKIPPED 
 beta-x86-64-aws-k8s-128-test                         Test                        passed                                             5                         0                      7386 
 beta-x86-64-aws-k8s-128                              Resource                    completed                                                                                                
 beta-x86-64-aws-k8s-128-instances                    Resource                    completed  

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@etungsten etungsten changed the title Endpointz eks-agent: allow endpoint override when querying pre-existing cluster info Sep 8, 2023
Instead of calling eksctl for certain cluster info, use AWS API calls
and AWS-CLI calls to gather cluster metadata.
@etungsten etungsten force-pushed the endpointz branch 2 times, most recently from 7f11d54 to da8e275 Compare September 8, 2023 19:31
We add a configuration in EKS cluster resource agent to allow EKS
service endpoint override when querying cluster information for a
pre-existing cluster.
@etungsten etungsten marked this pull request as ready for review September 8, 2023 21:31
@etungsten etungsten merged commit cbd073e into bottlerocket-os:develop Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants