Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EnvoyFleet customization improvements #143

Merged
merged 8 commits into from
Dec 13, 2021

Conversation

Tarick
Copy link
Contributor

@Tarick Tarick commented Dec 10, 2021

This PR closes #133

Changes

  • Adds the ability to customize EnvoyFleet deployment, including:
    -- Scheduler hints (NodeSelector, Tollerations, Affinity/Antiaffinity)
    -- CPU and Memory Resources (limits and requests)
    -- setting custom public Envoy Image
    -- Adding custom annotations (for e.g. Prometheus scrapping)
    -- Service customization - the ability to set ports, type and annotations to fully support ingress Service in any environment
  • adds setting Status.State field for EnvoyFleet, allowing to hint the user if EnvoyFleet deployment succeeded (visible via kubectl describe envoyfleet default. Still requires more improvements since technically Service, Deployment and Config map statuses are not checked.

As part of the implementation, local development environment was reworked to support the same labels and resources names as in the Helm chart and subsequently

Updates

  • Go-control-plane is updated to 0.10.1

Follow up items:

These tasks needs to be done just before we tag the new version:

  • the documentation for the EnvoyFleet CRD isn't changed since the site is deployed from main branch and it will be available immediately while must be visible only when we make the new tag and Helm chart.
  • Helm charts needs to be updated as well before we tag new version.

Follow up issue will be created.

Fixes

  • Moved EnvoyFleet handling from k8sutils package since it had leaked EnvoyFleet implemenation.
  • Reworked hardcoding of XDS service URL to the discovery mechanism based on labels. Needed for the situations when kusk-gateway is deployed to the different namespace.
  • Removed mangling of the Kustomization files during the debugging.

Checklist

  • tested locally
  • added new dependencies
  • updated the docs
  • added a test

@Tarick Tarick added this to the 0.0.0-alpha.2 milestone Dec 10, 2021
@Tarick Tarick linked an issue Dec 10, 2021 that may be closed by this pull request
@kylehodgetts
Copy link
Contributor

adds setting Status.State field for EnvoyFleet, allowing to hint the user if EnvoyFleet deployment succeeded (visible via kubectl describe envoyfleet default. Still requires more improvements since technically Service, Deployment and Config map statuses are not checked.

Perhaps we should open another issue for this @Tarick ?

@Tarick Tarick mentioned this pull request Dec 10, 2021
@kylehodgetts
Copy link
Contributor

kylehodgetts commented Dec 10, 2021

local development environment was reworked to support the same labels and resources names as in the Helm chart

Will you update the docs?
Yes, in #144

@Tarick Tarick linked an issue Dec 10, 2021 that may be closed by this pull request
sharedLabels map[string]string
}

func NewEnvoyFleetResources(ctx context.Context, client client.Client, ef *gatewayv1alpha1.EnvoyFleet) (*EnvoyFleetResources, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im not sure about this one

Its doing a lot. I feel it would be better to have separate functions for creating each of the required resources. For transparency more than anything

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that some of those resources depend on others. Configmap is separate, but Deployment uses this configmap, Service uses deployment labels selector - so we need to create specific variables and share between these. Thus the struct, thus the specific order too in NewEnvoyFleetResources.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To specify dependencies, you specify parameters to the functions that depend on other things

configMap := createConfig()
deployment := createDeployment(configMap)
svc := createService(deployment)

For example ☝️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or use Class :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dobegor what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me this is not a deal breaker for approving your PR. Just a personal opinion of mine. Im curious to find out Egor's opinion

Copy link
Contributor

@dobegor dobegor Dec 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish these functions would return objects that they generate and the final thing would be composed of them here rather than having everything done in a procedural style... However we can still pull this in and refactor it afterwards.

controllers/envoyfleet_resources.go Outdated Show resolved Hide resolved
controllers/envoyfleet_resources.go Show resolved Hide resolved
docs/development.md Show resolved Hide resolved
@kylehodgetts kylehodgetts self-requested a review December 10, 2021 13:52
kylehodgetts
kylehodgetts previously approved these changes Dec 10, 2021
@kylehodgetts kylehodgetts self-requested a review December 10, 2021 16:23
kylehodgetts
kylehodgetts previously approved these changes Dec 10, 2021
Copy link
Contributor

@dobegor dobegor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Kyle's comments and I'd like to refactor the resource generation bits but I think it's already big enough to merge it 😃

@Tarick Tarick merged commit 93feb2b into main Dec 13, 2021
@Tarick Tarick deleted the tyatsurak/enhancement/133-envoyfleet-config branch December 13, 2021 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EnvoyFleet: add more parameters for the configuration
3 participants