Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on duplicate fields in Kustomization #4929

Conversation

koba1t
Copy link
Member

@koba1t koba1t commented Dec 13, 2022

  • using UnmarshalStrict when unmarshaling kustomization.yaml
  • fix hidden errors appear by change to strict

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 13, 2022
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 13, 2022
@koba1t koba1t force-pushed the refactor/cleanup_Unmarshal_kustomization branch from 532d1a3 to 5fed0f7 Compare December 13, 2022 21:05
}
assert.Contains(t, err.Error(),
"kustomization unmarshal error: error converting YAML to JSON: yaml: unmarshal errors:\n"+
" line 13: key \"literals\" already set in map\n line 18: key \"files\" already set in map")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

be an error like this TODO comment.

Comment on lines -142 to -144
th.WriteK("/whatever", `
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove apiVersion: kustomize.config.k8s.io/v1beta1 and kind: Kustomization,
because th.WriteK is always added above two lines.

@koba1t
Copy link
Member Author

koba1t commented Dec 14, 2022

I think it is related to #4861

/cc @natasha41575

@KnVerey KnVerey added this to the v5.0.0 milestone Dec 14, 2022
@KnVerey
Copy link
Contributor

KnVerey commented Dec 14, 2022

I think it is related to #4861

If this PR does fix that issue, can you please add a test showing it?

As you noticed in our own tests, this can be a breaking change, in that some users who were previously successfully building their Kustomizations will now encounter an error. Since we were already throwing an error on unknown fields, the novel case is duplicate fields, as in your test changes. Per the kubernetes-sigs/yaml docs, previously "Duplicate fields, including in-case-sensitive matches, are ignored in an undefined order." (emphasis mine). That last part in particular makes me lean towards accepting this change for inclusion in the 5.0 release--not only is there a risk of affected users not getting the result they want, but there's also a risk that what they get will be inconsistent (which is obviously not good in general and also against our reproducibility principle for kustomize build).

@natasha41575 wdyt?

/triage under-consideration
/retitle Error on duplicate fields in Kustomization

@k8s-ci-robot k8s-ci-robot changed the title refactoring for Kustomization unmarshal function Error on duplicate fields in Kustomization Dec 14, 2022
@koba1t
Copy link
Member Author

koba1t commented Dec 15, 2022

@KnVerey

I think it is related to #4861

If this PR does fix that issue, can you please add a test showing it?

Sorry, This PR can't fix that issue.
I think a strict unmarshal step helps to resolve that issue and prevents happen to similar problems on the unmarshal step.

@natasha41575
Copy link
Contributor

That last part in particular makes me lean towards accepting this change for inclusion in the 5.0 release--not only is there a risk of affected users not getting the result they want, but there's also a risk that what they get will be inconsistent (which is obviously not good in general and also against our reproducibility principle for kustomize build).

SGTM.

/triage accepted

@k8s-ci-robot k8s-ci-robot added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Dec 15, 2022
api/internal/localizer/localizer_test.go Outdated Show resolved Hide resolved
if err != nil {
return err
if err := yaml.UnmarshalStrict(y, &k); err != nil {
return fmt.Errorf("kustomization unmarshal error: %w", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the message from err itself already mentions unmarshalling, I suggest changing this to

Suggested change
return fmt.Errorf("kustomization unmarshal error: %w", err)
return errors.WrapPrefixf(err, "invalid Kustomization")

(and using sigs.k8s.io/kustomize/kyaml/errors for wrapping)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix

@k8s-ci-robot
Copy link
Contributor

@koba1t: This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@KnVerey
Copy link
Contributor

KnVerey commented Jan 6, 2023

Thanks for the quick follow-up!
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 6, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: KnVerey, koba1t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 6, 2023
@koba1t
Copy link
Member Author

koba1t commented Jan 6, 2023

@KnVerey
Thanks for your reviews of error messages!

I fix a few points from your comments.
Could you recheck it?

@k8s-ci-robot k8s-ci-robot merged commit c6ca3ff into kubernetes-sigs:master Jan 6, 2023
@koba1t koba1t deleted the refactor/cleanup_Unmarshal_kustomization branch January 6, 2023 18:40
@KnVerey KnVerey added the release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. label Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants