Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update hack scripts to shim temp GOPATH as needed #4465

Merged
merged 1 commit into from Jan 19, 2022
Merged

Update hack scripts to shim temp GOPATH as needed #4465

merged 1 commit into from Jan 19, 2022

Conversation

ghost
Copy link

@ghost ghost commented Jan 11, 2022

Changes

When we run our codegen and openapigen scripts from a directory that is
not under GOPATH the generated output uses relative prefixes instead of
absolute prefixes.

The new hack/setup-temporary-gopath.sh script detects if the repo is not in
GOPATH and creates a temporary one under ".gopath" if so. codegen and
openapigen scripts now depend on setup-temporary-gopath.sh before
generating anything.

Fixes #3884
Fixes #4430

... for some definition of "fixes" ...

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Release Notes

NONE

@tekton-robot tekton-robot added the release-note-none Denotes a PR that doesnt merit a release note. label Jan 11, 2022
@tekton-robot tekton-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 11, 2022
@ghost
Copy link
Author

ghost commented Jan 11, 2022

/kind bug

@tekton-robot tekton-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 11, 2022
@ghost
Copy link
Author

ghost commented Jan 11, 2022

This uses the workaround that @vdemeester documented here: #3884 (comment)

It is not pretty but it works around the problem.

@ghost
Copy link
Author

ghost commented Jan 11, 2022

/test pull-tekton-pipeline-integration-tests

@@ -82,3 +85,5 @@ ${REPO_ROOT_DIR}/hack/update-deps.sh

# Make sure the OpenAPI specification and Swagger file are up-to-date
${REPO_ROOT_DIR}/hack/update-openapigen.sh

shim_gopath_clean
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe trap shim_gopath_clean EXIT and move this above, so this gets done even if the script is interrupted? Otherwise lgtm, and thanks for doing this!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works well, cheers!

@ghost
Copy link
Author

ghost commented Jan 11, 2022

The failure here was our old friend TestSidecarTaskSupport/A_sidecar_that_terminates_early_does_not_cause_problems_running_Steps

/test pull-tekton-pipeline-alpha-integration-tests

@imjasonh
Copy link
Member

/approve

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 11, 2022
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question around the path used. For reference, in docker, in the early days (before vendor was supported by go, etc..) we would use a .gopath with links in the root of the source (and have it in the .gitignore of course) for it to work no matter which system it would be on.
Seems like it's still there.

# and openapigen to execute in. This is only done if
# the current repo directory is not within GOPATH.
function shim_gopath() {
local TEMP_GOPATH="/tmp/tekton-pipelines-codegen-gopath"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to use than instead of mktemp ?
One reason to use mktemp would be that not all systems use /tmp as temporary directory, and usually mktemp handle this 🙃.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially avoided that because mktemp on Mac has non-standard behaviour, meaning more shell code to work around it. I don't feel strongly about it but in the moment writing it felt like just using /tmp was going to be clearer if / when it breaks on a system without a /tmp directory.

@ghost
Copy link
Author

ghost commented Jan 12, 2022

Something really strange is happening when the dependencies install to the temporary GOPATH: many packages end up created with -r--r--r-- perms, which means that deleting the temp GOPATH requires sudo. I don't totally understand where those perms are coming from so, for now, I will just have it print a comment to the user that they can safely delete .gopath after the script is done.

@ghost
Copy link
Author

ghost commented Jan 12, 2022

OK the .gopath approach works nicely, going with that thanks @vdemeester !

When we run our codegen and openapigen scripts from a directory that is
not under GOPATH the generated output uses relative prefixes instead of
absolute prefixes.

The new hack/setup-temporary-gopath.sh script detects if the repo is not in
GOPATH and creates a temporary one under .gopath if so. codegen and
openapigen scripts now depend on setup-temporary-gopath.sh before
generating anything.
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/meow

@tekton-robot
Copy link
Collaborator

@vdemeester: cat image

In response to this:

/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: imjasonh, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [imjasonh,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Member

@jerop jerop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @sbwsg 🎉

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2022
@jerop
Copy link
Member

jerop commented Jan 18, 2022

=== RUN   TestRealRunnerTimeout
    runner_test.go:37: step didn't timeout
--- FAIL: TestRealRunnerTimeout (0.02s)

previously seen as a flake in #3878 (comment)

/test pull-tekton-pipeline-unit-tests

@jerop
Copy link
Member

jerop commented Jan 19, 2022

/test pull-tekton-pipeline-unit-tests

@jerop
Copy link
Member

jerop commented Jan 19, 2022

/test tekton-pipeline-unit-tests

@tekton-robot tekton-robot merged commit cb5de41 into tektoncd:main Jan 19, 2022
@lbernick
Copy link
Member

I think this does not work if GOPATH isn't set:
/Users/leebernick/tekton/pipeline/hack/setup-temporary-gopath.sh: line 20: GOPATH: unbound variable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesnt merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update-openapigen.sh must be run in GOPATH update-codegen.sh doesn't work outside of GOPATH
5 participants