Change the way we run e2e tests with feature gates in pipelines #3930
Conversation
Prior to this commit pipelines ran e2e tests for both "stable" and "alpha" feature gates in a single prow job. This makes the tests take twice as long as running running either individually. In a parallel commit to Plumbing we are introducing a new prow job that is specifically for running alpha e2e tests. This means both the alpha and stable tests can be run concurrently and hopefully should reduce the iteration time on PRs - failures should show up sooner. The alpha feature gate prow job will set the PIPELINE_FEATURE_GATE env var when it is initiated. Our e2e script recognizes this var and sets the feature-flags configmap accordingly. If the env var is not set then we default to running under "stable" feature gate.
|
Here's the PR in plumbing to add the PIPELINE_FEATURE_GATE env var: tektoncd/plumbing#833 This PR will end up disabling the "alpha" tests until the one above is merged. |
|
Let's see if this works so soon after the plumbing merge: /test pull-tekton-pipeline-alpha-integration-tests Edit: ok wow, that actually did start
|
|
Re-running integration too to see if that now skips alpha: /test pull-tekton-pipeline-integration-tests Looking really good so far - both integration runs have started by patching the feature-flags configmap with the expected gate. |
|
It looks like a timeout test failed because of cluster slowness. Let's rerun and see what happens. /test pull-tekton-pipeline-alpha-integration-tests |
|
nice. everything's passed and the integration runs each set their own feature gates as expected. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: vdemeester The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| @@ -52,11 +52,14 @@ function run_e2e() { | |||
| go_test_e2e -tags=examples -timeout=20m ./test/ || failed=1 | |||
| } | |||
|
|
|||
| set_feature_gate "stable" | |||
| run_e2e | |||
| if [ "$PIPELINE_FEATURE_GATE" == "" ]; then | |||
There was a problem hiding this comment.
NIT: We could also do:
set_feature_gate "${PIPELINE_FEATURE_GATE:-stable}"
run_e2eBut this works too.
|
@sbwsg: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/retest |
Changes
Prior to this commit pipelines ran e2e tests for both "stable" and
"alpha" feature gates in a single prow job. This makes the tests
take twice as long as running running either individually.
In a parallel commit to Plumbing we are introducing a new prow job
that is specifically for running alpha e2e tests. This means both
the alpha and stable tests can be run concurrently and hopefully
should reduce the iteration time on PRs - failures should show up
sooner.
The alpha feature gate prow job will set the PIPELINE_FEATURE_GATE
env var when it is initiated. Our e2e script recognizes this var
and sets the feature-flags configmap accordingly. If the env var
is not set then we default to running under "stable" feature gate.
Submitter Checklist
As the author of this PR, please check off the items in this checklist:
functionality, content, code)
Release Notes