Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how should we enable staging #130

Closed
jku opened this issue Mar 14, 2024 · 1 comment · Fixed by #131
Closed

how should we enable staging #130

jku opened this issue Mar 14, 2024 · 1 comment · Fixed by #131
Labels
question Further information is requested

Comments

@jku
Copy link
Member

jku commented Mar 14, 2024

So I'm looking at doing some final testing before enabling staging test by default (changing the default value of the action input) however...

I started thinking about how this goes in the future (assume we have many more tests that support testing on staging):

  • do we need to enable xfail for staging but not for prod? Probably yes because both get run on the same action invocation
  • we could invent a syntax of our own (xfail: "test_name@staging" or something) and handle that in conformance_xfail() fixture -- or add another input to the action like xfail_staging

Alternatively we might want to require running the action separately for staging: this makes it easy to manage the xfail lists separately.

@jku jku added the question Further information is requested label Mar 14, 2024
@jku
Copy link
Member Author

jku commented Mar 14, 2024

I think I kind of like the idea of separate action invocations: this also looks more meaningful on GitHub: "failure in Staging conformance" requires very different action from "failure in conformance"

I'll make a PR of this and will do some testing...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant