You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So I'm looking at doing some final testing before enabling staging test by default (changing the default value of the action input) however...
I started thinking about how this goes in the future (assume we have many more tests that support testing on staging):
do we need to enable xfail for staging but not for prod? Probably yes because both get run on the same action invocation
we could invent a syntax of our own (xfail: "test_name@staging" or something) and handle that in conformance_xfail() fixture -- or add another input to the action like xfail_staging
Alternatively we might want to require running the action separately for staging: this makes it easy to manage the xfail lists separately.
The text was updated successfully, but these errors were encountered:
I think I kind of like the idea of separate action invocations: this also looks more meaningful on GitHub: "failure in Staging conformance" requires very different action from "failure in conformance"
I'll make a PR of this and will do some testing...
So I'm looking at doing some final testing before enabling staging test by default (changing the default value of the action input) however...
I started thinking about how this goes in the future (assume we have many more tests that support testing on staging):
xfail: "test_name@staging"
or something) and handle that inconformance_xfail()
fixture -- or add another input to the action likexfail_staging
Alternatively we might want to require running the action separately for staging: this makes it easy to manage the xfail lists separately.
The text was updated successfully, but these errors were encountered: