Skip to content
This repository has been archived by the owner on Jan 8, 2024. It is now read-only.

Tests may fail when run via bazel test #105

Closed
brandjon opened this issue May 13, 2019 · 1 comment
Closed

Tests may fail when run via bazel test #105

brandjon opened this issue May 13, 2019 · 1 comment

Comments

@brandjon
Copy link
Contributor

brandjon commented May 13, 2019

After the fix to tests for #98, tests may fail when run directly via bazel test as opposed to through the runner script run_tests.sh. I see two modes of failure:

  1. When --incompatible_use_python_toolchains is disabled, //tests:version_test may fail if /usr/bin/env python is a Python 2 interpreter, due to py_binary doesn't automatically pick a Python runtime matching the requested version bazelbuild/bazel#4815.

  2. When --incompatible_use_python_toolchains is enabled, the default toolchain's py[2|3]wrapper.sh script is added to runfiles, causing brittle assertions in many tests to fail. This doesn't appear under run_tests.sh because it has its own custom toolchain that adds no runfiles.

This isn't an immediate problem because CI still passes, but it is annoying for the developer workflow. For 1), I'd just wait for the toolchain flag flip. For 2), the test assertions should be made more robust. In the meantime, use run_tests.sh instead of bazel test //....

@brandjon
Copy link
Contributor Author

Turns out it is a problem for Bazel downstream CI, just not subpar's own CI.

brandjon added a commit that referenced this issue May 13, 2019
Fixes #105 by 1) enabling toolchains in the workspace .bazelrc, and 2) making the test harness filter out runfiles added by the autodetecting toolchain.

Previously, the tests worked when run from run_tests.sh but failed in `bazel test //...`. This caused travis to pass but buildkite to fail.
brandjon added a commit that referenced this issue May 13, 2019
Fixes #105 by 1) enabling toolchains in the workspace .bazelrc, and 2) making the test harness filter out runfiles added by the autodetecting toolchain.

Previously, the tests worked when run from run_tests.sh but failed in `bazel test //...`. This caused travis to pass but buildkite to fail.
brandjon added a commit that referenced this issue May 13, 2019
Fixes #105 by 1) enabling toolchains in the workspace .bazelrc, and 2) making the test harness filter out runfiles added by the autodetecting toolchain.

Previously, the tests worked when run from run_tests.sh but failed in `bazel test //...`. This caused travis to pass but buildkite to fail.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant