Bazel reports build success when it was not successful

[meteorcloudy]

This bug was filed at bazelbuild/continuous-integration#449, but it should be a Bazel bug

/cc @c-parsons @laurentlb

[laurentlb]

In the interpreter, we throw an exception for this. It is properly reported one level up: package contains errors: e2e/output_runner. But later, something swallows the error. I don't know what happens, I suspect it's related to --keep_going, so assigning to core team to investigate.

[laurentlb]

Repro:

touch WORKSPACE
mkdir sub
touch test.sh && chmod +x test.sh
echo 'sh_test(name = "test", srcs = ["test.sh"])' > BUILD
echo 'parse error' > sub/BUILD
bazel test //... -k

If one package cannot be loaded, but all other tests pass, Bazel returns 0.

[laurentlb]

In the repro above: if sub/BUILD declares at least one target, the exit code is properly set to 1.

cc @ericfelly who should own this?

[ericfelly]

Nathan, can you take a look?

[haxorz]

is this really P1?

[jin]

@haxorz if it's not a P1 according to the definitions, please reassign the correct priority. thanks!

[meteorcloudy]

If we don't fix this issue, Bazel users (including us) could miss potential failures in the build, I think it's a pretty serious issue (P1).

[laurentlb]

I had a quick look yesterday. This issue also affects Google (Blaze) and has been around for at least a year. I'm surprised it didn't come up before.

[haxorz]

ok, i will look into this today

[meteorcloudy]

Thank you!