Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

execution log doesn't contain remote-cacheable key #13495

Closed
alexeagle opened this issue May 20, 2021 · 0 comments
Closed

execution log doesn't contain remote-cacheable key #13495

alexeagle opened this issue May 20, 2021 · 0 comments
Assignees
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: feature request

Comments

@alexeagle
Copy link
Contributor

src/main/protobuf/spawn.proto contains a cacheable key, which is used to log the spawn in

builder.setCacheable(Spawns.mayBeCached(spawn));

This allows you to filter the execution log for actions that are marked with no-cache tags (or are not cacheable due to some other condition)

However the no-remote-cache tag isn't reflected in the execution log at all.

So for example building

genrule(
    name = "large-output",
    srcs = [],
    outs = ["out"],
    cmd = "head -50104 /dev/random > $@",
    tags = ["no-remote-cache"],
)

results in execution log with only these boolean fields

  "remotable": true,
  "cacheable": true,
  "progressMessage": "Executing genrule //:large-output",
  "actualOutputs": [
    {
      "path": "bazel-out/k8-fastbuild/bin/out",
      "digest": {
        "hash": "c3df9f58fa71c57b788f3fbd224d0e3d5ebfc8cd5acc20ab752e8de2ae07885c",
        "sizeBytes": "12824891",
        "hashFunctionName": "SHA-256"
      }
    }
  ],
  "remoteCacheHit": false,

As a result, there is currently no way to find actions which produce large outputs unsuitable for remote caching, and which are not correctly tagged to avoid uploading those large outputs (more context in #11182 )

@jin jin added team-Remote-Exec Issues and PRs for the Execution (Remote) team untriaged labels May 20, 2021
coeuvre added a commit to coeuvre/bazel that referenced this issue Jun 3, 2021
@coeuvre coeuvre self-assigned this Jun 3, 2021
@coeuvre coeuvre added P2 We'll consider working on this in future. (Assignee optional) type: feature request and removed untriaged labels Jun 3, 2021
@coeuvre coeuvre added P1 I'll work on this now. (Assignee required) and removed P2 We'll consider working on this in future. (Assignee optional) labels Jun 3, 2021
katre pushed a commit that referenced this issue Jul 12, 2021
Fixes #13495.

Closes #13551.

PiperOrigin-RevId: 378090621
coeuvre added a commit to coeuvre/bazel that referenced this issue Jul 15, 2021
coeuvre added a commit to coeuvre/bazel that referenced this issue Jul 15, 2021
coeuvre added a commit to coeuvre/bazel that referenced this issue Jul 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team type: feature request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants