Skip to content

Commit

Permalink
internal/ci: restore git modified times after cloning
Browse files Browse the repository at this point in the history
On one hand, "git clone" creates every file anew,
with their modified times representing the current time.
On the other, Go's test caching tracks the attributes and contents of
the files opened by each test as part of its cache key hashes.
Unfortunately, the modified times are always part of that hash,
even when the files are small. See https://go.dev/issues/58571.

Due to those factors, the trybots always re-run nearly all tests.
In particular, we use testdata files in quite a few packages,
so none of those can realistically hit the test cache in CI.

Until Go's test caching can be made a bit smarter about modified times,
use a python script packaged as a GitHub Action to restore each file
tracked by git to the commit time when they were last modified.
Luckily, the action only runs a simple python script,
so it seems to consistently finish in under a second.

We also have to fetch the git repository with full history,
since actions/checkout defaults to a shallow clone and the python script
needs full git history to properly reconstruct modified times.
Luckily, GitHub seem to have optimized git cloning from Actions,
so the actions/checkout step still seems to finish in around one second.

Signed-off-by: Daniel Martí <mvdan@mvdan.cc>
Change-Id: I7f1decd2ad1484f1a30e092b530b93833f6099c2
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/551227
Reviewed-by: Paul Jolly <paul@myitcv.io>
TryBot-Result: CUEcueckoo <cueckoo@cuelang.org>
  • Loading branch information
mvdan committed Mar 18, 2023
1 parent b05585b commit 829bd64
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 1 deletion.
3 changes: 3 additions & 0 deletions .github/workflows/trybot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ jobs:
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Restore git file modification times
uses: chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe
- name: Install Go
uses: actions/setup-go@v3
with:
Expand Down
10 changes: 10 additions & 0 deletions internal/ci/base/base.cue
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,16 @@ import (
uses: "actions/checkout@v3"
}

// #restoreGitModTimes works around test cache misses due to https://go.dev/issues/58571.
// Note that this action requires actions/checkout to use a fetch-depth of 0.
// Since this is a third-party action which runs arbitrary code,
// we pin a commit hash for v2 to be in control of code updates.
// TODO(mvdan): May be unnecessary once the Go bug above is fixed.
#restoreGitModTimes: json.#step & {
name: "Restore git file modification times"
uses: "chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe"
}

#earlyChecks: json.#step & {
name: "Early git and code sanity checks"
run: """
Expand Down
6 changes: 5 additions & 1 deletion internal/ci/github/trybot.cue
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,12 @@ trybot: _base.#bashWorkflow & {
// testing the PR's HEAD merged on top of the master branch.
// For consistency with Gerrit, avoid that merge commit entirely.
// This doesn't affect "push" builds, which never used merge commits.
with: ref: "${{ github.event.pull_request.head.sha }}"
with: {
ref: "${{ github.event.pull_request.head.sha }}"
"fetch-depth": 0 // see #restoreGitModTimes's doc
}
},
_base.#restoreGitModTimes,
_base.#installGo,

// cachePre must come after installing Node and Go, because the cache locations
Expand Down

0 comments on commit 829bd64

Please sign in to comment.