-
Notifications
You must be signed in to change notification settings - Fork 241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve building process with cache goes dependencies #689
Conversation
.github/workflows/CI.yml
Outdated
#Samples codes for different OS: https://github.com/actions/cache/blob/main/examples.md#go---modules | ||
#Since we are using Linux, the go packages are in /go/pkg/mod | ||
|
||
- name: Cache go dependencies in go.mod |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These aren't only dependencies but build cache too so don't need to mention the word dependencies
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will cache tests. Do we want to do that? I would say yes since tests related to changed files will rerun if files are changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will only be caching unit tests I think. Not the integration tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might want to create conditionals to cache the build only on pr not on merge
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will only be caching unit tests I think. Not the integration tests.
Yep, the integration tests aren't run using go test
, so the caching wouldn't affect them. Will this cache tests though? I thought based on the documentation, the options we use with make test
will prevent caching. https://pkg.go.dev/cmd/go/internal/test
The rule for a match in the cache is that the run involves the same
test binary and the flags on the command line come entirely from a
restricted set of 'cacheable' test flags, defined as -benchtime, -cpu,
-list, -parallel, -run, -short, and -v. If a run of go test has any test
or non-test flags outside this set, the result is not cached. To
disable test caching, use any test flag or argument other than the
cacheable flags.
Line 28 in 9153ede
GOTEST_OPT?= -short -coverprofile coverage.txt -v -race -timeout 180s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes you are right
.github/workflows/CI.yml
Outdated
# instead of adding and update requirements like go get does | ||
# Contribute to: https://stackoverflow.com/questions/66356034/what-is-the-difference-between-go-get-command-and-go-mod-download-command | ||
#We shouldn't use go mod tidy since it has the same goals with go get by updating all dependencies and remove all unnecessary requirements | ||
- name: Install go dependencies if cache dependencies did not exist |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need this step, the build will automatically download needed dependencies and they'll be cached anyways. go mod download tends to download more than actually needed for the build.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting thought !!!. Will change this in the upcoming commit
.github/workflows/CI.yml
Outdated
restore-keys: | | ||
${{ runner.os }}-go- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't do anything, does it? Since nothing will ever store in the cache with this key there will never be anything to restore.
restore-keys: | | |
${{ runner.os }}-go- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes it does. The restore-keys are the partial key, not the full key itself. What the flow actually is:
- When there is an event that has the exact match, the actions restore the file in the cache file
- If there are no exact matches, the action searches for partial matches of the restore keys. When the action finds a partial match, the most recent cache is restored to the path directory.
Here is the document that states my respond:
https://docs.github.com/en/actions/advanced-guides/caching-dependencies-to-speed-up-workflows#example-using-the-cache-action
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After thinking about cache behaviors, IMO it should be removed since we don't want to cache the old modules.
…he on pull_request and
Codecov Report
@@ Coverage Diff @@
## main #689 +/- ##
=======================================
Coverage 54.54% 54.54%
=======================================
Files 11 11
Lines 297 297
=======================================
Hits 162 162
Misses 118 118
Partials 17 17 Continue to review full report at Codecov.
|
.github/workflows/CI.yml
Outdated
@@ -19,6 +19,7 @@ on: | |||
- main | |||
- release-branch-v* | |||
- dev | |||
- fix-issue-688 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reminder to remove after testing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes removing it for the final commit.
Description: <Describe what has changed.
Testing: Did three tests with the caches and
make test
turns from 4m to 25s andmake build
turns from 20m to roughly 2Documentation:
Contribution: Thanks Seth and Jeffrey for helping me with all the documents and samples codes: