Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't segregate cache entries per Workflow definition #16

Open
1 task done
ubiratansoares opened this issue Jul 21, 2023 · 1 comment
Open
1 task done

Can't segregate cache entries per Workflow definition #16

ubiratansoares opened this issue Jul 21, 2023 · 1 comment

Comments

@ubiratansoares
Copy link

Troubleshooting

  • I've searched discuss.bitrise.io for possible solutions.
  • Which version of the step is effected? All versions
  • Is the issue reproducible with the latest version? YES
  • Does the issue happen sporadically, or every time? EVERY TIME
  • Is the issue reproducible locally by following our local debug guide? NOT APPLICABLE

Issue description

Right now, this step computes the same cache key regardless the Gradle task being executed and regardless which Workflow invoked that task for the a given commit hash. This means that in a complex pipeline architecture like :

  • Workflow A -> executes unit tests (JVM)
  • Workflow B -> builds releaseable artifacts (eg, AAB)
  • Workflow C -> runs static analysis (ktlint, detekt, spotless, via Gradle)

under the same trigger, all of them will share the same Gradle cache entry for the same commit hash when applying step-restore-gradle-cache, which might not be the ideal.

For instance, some build tasks related to annotation processors (like kapt) may corrupt cache entries since build outputs are not guaranteed to be stable, which means that we have to invalidate the whole cache for all Workflows despite an offending task corrupting the CI cache belonging to an specific Workflow.

A possible way to improve on such a situation is aggregating information from the Workflow/Job name as part of the cache-key. This is what gradle-build-action does for GHA Jobs, for example.

In this step, this is not the current behavior as per the definition of the key

key = `{{ .OS }}-{{ .Arch }}-gradle-cache-{{ checksum "**/*.gradle*" "**/gradle-wrapper.properties" "**/gradle.properties" "**/gradle/libs.versions.toml" }}`

Hence, I'd love to learn from the team if we could have such a behavior, eventually as opt-in.

A couple of possibilities for a new public API in this step

steps:

# ...

- save-gradle-cache@1:
  inputs:
    segregate-per-workflow: true | false (default)
steps:

# ...

- save-gradle-cache@1:
  inputs:
    cache-hint: none (default) | $BITRISE_TRIGGERED_WORKFLOW_ID  | my-hint

I'm aware we could implement such a behavior with the step-save-cache (and its counterpart) and I'm also aware that such a change must reflect on step-restore-gradle-cache, but since corrupted Gradle caches is a common scenario in large / old Android projects / builds out there, perhaps we can consider this addition.

@garry-jeromson
Copy link

garry-jeromson commented Dec 6, 2024

This would be a great addition, and especially helpful for complex Kotlin Multiplatform pipelines. In these cases the cache contents for Android and iOS build workflows often varies quite a lot; without splitting them we end up lugging a lot of unhelpful cache data back and forth..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants