Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sch: Temporarily disable chunk cache #465

Merged
merged 1 commit into from
Jan 23, 2024
Merged

Sch: Temporarily disable chunk cache #465

merged 1 commit into from
Jan 23, 2024

Conversation

jpsamaroo
Copy link
Member

The scheduler chunk cache has been around for a while, but unfortunately was never really correct in the face of mutating data. If data is mutated by any Dagger task, Dagger's chunk cache may still serve the stale, non-mutated data copy. This issue has come up in a number of places, and so it needs to be resolved now before causing further issues. A proper fix is still in the works (as part of a larger set of changes), but in the meantime, this hotfix will have to do.

@jpsamaroo
Copy link
Member Author

Aside: according to Dagger's current semantics (that tasks are functional in nature, and so arguments should not be mutated), this is technically correct behavior, but it is also confusing and hard to disable when you do need mutation. It's also correct to not cache chunk values, so it's valid to remove it. Future changes will enable mutation of arguments with some simple argument annotations, so this work is really preparing for that, and avoiding a quiet, hard-to-detect footgun until that support lands.

@jpsamaroo jpsamaroo mentioned this pull request Jan 23, 2024
@jpsamaroo jpsamaroo merged commit 276e4f0 into master Jan 23, 2024
13 checks passed
@jpsamaroo jpsamaroo deleted the jps/no-cache branch January 23, 2024 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant