Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

push: make sure source files exist before applying changes #10185

Closed
efiop opened this issue Dec 20, 2023 · 1 comment
Closed

push: make sure source files exist before applying changes #10185

efiop opened this issue Dec 20, 2023 · 1 comment
Assignees
Labels
bug Did we break something?

Comments

@efiop
Copy link
Contributor

efiop commented Dec 20, 2023

In cloud versioning case when we delete our cache and workspace data and try to dvc push, it will think that it needs to modify/relink a file and in order to do that will delete it but then fail to create a new one because it will turn out that there is no source for data.

We need to add a step that will filter out missing data (probably somewhere even before index.checkout.compare()) from a desired index, so that we don't even think that we can create those missing files.

One of our users already ran into this.

This also applies to dvc checkout and possibly other operations (don't think so, but need to check).

@efiop efiop self-assigned this Dec 20, 2023
@efiop efiop added bug Did we break something? A: cloud-versioning labels Dec 20, 2023
@efiop efiop added this to DVC Dec 20, 2023
@efiop efiop moved this to Done in DVC Dec 20, 2023
@efiop efiop moved this from Done to In Progress in DVC Dec 20, 2023
@efiop efiop changed the title push: make sure source files exists before applying changes push: make sure source files exist before applying changes Dec 20, 2023
@efiop
Copy link
Contributor Author

efiop commented Dec 20, 2023

Fixed by iterative/dvc-data#482 in #10181

Regarding checkout, it is mitigated by the fact that we double check files that we might delete, though that behaviour is a bit contraversial as well and might be changed in the future (to not delete unless we have a file to replace it with, even if deleted file is also in the cache).

There is one more unrelated issue about pulling missing files, but I'll introduce a fix separately. (EDIT: see #10188)

@efiop efiop closed this as completed Dec 20, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in DVC Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Did we break something?
Projects
No open projects
Archived in project
Development

No branches or pull requests

1 participant