dvc status --remote

[helpmefindaname]

Hello,

my team and I are currently utilzing a dvc pipeline that checks if all data is pushted to the remote storage. This is currently done by pulling all data and then run dvc status -q. Pulling all data is slow, especially if you are storing many images. A faster way would be to only download all .dir cache files and just verify the existence of the respective hashes on the remote server, not having to spend much time on downloading all the files.

Ideally dvc status --remote or dvc status -r would do such a check for me.

[Otterpatsch]

Hey fellow MLer,

Did you try to run dvc data status --not-in-remote --json | grep -v not_in_remote it will fail if there is missing data on remote and succeed otherwise. Its working like it should for me and my teams pipeline.
As a reference see https://dvc.org/doc/user-guide/pipelines/running-pipelines#verify-pipeline-status

But if you want also to use dvc status to check if a stage or something has to be run again dvc exp run --allow-missing --dry or dvc repro--allow-missing --dry` is currently not working, at least for me and my team. See #9818 #9817

[helpmefindaname]

Thank you,
I will try that out instead.

[dberenbaum]

@helpmefindaname Let me know if it fixes the issue for you. How do you plan to use it? Are you running a CI job to check that everything is up to date?