Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

az storage blob sync's deletion behavior can lead to unexpected data loss #21414

Closed
jayqi opened this issue Feb 23, 2022 · 1 comment · Fixed by #21662
Closed

az storage blob sync's deletion behavior can lead to unexpected data loss #21414

jayqi opened this issue Feb 23, 2022 · 1 comment · Fixed by #21662
Assignees
Labels
Auto-Assign Auto assign by bot customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage az storage
Milestone

Comments

@jayqi
Copy link

jayqi commented Feb 23, 2022

Is your feature request related to a problem? Please describe.

Currently, the az storage blob sync command will delete blobs at the destination that are not present in the source. This is hardcoded into the command, with no documentation of its behavior or an option to change it. Additionally, this is inconsistent with the default behavior of other prominent tools for Azure as well as other cloud providers' storage services, including azcopy, which include a flag to enable deleting but do not do so by default.

In fact, this command is implemented in azure-cli by wrapping azcopy sync but then overriding azcopy's default. Relevant source code:

def storage_blob_sync(cmd, client, source, destination, exclude_pattern=None, include_pattern=None,
exclude_path=None):
azcopy = _azcopy_blob_client(cmd, client)
flags = ['--delete-destination=true']

This is a risky design that can lead to unexpected deletion by users and result in data loss. (This happened to me.)

Describe the solution you'd like

  1. Add a --delete-destination to match the functionality of azcopy sync.
  2. Have that flag default to false.
@ghost ghost added customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage az storage labels Feb 23, 2022
@ghost ghost added this to the Backlog milestone Feb 23, 2022
@ghost ghost assigned evelyn-ys Feb 23, 2022
@ghost ghost added the Auto-Assign Auto assign by bot label Feb 23, 2022
@yonzhan
Copy link
Collaborator

yonzhan commented Feb 23, 2022

storage

evelyn-ys added a commit that referenced this issue Apr 18, 2022
…destination` default to false (#21662)

* [Storage] Fix#21414:�z storage blob sync:Fix the flag --delete-destination default to false

* add storage blob sync flag --delete-destination

* add storage blob sync flag --delete-destination

* add storage blob sync flag --delete-destination

* fix suggestions

* fix suggestions

* fix suggestions

* Update src/azure-cli/azure/cli/command_modules/storage/operations/azcopy.py

Co-authored-by: Yishi Wang <yishiwang@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage az storage
Projects
None yet
3 participants