Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azure.storage.blob.ContainerClient() initializer does not support emulated blob storage from Azurite #12568

Closed
chrisyeh96 opened this issue Jul 16, 2020 · 8 comments
Assignees
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. feature-request This issue requires a new behavior in the product in order be resolved. Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)
Milestone

Comments

@chrisyeh96
Copy link
Contributor

  • Package Name: azure-storage-blob
  • Package Version: 12.3.2
  • Operating System: Windows 10 Version 10.0.19041 Build 19041, running WSL 1
  • Python Version: 3.8.3

Describe the bug
azure.storage.blob.ContainerClient() throws errors on emulated local blob storage.

To Reproduce
Steps to reproduce the behavior:

from azure.storage.blob import ContainerClient
ContainerClient(
    account_url='http://127.0.0.1:1000/devstoreaccount1',
    container_name='newcontainer',
    credential='Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==')

Running Code in Python REPL:

>>> from azure.storage.blob import ContainerClient
>>> ContainerClient(
...     account_url='http://127.0.0.1:1000/devstoreaccount1',
...     container_name='newcontainer',
...     credential='Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cyeh/miniconda3/envs/env/lib/python3.8/site-packages/azure/storage/blob/_container_client.py", line 150, in __init__
    super(ContainerClient, self).__init__(parsed_url, service='blob', credential=credential, **kwargs)
  File "/home/cyeh/miniconda3/envs/env/lib/python3.8/site-packages/azure/storage/blob/_shared/base_client.py", line 90, in __init__
    self.credential = format_shared_key_credential(account, credential)
  File "/home/cyeh/miniconda3/envs/env/lib/python3.8/site-packages/azure/storage/blob/_shared/base_client.py", line 330, in format_shared_key_credential
    raise ValueError("Unable to determine account name for shared key credential.")
ValueError: Unable to determine account name for shared key credential.
>>>

Expected behavior

There should not be any error. Both Azurite and the Azure storage emulator assume the 'http://127.0.0.1:1000/devstoreaccount1' account URL, but it seems like the Azure Python SDK sadly does not support this.

@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jul 16, 2020
@kaerm kaerm added Client This issue points to a problem in the data-plane of the library. Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files) labels Jul 16, 2020
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jul 16, 2020
@ghost
Copy link

ghost commented Jul 16, 2020

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

@kaerm
Copy link
Contributor

kaerm commented Jul 16, 2020

cc: @jongio

@lmazuel lmazuel added this to the Backlog milestone Jul 16, 2020
@lmazuel lmazuel added feature-request This issue requires a new behavior in the product in order be resolved. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jul 20, 2020
@xiafu-msft
Copy link
Contributor

xiafu-msft commented Jul 21, 2020

Hi @chrisyeh96
The second commit in #12619 is to fix this problem.
Currently you can use this as a workaround to create a container client

credential={
    "account_name": "devstoreaccount1",
    "account_key": "youraccouantkey"
}

@xiafu-msft
Copy link
Contributor

The package has been released, you can install azure-storage-blob==12.4.0 to use the package with the problem fixed.

@chrisyeh96
Copy link
Contributor Author

Thank you @xiafu-msft! Finally got around to updating to the latest version of azure-storage-blob (12.5.0), and I can confirm that this issue has been fixed.

@CHBrianLiu
Copy link

Hi there,

I was trying to use Docker to run my application which involves interaction with Azure blob storage. However, when I used Azurite to test it locally, I assigned the account_url http://host.docker.internal:10000/devstoreaccount1, but the client library seemed not able to parse the URL.

container_name: ContainerClient(
    account_url="http://host.docker.internal:10000/devstoreaccount1",
    container_name="container_name",
    credential="Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
)
  File "./app/models/azure_storage_blob.py", line 20, in <dictcomp>
    container_name: ContainerClient(
  File "/usr/local/lib/python3.8/site-packages/azure/storage/blob/_container_client.py", line 151, in __init__
    super(ContainerClient, self).__init__(parsed_url, service='blob', credential=credential, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/azure/storage/blob/_shared/base_client.py", line 94, in __init__
    self.credential = _format_shared_key_credential(self.account_name, credential)
  File "/usr/local/lib/python3.8/site-packages/azure/storage/blob/_shared/base_client.py", line 346, in _format_shared_key_credential
    raise ValueError("Unable to determine account name for shared key credential.")
ValueError: Unable to determine account name for shared key credential.

Then I took a look at the commit @xiafu-msft mentioned above, it seems that the logic to determine if the user is using Azurite is by localhost and 127.0.0.1 keywords.

@varun-shahdadpuri
Copy link

@xiafu-msft How do we use Azurite in case we cannot expose Azurite using localhost or 127.0.0.1?

@amishra-dev
Copy link

Hi @varun-shahdadpuri it might be better to ask that question here
I think the SDK can parse IP addresses for the azurite case

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. feature-request This issue requires a new behavior in the product in order be resolved. Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

7 participants