Storage blob track2 (#5894)

* Blob storage track 2

* Started pylint fixes

* Clean pylint

* Started blob walk

* Started docstrings

* Support walk blob

* Pylint

* Refactored upload

* Pylint

.gitignore

@@ -70,4 +70,5 @@ src/build
 # [begoldsm] ignore virtual env if it exists.
 adlEnv/
 
-code_reports
+code_reports
+sdk/storage/azure-storage-blob/tests/settings_real.py

sdk/storage/azure-storage-blob/HISTORY.md

@@ -0,0 +1,54 @@
+# Change Log azure-storage-blob
+
+> See [BreakingChanges](BreakingChanges.md) for a detailed list of API breaks.
+
+## Version 2.0.0:
+
+- New API.
+
+## Version 1.5.0:
+
+- Added new method list_blob_names to efficiently list only blob names in an efficient way.
+
+## Version 1.4.0:
+
+- azure-storage-nspkg is not installed anymore on Python 3 (PEP420-based namespace package)
+- copy_blob method added to BlockBlobService to enable support for deep sync copy.
+
+## Version 1.3.1:
+
+- Fixed design flaw where get_blob_to_* methods buffer entire blob when max_connections is set to 1.
+- Added support for access conditions on append_blob_from_* methods.
+
+## Version 1.3.0:
+
+- Support for 2018-03-28 REST version. Please see our REST API documentation and blog for information about the related added features.
+- Added support for setting static website service properties.
+- Added support for getting account information, such as SKU name and account kind.
+- Added support for put block from URL(synchronously).
+
+## Version 1.2.0rc1:
+
+- Support for 2017-11-09 REST version. Please see our REST API documentation and blog for information about the related added features.
+- Support for write-once read-many containers.
+- Added support for OAuth authentication for HTTPS requests(Please note that this feature is available in preview).
+
+## Version 1.1.0:
+
+- Support for 2017-07-29 REST version. Please see our REST API documentation and blogs for information about the related added features.
+- Added support for soft delete feature. If a delete retention policy is enabled through the set service properties API, then blobs or snapshots could be deleted softly and retained for a specified number of days, before being permanently removed by garbage collection.
+- Error message now contains the ErrorCode from the x-ms-error-code header value.
+
+## Version 1.0.0:
+
+- The package has switched from Apache 2.0 to the MIT license.
+- Fixed bug where get_blob_to_* cannot get a single byte when start_range and end_range are both equal to 0.
+- Optimized page blob upload for create_blob_from_* methods, by skipping the empty chunks.
+- Added convenient method to generate container url (make_container_url).
+- Metadata keys are now case-preserving when fetched from the service. Previously they were made lower-case by the library.
+
+## Version 0.37.1:
+
+- Enabling MD5 validation no longer uses the memory-efficient algorithm for large block blobs, since computing the MD5 hash requires reading the entire block into memory.
+- Fixed a bug in the _SubStream class which was at risk of causing data corruption when using the memory-efficient algorithm for large block blobs.
+- Support for AccessTierChangeTime to get the last time a tier was modified on an individual blob.

sdk/storage/azure-storage-blob/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 Microsoft
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

sdk/storage/azure-storage-blob/MANIFEST.in

@@ -0,0 +1,4 @@
+include *.md
+include azure/__init__.py
+include azure/storage/__init__.py
+include LICENSE.txt

sdk/storage/azure-storage-blob/README.md

@@ -0,0 +1,15 @@
+
+
+# Azure Storage Blobs client library for Python
+
+## Getting started
+
+## Key concepts
+
+## Examples
+
+## Troubleshooting
+
+## Next steps
+
+## Contributing

sdk/storage/azure-storage-blob/azure/__init__.py

@@ -0,0 +1 @@
+__path__ = __import__('pkgutil').extend_path(__path__, __name__)  # type: str

sdk/storage/azure-storage-blob/azure/storage/__init__.py

@@ -0,0 +1 @@
+__path__ = __import__('pkgutil').extend_path(__path__, __name__)  # type: str

sdk/storage/azure-storage-blob/azure/storage/blob/__init__.py

@@ -0,0 +1,28 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+__version__ = "3.0.0a8"
+
+from .common import BlobType, StorageErrorCode, LocationMode
+from .blob_client import BlobClient
+from .container_client import ContainerClient
+from .blob_service_client import BlobServiceClient
+from .lease import LeaseClient
+from ._policies import ExponentialRetry, LinearRetry, NoRetry
+
+
+__all__ = [
+    'BlobServiceClient',
+    'ContainerClient',
+    'BlobClient',
+    'BlobType',
+    'LeaseClient',
+    'StorageErrorCode',
+    'ExponentialRetry',
+    'LinearRetry',
+    'NoRetry',
+    'LocationMode'
+]

sdk/storage/azure-storage-blob/azure/storage/blob/_deserialize.py

@@ -0,0 +1,42 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for
+# license information.
+# --------------------------------------------------------------------------
+
+from .models import BlobProperties
+from .models import ContainerProperties
+
+
+def deserialize_metadata(response, _, headers):  # pylint: disable=unused-argument
+    raw_metadata = {k: v for k, v in response.headers.items() if k.startswith("x-ms-meta-")}
+    return {k[10:]: v for k, v in raw_metadata.items()}
+
+
+def deserialize_blob_properties(response, obj, headers):
+    metadata = deserialize_metadata(response, obj, headers)
+    blob_properties = BlobProperties(
+        metadata=metadata,
+        **headers
+    )
+    if 'Content-Range' in headers:
+        if 'x-ms-blob-content-md5' in headers:
+            blob_properties.content_settings.content_md5 = headers['x-ms-blob-content-md5']
+        else:
+            blob_properties.content_settings.content_md5 = None
+    return blob_properties
+
+
+def deserialize_blob_stream(response, obj, headers):
+    blob_properties = deserialize_blob_properties(response, obj, headers)
+    obj.properties = blob_properties
+    return response.location_mode, obj
+
+
+def deserialize_container_properties(response, obj, headers):
+    metadata = deserialize_metadata(response, obj, headers)
+    container_properties = ContainerProperties(
+        metadata=metadata,
+        **headers
+    )
+    return container_properties