[blob] - HTTP header lookup should be case-insensitive

[maorleger]

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.16

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

npm

What's the Node.js version?

16

What problem was encountered? When using azurite with @azure/storage-blob, BlobSharedKeyAuthenticator fails to validate signature due to case-sensitive header lookup

Steps to reproduce the issue?

Found when investigating Azure/azure-sdk-for-js#20690

Steps to reproduce the behavior:

1. Clone repro case from https://github.com/nulltoken/AppInsights-azure-storage-blob-repro-case
2. yarn
3. yarn azurite:start (with loose mode enabled)
4. yarn test

Debug logs are provided for both a healthy case (using lowercased headers) and a repro case (using original header casing)

debug_healthy.log
debug_repro.log

Have you found a mitigation/solution?

Downgrading to @azure/core-http@2.2.2 works as a mitigation, but is not possible for everyone.

According to the http spec and MDN, header names are case-insensitive. but it does not seem to be the case when using azurite to fetch the header fields used to create the signature in BlobSharedKeyAuthenticator and possible others, leading to failure to authenticate as shown in the attached logs...

[blueww]

@maorleger

I see a 403 error in debug_repro.log, after looks deep into it, I see it's cause by the stringTosign generate signature from Azurite v3, is not aligned with the one in the request.

In Azurite:

• signature: BFNpl+XD0qgZ6boQoFX+43zs50FLye0ZoDmtpaWU118=
• StringTosign: "GET\n\n\n\n\n\n\n\n\nNOT_INITIALIZED_YET\n\n\nx-ms-client-request-id:ac3eb318-2399-41d7-9bf2-c79721d8d633\nx-ms-date:Tue, 08 Mar 2022 18:38:22 GMT\nx-ms-encryption-algorithm:AES256\nx-ms-request-id:8cd42e6c74d94c82a9d1b3920129879a\nx-ms-request-root-id:|d5f24f35310e41ea90b22195a0110bb3.a2937e87ca314a47.\nx-ms-version:2020-10-02\n/devstoreaccount1/devstoreaccount1/2460d96b-132d-4d46-9e30-23d209178ebe/something.json"

In the request:

• signature : iK7QVWpnbOXUUJi/ikFdAwVsg2l62HjUtwvvpse6vdA=
• In my test, It matches StringTosign as : "GET\n\n\n\n\n\n\n\n\nNOT_INITIALIZED_YET\n\n\nx-ms-client-request-id:ac3eb318-2399-41d7-9bf2-c79721d8d633\nx-ms-date:Tue, 08 Mar 2022 18:38:22 GMT\nx-ms-encryption-algorithm:AES256\nx-ms-version:2020-10-02\n/devstoreaccount1/devstoreaccount1/2460d96b-132d-4d46-9e30-23d209178ebe/something.json"

So remove "x-ms-request-id:8cd42e6c74d94c82a9d1b3920129879a\nx-ms-request-root-id:|d5f24f35310e41ea90b22195a0110bb3.a2937e87ca314a47.\n" from azurite generated stringTosign, the signature will match.

So it looks the mismatch is caused by :

• the client not include "x-ms-request-id", "x-ms-request-root-id" when generated StringTosign
• But Azurite V3 will include the 2 headers in generate stringTosign.

From the debug_healthy.log, the 2 headers are not included in request, so the issue not happen.

From the doc https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key#constructing-the-canonicalized-headers-string, it looks the 2 headers should be included in StringTosign (all headers start with "x-ms-" should be included), so it looks Azurite v3 behavior is aligned with doc.

So would you please confirm:

1. Does the same request works on Azure server? (include header: "x-ms-request-id", "x-ms-request-root-id")
   If so, please give the request ID.

[maorleger]

Thank you for your help and insight here! We've identified the problem thanks to your observation of the added headers. It seems like the case sensitivity check needed to happen in application insights and that has been merged, going to close this via microsoft/ApplicationInsights-node.js#921