An error occurred when calling the HeadBucket operation: Bad Request #896
Labels
bug 🐞
Something isn't working
Comments
|
Sorry about that. To help me debugging this can you provide the region of your S3 bucket and the region the profile is set up as. I want to double check there isn't a redirect issue. |
|
I'm running in eu-west-3 running gitpod Login to AWS is basically |
|
I initially thought it could be something weird with the bucket region but it looks like paws is working fine with it. I wasn't able to reproduce the error using my sso role. However it is using aws-cli to initialise the role. options(paws.aws_sso_creds = T)
library(paws.storage)
sso_profile <- "paws_sso_profile"
region <- "eu-west-3"
client <- s3(config(credentials(profile = sso_profile), region = region))
bucket <- sprintf("paws_%s", paste(sample(letters, 8, T), collapse = ""))
key <- "to/some/file/xyz.txt"
client$create_bucket(
Bucket = bucket,
CreateBucketConfiguration = list(LocationConstraint = region)
)
resp <- client$put_object(
Bucket = bucket,
Key = key,
Body = charToRaw("dummy data")
)
resp <- client$list_objects_v2(
Bucket = bucket
)
resp <- client$head_object(
Bucket = bucket,
Key = key
)
resp <- client$delete_object(Bucket = bucket, Key = key)library(s3fs)
s3_file_system(profile_name = sso_profile, region_name = region)
s3_dir_ls(s3_path(bucket))
s3_file_stream_out(charToRaw("dummy data"), s3_path(bucket, key))
s3_dir_ls(s3_path(bucket), recurse = T)
s3_file_exists(s3_path(bucket, key))
s3_file_delete(s3_path(bucket, key)) |
|
How what would be the appropriate way to find out what credentials are being sent to AWS? |
|
You should be able to get to the logs using |
|
In order to compare the output with what is sent with paws.commons 0.7.7, paws.storage 0.7.0 and s3fs 0.1.7 (where there is no BAD request), what would be the appropriate way to get what is sent to AWS using these versions, such that I can make the comparison between the 2? |
|
Paws is able to log the request to aws: paws.common 0.8.0: options(paws.log_level = 4)
library(paws.storage)
sso_profile <- "paws"
bucket <- "paws-region"
client <- s3(config(credentials(profile = sso_profile)))
resp <- client$head_object(Bucket = bucket, Key = "to/some/file/xyz.txt")paws.common: 0.7.7: options(paws.log_level = 3)
library(paws.storage)
sso_profile <- "paws"
bucket <- "paws-region"
client <- s3(config(credentials(profile = sso_profile)))
resp <- client$head_object(Bucket = bucket, Key = "to/some/file/xyz.txt") |
|
Note: logs from the request to aws have been moved to DEBUG level in paws.common 0.8.0 this was to prevent the INFO log level being saturated. |
I'm seeing with 0.7.7 On paws.commons 0.8.1 I see |
|
With the paws.common 0.7.7 does it do a redirect? In my example above paws does a redirect. I wonder if it is failing in the redirect 🤔 Initially it does a Bad Request due to the wrong region getting signed it then will do a redirect getting the region details from the initial Bad Request |
|
This is the 0.8.1 where I replaced my bucket to mybucket and the key This is the 0.7.0 where I replaced my bucket to mybucket and the key |
|
if you set |
|
This is 0.7.0 where I replaced my bucket to mybucket and the key This is 0.8.0 The X-Amz-Security-Token are different between the 2 versions. |
|
I wouldn't expect X-Amz-Security-Token to be the same with each session. Paws does an sso call to get an assumed role for each session. So I would expect different X-Amz-Security-Token for each R session of paws.common 0.7.7 |
|
What happens when you have paws.storage 0.7.0 and paws.common 0.8.0? I am wondering if it is the endpoints that is causing this 🤔. As there is no redirect in 0.7.7 paws.common |
|
This is a very odd logging message: |
In 0.8.0 I see for X-Amz-Security-Token a sequence of 235letters/numbers |
|
Oh that is interesting. Yes i would expect a similar length for the security token. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
With version paws.commons 0.8.1, paws.storage 0.8.0 and s3fs 0.1.7 I get for the following
This is not the case with paws.commons 0.7.7, paws.storage 0.7.0 and s3fs 0.1.7
Where should I look in order to try to debug this?
Is this related to #844?
The text was updated successfully, but these errors were encountered: