ssm connection: pull bucket region info rather than taking from region var #1176
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Docs Build 📝Thank you for contribution!✨ This PR has been merged and your docs changes will be incorporated when they are next published. |
charles-paul-mox
suggested changes
Jun 1, 2022
markuman
approved these changes
Jun 30, 2022
tremble
approved these changes
Jun 30, 2022
|
Pull request merge failed: Resource not accessible by integration, You may need to manually rebase your PR and retry. |
Backport to stable-3: 💚 backport PR created✅ Backport PR branch: Backported as #1290 🤖 @patchback |
patchback bot
pushed a commit
that referenced
this pull request
Jun 30, 2022
…n var (#1176) SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. (cherry picked from commit 1be7da1)
Backport to stable-4: 💚 backport PR created✅ Backport PR branch: Backported as #1291 🤖 @patchback |
patchback bot
pushed a commit
that referenced
this pull request
Jun 30, 2022
…n var (#1176) SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. (cherry picked from commit 1be7da1)
softwarefactory-project-zuul bot
pushed a commit
that referenced
this pull request
Jun 30, 2022
…n var (#1176) (#1291) [PR #1176/1be7da11 backport][stable-4] ssm connection: pull bucket region info rather than taking from region var This is a backport of PR #1176 as merged into main (1be7da1). SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. Reviewed-by: Mark Chappell <None>
softwarefactory-project-zuul bot
pushed a commit
that referenced
this pull request
Jun 30, 2022
…n var (#1176) (#1290) [PR #1176/1be7da11 backport][stable-3] ssm connection: pull bucket region info rather than taking from region var This is a backport of PR #1176 as merged into main (1be7da1). SUMMARY Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket. Fixes #1190, #637 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin ADDITIONAL INFORMATION When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region) fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"} The tmp file has the following contents <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error> Steps to reproduce: run an ansible playbook using the following: aws_ec2 inventory aws_ssm connection type specify an s3 bucket (ansible_aws_ssm_bucket_name var) that is in a different location to the target node The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the _get_url function. Reviewed-by: Mark Chappell <None>
This was referenced Aug 27, 2022
This was referenced Aug 30, 2022
|
FYI @JLukeBlakey -- See #1428 |
|
Hi @phene - please see this (closed) PR as well: https://github.com/ansible-collections/community.aws/pull/743/files |
softwarefactory-project-zuul bot
pushed a commit
that referenced
this pull request
Oct 18, 2022
Fix detection of ssm connection bucket region Fix detection of ssm connection bucket region by ensuring that the boto client is created normally and able to use supported credential sources SUMMARY PR #1176 introduced detection of an S3 bucket's region to handle cases where the bucket is in a different region than the SSM connection itself. This change did not use the preferred mechanism for creating client objects, which caused it to not have access to credentials from all supported sources. It also broke the ability to use this plugin in partitions other than aws. (e.g. aws-us-gov). This change fixes this by building the bucket location client using _get_boto_client and the region for the connection to ensure it is both getting the proper credentials and starting in a region from the same partition as the client itself. From the default global region (or a hard-coded region), it will detect the bucket's region and continue S3 API calls using the bucket's own region. Fixes bug introduced from #1176 Fixes #1413 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin Reviewed-by: Markus Bergholz <git@osuv.de> Reviewed-by: Alina Buzachis <None> Reviewed-by: Mark Chappell <None>
patchback bot
pushed a commit
that referenced
this pull request
Oct 18, 2022
Fix detection of ssm connection bucket region Fix detection of ssm connection bucket region by ensuring that the boto client is created normally and able to use supported credential sources SUMMARY PR #1176 introduced detection of an S3 bucket's region to handle cases where the bucket is in a different region than the SSM connection itself. This change did not use the preferred mechanism for creating client objects, which caused it to not have access to credentials from all supported sources. It also broke the ability to use this plugin in partitions other than aws. (e.g. aws-us-gov). This change fixes this by building the bucket location client using _get_boto_client and the region for the connection to ensure it is both getting the proper credentials and starting in a region from the same partition as the client itself. From the default global region (or a hard-coded region), it will detect the bucket's region and continue S3 API calls using the bucket's own region. Fixes bug introduced from #1176 Fixes #1413 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin Reviewed-by: Markus Bergholz <git@osuv.de> Reviewed-by: Alina Buzachis <None> Reviewed-by: Mark Chappell <None> (cherry picked from commit fa58965)
patchback bot
pushed a commit
that referenced
this pull request
Oct 18, 2022
Fix detection of ssm connection bucket region Fix detection of ssm connection bucket region by ensuring that the boto client is created normally and able to use supported credential sources SUMMARY PR #1176 introduced detection of an S3 bucket's region to handle cases where the bucket is in a different region than the SSM connection itself. This change did not use the preferred mechanism for creating client objects, which caused it to not have access to credentials from all supported sources. It also broke the ability to use this plugin in partitions other than aws. (e.g. aws-us-gov). This change fixes this by building the bucket location client using _get_boto_client and the region for the connection to ensure it is both getting the proper credentials and starting in a region from the same partition as the client itself. From the default global region (or a hard-coded region), it will detect the bucket's region and continue S3 API calls using the bucket's own region. Fixes bug introduced from #1176 Fixes #1413 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin Reviewed-by: Markus Bergholz <git@osuv.de> Reviewed-by: Alina Buzachis <None> Reviewed-by: Mark Chappell <None> (cherry picked from commit fa58965)
|
Changes here introduced the usage of Is that call too conservative in this case considering that:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html |
softwarefactory-project-zuul bot
pushed a commit
that referenced
this pull request
Nov 20, 2022
[PR #1428/fa58965f backport][stable-5] Fix detection of ssm connection bucket region This is a backport of PR #1428 as merged into main (fa58965). Fix detection of ssm connection bucket region by ensuring that the boto client is created normally and able to use supported credential sources SUMMARY PR #1176 introduced detection of an S3 bucket's region to handle cases where the bucket is in a different region than the SSM connection itself. This change did not use the preferred mechanism for creating client objects, which caused it to not have access to credentials from all supported sources. It also broke the ability to use this plugin in partitions other than aws. (e.g. aws-us-gov). This change fixes this by building the bucket location client using _get_boto_client and the region for the connection to ensure it is both getting the proper credentials and starting in a region from the same partition as the client itself. From the default global region (or a hard-coded region), it will detect the bucket's region and continue S3 API calls using the bucket's own region. Fixes bug introduced from #1176 Fixes #1413 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin Reviewed-by: Mark Chappell <None>
softwarefactory-project-zuul bot
pushed a commit
that referenced
this pull request
Dec 2, 2022
[PR #1428/fa58965f backport][stable-4] Fix detection of ssm connection bucket region This is a backport of PR #1428 as merged into main (fa58965). Fix detection of ssm connection bucket region by ensuring that the boto client is created normally and able to use supported credential sources SUMMARY PR #1176 introduced detection of an S3 bucket's region to handle cases where the bucket is in a different region than the SSM connection itself. This change did not use the preferred mechanism for creating client objects, which caused it to not have access to credentials from all supported sources. It also broke the ability to use this plugin in partitions other than aws. (e.g. aws-us-gov). This change fixes this by building the bucket location client using _get_boto_client and the region for the connection to ensure it is both getting the proper credentials and starting in a region from the same partition as the client itself. From the default global region (or a hard-coded region), it will detect the bucket's region and continue S3 API calls using the bucket's own region. Fixes bug introduced from #1176 Fixes #1413 ISSUE TYPE Bugfix Pull Request COMPONENT NAME aws_ssm connection plugin Reviewed-by: Mark Chappell <None>
|
This should be roll-back.
|
|
@Hokwang , Thank you for taking the time to look into this. However, adding a comment to PR that was merged over a year ago generally isn't the best way to raise issues.
If you're seeing a specific issue then please open an Issue and be more specific about what you believe is broken (i.e. error messages, which version of the code you're using, etc). "It has a problem" does nothing to aid in the diagnosis of the issue and if we can't reproduce the issue, we can't test fixes.
This change predated amazon's recommendation by about 9 months. You're welcome to open a PR if you believe you have a better way to handle this. |
abikouo
pushed a commit
to abikouo/community.aws
that referenced
this pull request
Oct 24, 2023
ec2_eip - Remove deprecated instance_id alias SUMMARY fixes: ansible-collections#1176 Removes deprecated instance_id alias ISSUE TYPE Feature Pull Request COMPONENT NAME ec2_eip ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
Fix issue where syntax error is reported if using ssm connection and the target node is located in a different region to the s3 bucket.
Fixes #1190, #637
ISSUE TYPE
COMPONENT NAME
aws_ssm connection plugin
ADDITIONAL INFORMATION
When using SSM for ansible connection and the target node is in a different region to the s3 bucket used, the playbook immediately errors with the following. (There are no issues when both target and s3 bucket are in the same region)
fatal: [i-04444a7f03cc2bffd]: FAILED! => {"ansible_facts": {}, "changed": false, "failed_modules": {"ansible.legacy.setup": {"failed": true, "module_stderr": "", "module_stdout": " File \"/tmp/ansible/ansible-tmp-1653576081.8378458-29658-258097978113216/AnsiballZ_setup.py\", line 1\r\r\n <?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\r\n ^\r\r\nSyntaxError: invalid syntax\r\r", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}}, "msg": "The following modules failed to execute: ansible.legacy.setup\n"}The tmp file has the following contents
<?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationQueryParametersError</Code><Message>Error parsing the X-Amz-Credential parameter; the region 'us-east-1' is wrong; expecting 'eu-west-1'</Message><Region>eu-west-1</Region><RequestId>4VTAGR4C1V9ATBJT</RequestId><HostId>OahjGsFQHlr3ihxobH/yyH7Mzxq98mwjcb6+J3Y2EifDU7FykCe8b6QJTNodIG5WSquVeJF+Zsk=</HostId></Error>Steps to reproduce: run an ansible playbook using the following:
The presigned url generated includes the region the s3 bucket is in, so this region must be used for the session obtained in the
_get_urlfunction.