Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Manual Backport][Stable-9] Synchronise tox configuration between amazon.aws and community.aws (#2486) #2494

Conversation

GomathiselviS
Copy link
Contributor

SUMMARY
tox configs needed a little cleanup to ensure that things were consistently in the import path when running pylint. ISSUE TYPE

Feature Pull Request

COMPONENT NAME
pyproject.toml
tests/unit/conftest.py
tox.ini
ADDITIONAL INFORMATION
See also: ansible-collections/community.aws#2219

Reviewed-by: Alina Buzachis
(cherry picked from commit 62ea880)

SUMMARY
ISSUE TYPE
  • Bugfix Pull Request
  • Docs Pull Request
  • Feature Pull Request
  • New Module Pull Request
COMPONENT NAME
ADDITIONAL INFORMATION

alinabuzachis and others added 30 commits October 28, 2024 12:46
SUMMARY

Bump release for main branch to 10.0.0-dev0

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
SUMMARY

Update main branch after releasing amazon.aws 9.0.0

ISSUE TYPE


Docs Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
SUMMARY

Add some missing FQCN on cloudwatch_metric_alarm integration tests,
and fix failure on downstream integration test run.

Refer: https://issues.redhat.com/browse/ACA-1961
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Helen Bailey <hebailey@redhat.com>
…et in vpc (#2374)

SUMMARY


Fixes: #2363
This PR handles the case in which the vpc does not have ipv6 set and the task to modify it has ipv6_cidr: false
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Bikouo Aubin
Reviewed-by: Alina Buzachis
Reviewed-by: Helen Bailey <hebailey@redhat.com>
…18.txt - Ignoring 'collection-deprecated-version' on 'plugins/modules/ec2_vpc_peering.py' is unnecessary (#2384)

SUMMARY

tests/sanity/ignore-2.18.txt:2:1: ansible-test: Ignoring 'collection-deprecated-version' on 'plugins/modules/ec2_vpc_peering.py' is unnecessary

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
…info (#2372)

SUMMARY

Adding a new module for retrieving parameter group info.
Can get info for

a specific RDS parameter group
the parameter group associated with a specified RDS instance
all parameter groups available in the current region.

Resolves #2313
ISSUE TYPE


New Module Pull Request

COMPONENT NAME

rds_instance_pram_group_info
ADDITIONAL INFORMATION
Created basic module, waiting on confirmation to move forward with adding tests, etc.

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Reviewed-by: Alina Buzachis
SUMMARY

When one or more TGWs exist with an empty description, the module will fail.
Fixes #2368

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ec2_transit_gateway
ADDITIONAL INFORMATION


Apparently, ansible-collections/community.aws#2086 failed to merge, and the ec2_transit_gateway module has now been migrated to amazon.aws. This PR just cherry-picks the commits on top of this repo.

Reviewed-by: Alina Buzachis
Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
SUMMARY


Fixes #2362
ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Mike Graves <mgraves@redhat.com>
Reviewed-by: Mark Chappell
SUMMARY


Fixes #2307.
Ansible previously generated warning is module output contained non UTF-8 data. Starting with version 2.18, it now throws an error, which prevents successful execution of lambda module.
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

lambda
ADDITIONAL INFORMATION

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Mike Graves <mgraves@redhat.com>
SUMMARY
Managing instances directly through autoscaling_group has lead to a significant sprawl in the code which is difficult to test.  With Amazon now also supporting "Instance Refresh" through its own API we can handle rolling replacements using less home-brew code (available through autoscaling_instance_refresh).  The autoscaling_instance also has support for a more complete set of attach/detach/standby/terminate functionality (including instance protection management).
Deprecates the following parameters:

replace_all_instances -> amazon.aws.autoscaling_instance_refresh
replace_batch_size (tied to replace_all_instances)
lc_check (tied to replace_all_instances)
lt_check (tied to replace_all_instances)
detach_instances -> amazon.aws.autoscaling_instance
replace_instances -> amazon.aws.autoscaling_instance
decrement_desired_capacity (tied to detach_instances and replace_instances)

This functionality provided through these parameters is now available through autoscaling_instance and autoscaling_instance_refresh
Also adds group_name as an alias for name on both autoscaling_instance_refresh and autoscaling_group for consistency with autoscaling_instance where name would be ambiguous.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME

autoscaling_group
autoscaling_instance
autoscaling_instance_refresh

ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Mark Chappell
SUMMARY
Fixes #2376.
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME
elbv2
elb_application_lb
ADDITIONAL INFORMATION

Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Reviewed-by: Bikouo Aubin
SUMMARY
With OpenSSH 9.5 the default format for SSH key generation was switched over to ed25519 (a good thing).  However, some older OSes don't support ed25519 (eg the older image we use for testing ec2_metadata_facts python 2 compatibility).
setup_sshkey now:

Explicitly generates an 4096 bit RSA key (key_material)
Generates the somewhat quirky md5sum based fingerprint AWS uses for RSA keys (fingerprint)
Explicitly generates an ed25519 key (another_key_material)
Pads the standard sha256 fingerprint to match the, technically correct, AWS format (another_fingerprint)

ec2_key test now also checks the fingerprint of the second imported key.
(Also fixes the issue seen in #2398 by switching ssh key generation back to the old 4096 bit RSA for key_material)
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
tests/integration/targets/setup_sshkey
tests/integration/targets/ec2_key
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
SUMMARY
Fixes #2293
A list of instance tag keys was added to the return values in version 5.5.0. This adds a new return value that includes the full key:value pairs for the instance tags.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
ec2_metadata_facts

Reviewed-by: Mark Chappell
Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Alina Buzachis
Reviewed-by: Bikouo Aubin
SUMMARY

Closes #2321

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ec2_instance

Reviewed-by: Alina Buzachis
Reviewed-by: Mike Graves <mgraves@redhat.com>
Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Bikouo Aubin
SUMMARY

Update FQCN of modules that moved to amazon.aws
Remove explicit call to collections from defaults stanza, now we're using FQCNs we don't need to pull it in.

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
tests/integration/targets
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
SUMMARY
Deletion state test forgot to re-fetch state after pause.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
kms_key
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Update main branch after release 9.1.0
SUMMARY
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
several
ADDITIONAL INFORMATION
Reviewed-by: Mark Chappell
Reviewed-by: Bikouo Aubin
SUMMARY


ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
SUMMARY

Closes #2390
Fix issue when trying to update RDS cluster with new domain
add unit tests to validate the fix

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

rds_cluster

Reviewed-by: Alina Buzachis
#2418)

SUMMARY

Closes #1872

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ec2_vpc_nat_gateway

Reviewed-by: Alina Buzachis
Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
… containing at least one rule with a managed prefix list (#2404)

SUMMARY
Closes #2373
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ec2_security_group

Reviewed-by: Alina Buzachis
…with create_changeset set (#2391)

SUMMARY

Fixes #2149
Fix bug where termination protection is not updated when create_changeset=true is used for stack updates

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

cloudformation
ADDITIONAL INFORMATION

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Bikouo Aubin
Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
SUMMARY
1.34 was released Nov 2023, 1.35 was released Aug 2024
Drops support for Python 3.7 (in line with botocore Python requirements)
Also fixes up the running the unit tests under tox.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
plugins/module_utils/botocore.py
ADDITIONAL INFORMATION
With Python/botocore/awscli updates we no longer need to pin PyYAML, coverage and flake8

Reviewed-by: Bikouo Aubin
SUMMARY

Fixes #2413
Adds support for record type 'SSHFP'

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

route53
ADDITIONAL INFORMATION

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Bikouo Aubin
SUMMARY

Closes #2402

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

inventory/aws_ec2

Reviewed-by: Helen Bailey <hebailey@redhat.com>
SUMMARY
Since we dropped support for the old boto SDK (#630) the boto3 parameter for get_aws_region() and get_aws_connection_info() has done nothing.  This finally removes the parameter (deprecated #2047)
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
plugins/module_utils/botocore.py
ADDITIONAL INFORMATION

Reviewed-by: Bikouo Aubin
Reviewed-by: Alina Buzachis
…gelogs (#2454)

##### SUMMARY

As has been recently noticed, we're being inconsistent with backporting things.

Based on the keys from the changelog fragments this action would perform the following actions:

- On "push" or applying the `mergeit` label
  -  If a PR contains **non-backportable** changes (`major_changes`, `breaking_changes`, `removed_features`) the `do_not_backport` label will be applied.  These change types are assumed to require a "major" release, and as such shouldn't be backported into a currently "stable" release.  At the same time it will remove any previously applied `backport-*` labels.
- When the "mergeit" label is applied:
  - If a PR contains changes that only need a **minor release** (minor_changes, deprecated_features) the `backport-9` label will be applied.
  - If a PR contains changes that only need a **patch release**, such as security/bug fixes (bugfixes, security_fixes), then `backport-8` and `backport-9` labels will both be applied.

##### ISSUE TYPE

- Feature Pull Request

##### COMPONENT NAME

.github/workflows

##### ADDITIONAL INFORMATION
…ating bucket on ceph cluster (#2457)

SUMMARY

Closes #2420
Do not use the default region as a location constraint when creating a bucket.

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

s3_bucket

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
SUMMARY
fixes various linting warnings:

redefined-builtin
redefined-outer-name
no-else-continue
simplifiable-if-statement
unused-import

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
plugins/modules/ec2_ami.py
plugins/modules/ec2_vpc_vpn.py
plugins/modules/s3_bucket.py
plugins/modules/s3_object.py
tests/unit/
ADDITIONAL INFORMATION
Also applies the "maybe_sleep" fixture to the ACM tests which have retries attached to them.

Reviewed-by: Alina Buzachis
SUMMARY
A few new features were released in version 9.0.0 but had been noted in the module documentation block with options added in version 8.3.0, which we never released. This just updates those documentation blocks to reflect the actual release version. I double checked and these were all included in the 9.0.0 release notes, so no changelog updates are needed.
ISSUE TYPE

Docs Pull Request

Reviewed-by: Mark Chappell
…ng the 'target' workflows rather than the workflow in the PR (#2461)
alinabuzachis and others added 16 commits January 15, 2025 15:37
SUMMARY

Terminator PR mattclay/aws-terminator#312
Closes #1976

ISSUE TYPE


New Module Pull Request

COMPONENT NAME

route53_key_signing_key
ADDITIONAL INFORMATION

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Alina Buzachis
Reviewed-by: Bikouo Aubin
SUMMARY
Syncing release changes for version 8.2.2.

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Alina Buzachis
Reviewed-by: Bikouo Aubin
SUMMARY
Minor ansible-lint issues from the ec2_vpc_nacl integration test
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
ec2_vpc_nacl
ADDITIONAL INFORMATION

Reviewed-by: Bikouo Aubin
Reviewed-by: Alina Buzachis
SUMMARY


ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
…hosted zone (#2421)

SUMMARY

route53_zone - Add support for enabling DNSSEC signing in a specific hosted zone
Will rebase once #2412 is merged.
Closes #1976

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

route53_zone
ADDITIONAL INFORMATION

Reviewed-by: Bikouo Aubin
Reviewed-by: Alina Buzachis
SUMMARY
Initial preparation work for reworking S3 error handling
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
module_utils/s3
ADDITIONAL INFORMATION
Shouldn't be backported until the relevant S3 module changes also land

Reviewed-by: Alina Buzachis
…kports fail (#2472)

SUMMARY

Add Github worflow that adds backport_failed label when automatic backports fail.
This will help keep track of any failed back doors that require manual actions to be taken.

ISSUE TYPE

    Bugfix Pull Request
    Docs Pull Request
    Feature Pull Request
    New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION
… file settings article, rather than the PHP SDK article (#2479)

SUMMARY
Fixed the link for Credentials Profile authentication documentation to point to the AWS CLI credential file settings article, rather than the PHP SDK article
ISSUE TYPE

Docs Pull Request

ADDITIONAL INFORMATION
It looks like maybe Amazon updated their documentation, as the old URL (https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_profiles.html) now automatically redirects to https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/welcome.html, which is their PHP SDK documentation. I updated the URL to a more accurate set of documentation for CLI configuration/Credentials Profiles. https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Signed-off-by: Alina Buzachis <abuzachis@redhat.com>
* Fixup patchback failure labelling

* Add debugging to backport tester
SUMMARY

Closes #2477

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

s3_object_info

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
…2486)

SUMMARY
tox configs needed a little cleanup to ensure that things were consistently in the import path when running pylint.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
pyproject.toml
tests/unit/conftest.py
tox.ini
ADDITIONAL INFORMATION
See also: ansible-collections/community.aws#2219

Reviewed-by: Alina Buzachis
SUMMARY
s3_bucket has some support for non-AWS "S3" APIs, this PR reworks s3_bucket module to more consistently handle the various Errors that get thrown when the non-AWS-S3 API doesn't cleanly support some of the more advanced S3 features.
Our testing for non-AWS APIs is near non-existent, so it's important that the patterns displayed in the module are easy to follow and cleanly handle things a feature not being supported if someone doesn't try to use it.
Also fixes idempotency when setting the simplistic "templated" ACLs.
ISSUE TYPE

Bugfix Pull Request
Feature Pull Request

COMPONENT NAME
s3_bucket
ADDITIONAL INFORMATION
I plan on migrating the remaining S3 modules in amazon.aws but this is just the first one.

Reviewed-by: Alina Buzachis
…s missing from network interface (#2488)

SUMMARY

Fixes https://issues.redhat.com/browse/ACA-2123

This PR addresses an issue where module tries to attach all security groups in the region that default SGs for any VPC in the region. Causing error below

fatal: [localhost]: FAILED! => {"boto3_version": "1.34.144", "botocore_version": "1.34.144", "changed": false,
"msg": "Could not apply change {'Groups': ['sg-xxxx', 'sg-yyyy', 'sg-zzzz']} to existing instance.: Failed to modify instance attribute"}


The subnet_id was previously passed as None when not found in the network_interface, causing failures when applying security group.
The logic was updated to use module.params.get("vpc_subnet_id") when provided in task, as a fallback, preventing None from being passed to discover_security_groups() and ensuring the correct subnet is used when the default security group is specified.


ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ec2_instance
ADDITIONAL INFORMATION


According to jira, the issue was introduced after 8.1.0 onwards (worked fine until and including 8.1.0)
Issue was only seen (during testing/reproducing) when network and vpc_subnet_id were both specified in the task.

Playbook used for testing/reproducing the issue.
---
- name: EC2 instance
  hosts: localhost
  gather_facts: no
  vars:
    instance_type: "t2.micro"
    subnet_id: "subnet-xxxxxxx"
    region: "ap-northeast-2"
     image_id: "ami-xxxxxxx"
  tasks:
    - name: Create the EC2 instance with proper tags
      amazon.aws.ec2_instance:
        image_id: "{{ image_id }}"
        instance_type: "{{ instance_type }}"
        network:
          assign_public_ip: false
          private_ip_address: "{{ ec2_private_ip | default(omit) }}"
        purge_tags: false
        region: "{{ region }}"
        security_groups: "{{ security_group | default('default') }}"
        tags:
          Owner: mandkulk
          Persistent: False
          Name: xxxxx-test-instance
        vpc_subnet_id: "{{ subnet_id }}"
        wait: true
        state: present
      register: ec2

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
Copy link
Contributor

brahmanim and others added 2 commits February 2, 2025 18:55
SUMMARY


Fix downstream integration test assertion failures when checking the AZ group_name attribute,
update assetion to use startswith instead of equal.
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

aws_az_info integration test target
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
SUMMARY

module_utils.botocore - fixed type aliasing.
module_utils.botocore - avoid assigning unused parts of exc_info return.
plugin_utils.botocore - fixed type aliasing.
module_utils._s3 - explicitly cast super to the parent type.
module_utils.exceptions - avoid assigning unused parts of exc_info return.

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
plugins/module_utils/_s3/common.py
plugins/module_utils/botocore.py
plugins/module_utils/exceptions.py
plugins/plugin_utils/botocore.py
ADDITIONAL INFORMATION

Reviewed-by: Bikouo Aubin
@GomathiselviS GomathiselviS changed the title [Manual Backport] Synchronise tox configuration between amazon.aws and community.aws (#2486) [Manual Backport][Stable-9] Synchronise tox configuration between amazon.aws and community.aws (#2486) Feb 3, 2025
alinabuzachis and others added 4 commits February 3, 2025 08:49
SUMMARY

Links update after branching stable-9

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
SUMMARY

Prep amazon.aws 9.0.0

ISSUE TYPE

Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Reviewed-by: Mark Chappell
SUMMARY

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

several
ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Bikouo Aubin
SUMMARY
Release prep for v9.1.1.

Reviewed-by: GomathiselviS <gomathiselvi@gmail.com>
Reviewed-by: Mark Chappell
…2486)

SUMMARY
tox configs needed a little cleanup to ensure that things were consistently in the import path when running pylint.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
pyproject.toml
tests/unit/conftest.py
tox.ini
ADDITIONAL INFORMATION
See also: ansible-collections/community.aws#2219

Reviewed-by: Alina Buzachis
(cherry picked from commit 62ea880)
@GomathiselviS GomathiselviS force-pushed the patchback/backports/stable-9/62ea880c6c5bb78dfbe2134b03fa7138acb516ec/pr-2486 branch from ac0160c to 36169ae Compare February 3, 2025 13:57
Copy link
Contributor

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/ansible-collections/amazon.aws for 2494,36169aeb4356c8e995b0ca8d514594cfdc3251ae

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.