Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lambda - Fix flaky integration tests which assume there are no other functions in the account #1279

Conversation

mdavis-xyz
Copy link
Contributor

@mdavis-xyz mdavis-xyz commented Nov 18, 2022

SUMMARY

The integration tests for lambda assume that there are no other lambdas in the account.
So when we index into [0], that's not necessarily the lambda we just created.

Note that this fix doesn't work. One of the assertions is failing.

TASK [lambda : lambda_info | Assert successfull retrieval of all information 2] ***
fatal: [testhost]: FAILED! => {
    "assertion": "lambda_infos_all.function[lambda_function_name].versions is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}

That attribute is defined. It's a list of 2 versions. I don't know why.
I'm just creating this PR anyway, so someone else can pickup where I left off.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

lambda_info

ADDITIONAL INFORMATION

Fixes #1277

@tremble tremble changed the title Fix flaky lambda integration tests which assume there are no other fu… Fix flaky lambda integration tests which assume there are no other functions in the account Nov 18, 2022
@tremble tremble changed the title Fix flaky lambda integration tests which assume there are no other functions in the account lambda - Fix flaky integration tests which assume there are no other functions in the account Nov 18, 2022
@softwarefactory-project-zuul
Copy link
Contributor

Build failed.

✔️ ansible-galaxy-importer SUCCESS in 4m 16s
✔️ build-ansible-collection SUCCESS in 5m 16s
ansible-test-sanity-aws-ansible-python38 FAILURE in 9m 07s (non-voting)
✔️ ansible-test-sanity-aws-ansible-2.12-python38 SUCCESS in 9m 29s
✔️ ansible-test-sanity-aws-ansible-2.13-python38 SUCCESS in 11m 11s
✔️ ansible-test-sanity-aws-ansible-2.14 SUCCESS in 9m 06s
✔️ ansible-test-units-amazon-aws-python36 SUCCESS in 6m 25s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 6m 07s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 6m 05s
✔️ cloud-tox-py3 SUCCESS in 4m 11s
✔️ ansible-test-changelog SUCCESS in 2m 13s
✔️ ansible-test-splitter SUCCESS in 2m 51s
integration-amazon.aws-1 FAILURE in 10m 54s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
⚠️ integration-amazon.aws-14 SKIPPED
⚠️ integration-amazon.aws-15 SKIPPED
⚠️ integration-amazon.aws-16 SKIPPED
⚠️ integration-amazon.aws-17 SKIPPED
⚠️ integration-amazon.aws-18 SKIPPED
⚠️ integration-amazon.aws-19 SKIPPED
⚠️ integration-amazon.aws-20 SKIPPED
⚠️ integration-amazon.aws-21 SKIPPED
⚠️ integration-amazon.aws-22 SKIPPED
⚠️ integration-community.aws-1 SKIPPED
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
⚠️ integration-community.aws-14 SKIPPED
⚠️ integration-community.aws-15 SKIPPED
⚠️ integration-community.aws-16 SKIPPED
⚠️ integration-community.aws-17 SKIPPED
⚠️ integration-community.aws-18 SKIPPED
⚠️ integration-community.aws-19 SKIPPED
⚠️ integration-community.aws-20 SKIPPED
⚠️ integration-community.aws-21 SKIPPED
⚠️ integration-community.aws-22 SKIPPED

@ansibullbot
Copy link

@ansibullbot ansibullbot added bug This issue/PR relates to a bug community_review has_issue integration tests/integration needs_triage plugins plugin (any type) tests tests labels Nov 18, 2022
@tremble
Copy link
Contributor

tremble commented Nov 18, 2022

Issue was introduced by ansible-collections/community.aws#1239 - the dict of dicts was deprecated for consistency with other modules.

We probably need to do something clever with selectattr :/

@mdavis-xyz
Copy link
Contributor Author

I've changed the tests to use selectattr now, instead of the deprecated .function.

But the assert .versions is not defined test fails. Why?

@github-actions
Copy link

github-actions bot commented Nov 28, 2022

Docs Build 📝

Thank you for contribution!✨

This PR has been merged and your docs changes will be incorporated when they are next published.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.

✔️ ansible-galaxy-importer SUCCESS in 4m 16s
✔️ build-ansible-collection SUCCESS in 5m 12s
ansible-test-sanity-aws-ansible-python38 FAILURE in 8m 11s (non-voting)
✔️ ansible-test-sanity-aws-ansible-2.12-python38 SUCCESS in 9m 33s
✔️ ansible-test-sanity-aws-ansible-2.13-python38 SUCCESS in 8m 50s
✔️ ansible-test-sanity-aws-ansible-2.14 SUCCESS in 8m 27s
✔️ ansible-test-units-amazon-aws-python36 SUCCESS in 6m 00s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 6m 01s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 6m 38s
✔️ cloud-tox-py3 SUCCESS in 3m 42s
✔️ ansible-test-changelog SUCCESS in 2m 18s
✔️ ansible-test-splitter SUCCESS in 2m 28s
integration-amazon.aws-1 FAILURE in 9m 32s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
⚠️ integration-amazon.aws-14 SKIPPED
⚠️ integration-amazon.aws-15 SKIPPED
⚠️ integration-amazon.aws-16 SKIPPED
⚠️ integration-amazon.aws-17 SKIPPED
⚠️ integration-amazon.aws-18 SKIPPED
⚠️ integration-amazon.aws-19 SKIPPED
⚠️ integration-amazon.aws-20 SKIPPED
⚠️ integration-amazon.aws-21 SKIPPED
⚠️ integration-amazon.aws-22 SKIPPED
⚠️ integration-community.aws-1 SKIPPED
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
⚠️ integration-community.aws-14 SKIPPED
⚠️ integration-community.aws-15 SKIPPED
⚠️ integration-community.aws-16 SKIPPED
⚠️ integration-community.aws-17 SKIPPED
⚠️ integration-community.aws-18 SKIPPED
⚠️ integration-community.aws-19 SKIPPED
⚠️ integration-community.aws-20 SKIPPED
⚠️ integration-community.aws-21 SKIPPED
⚠️ integration-community.aws-22 SKIPPED

Copy link
Contributor

@tremble tremble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the "defined" issue you're seeing is caused by testing against lambda_infos_all rather than lambda_infos_query_config

I have a slight preference for using a temporary variable when using something so complex to access a list/dict item, but changing that's certainly not a blocker.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

✔️ ansible-galaxy-importer SUCCESS in 4m 17s
✔️ build-ansible-collection SUCCESS in 5m 17s
ansible-test-sanity-aws-ansible-python38 FAILURE in 8m 00s (non-voting)
✔️ ansible-test-sanity-aws-ansible-2.12-python38 SUCCESS in 10m 42s
✔️ ansible-test-sanity-aws-ansible-2.13-python38 SUCCESS in 9m 34s
✔️ ansible-test-sanity-aws-ansible-2.14 SUCCESS in 8m 56s
✔️ ansible-test-units-amazon-aws-python36 SUCCESS in 7m 13s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 7m 57s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 5m 57s
✔️ cloud-tox-py3 SUCCESS in 3m 19s
✔️ ansible-test-changelog SUCCESS in 2m 12s
✔️ ansible-test-splitter SUCCESS in 2m 52s
✔️ integration-amazon.aws-1 SUCCESS in 9m 49s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
⚠️ integration-amazon.aws-14 SKIPPED
⚠️ integration-amazon.aws-15 SKIPPED
⚠️ integration-amazon.aws-16 SKIPPED
⚠️ integration-amazon.aws-17 SKIPPED
⚠️ integration-amazon.aws-18 SKIPPED
⚠️ integration-amazon.aws-19 SKIPPED
⚠️ integration-amazon.aws-20 SKIPPED
⚠️ integration-amazon.aws-21 SKIPPED
⚠️ integration-amazon.aws-22 SKIPPED
⚠️ integration-community.aws-1 SKIPPED
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
⚠️ integration-community.aws-14 SKIPPED
⚠️ integration-community.aws-15 SKIPPED
⚠️ integration-community.aws-16 SKIPPED
⚠️ integration-community.aws-17 SKIPPED
⚠️ integration-community.aws-18 SKIPPED
⚠️ integration-community.aws-19 SKIPPED
⚠️ integration-community.aws-20 SKIPPED
⚠️ integration-community.aws-21 SKIPPED
⚠️ integration-community.aws-22 SKIPPED

@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label Nov 29, 2022
@tremble
Copy link
Contributor

tremble commented Nov 29, 2022

Thanks for the fix.

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded (gate pipeline).

✔️ ansible-galaxy-importer SUCCESS in 4m 29s
✔️ build-ansible-collection SUCCESS in 5m 04s
✔️ ansible-test-splitter SUCCESS in 2m 25s
✔️ integration-amazon.aws-1 SUCCESS in 10m 38s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
⚠️ integration-amazon.aws-14 SKIPPED
⚠️ integration-amazon.aws-15 SKIPPED
⚠️ integration-amazon.aws-16 SKIPPED
⚠️ integration-amazon.aws-17 SKIPPED
⚠️ integration-amazon.aws-18 SKIPPED
⚠️ integration-amazon.aws-19 SKIPPED
⚠️ integration-amazon.aws-20 SKIPPED
⚠️ integration-amazon.aws-21 SKIPPED
⚠️ integration-amazon.aws-22 SKIPPED
⚠️ integration-community.aws-1 SKIPPED
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
⚠️ integration-community.aws-14 SKIPPED
⚠️ integration-community.aws-15 SKIPPED
⚠️ integration-community.aws-16 SKIPPED
⚠️ integration-community.aws-17 SKIPPED
⚠️ integration-community.aws-18 SKIPPED
⚠️ integration-community.aws-19 SKIPPED
⚠️ integration-community.aws-20 SKIPPED
⚠️ integration-community.aws-21 SKIPPED
⚠️ integration-community.aws-22 SKIPPED

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit 737ee75 into ansible-collections:main Nov 29, 2022
@tremble tremble added the backport-5 PR should be backported to the stable-5 branch label Nov 29, 2022
@patchback
Copy link

patchback bot commented Nov 29, 2022

Backport to stable-5: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-5/737ee750109dcf3c9132c60db5e440599b4af3bf/pr-1279

Backported as #1289

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback bot pushed a commit that referenced this pull request Nov 29, 2022
…functions in the account (#1279)

lambda - Fix flaky integration tests which assume there are no other functions in the account

SUMMARY
The integration tests for lambda assume that there are no other lambdas in the account.
So when we index into [0], that's not necessarily the lambda we just created.
Note that this fix doesn't work. One of the assertions is failing.
TASK [lambda : lambda_info | Assert successfull retrieval of all information 2] ***
fatal: [testhost]: FAILED! => {
    "assertion": "lambda_infos_all.function[lambda_function_name].versions is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}

That attribute is defined. It's a list of 2 versions. I don't know why.
I'm just creating this PR anyway, so someone else can pickup where I left off.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
lambda_info
ADDITIONAL INFORMATION
Fixes #1277

Reviewed-by: Mark Chappell <None>
Reviewed-by: Matthew Davis <None>
(cherry picked from commit 737ee75)
softwarefactory-project-zuul bot pushed a commit that referenced this pull request Nov 29, 2022
…functions in the account (#1279) (#1289)

[PR #1279/737ee750 backport][stable-5] lambda - Fix flaky integration tests which assume there are no other functions in the account

This is a backport of PR #1279 as merged into main (737ee75).
SUMMARY
The integration tests for lambda assume that there are no other lambdas in the account.
So when we index into [0], that's not necessarily the lambda we just created.
Note that this fix doesn't work. One of the assertions is failing.
TASK [lambda : lambda_info | Assert successfull retrieval of all information 2] ***
fatal: [testhost]: FAILED! => {
    "assertion": "lambda_infos_all.function[lambda_function_name].versions is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}

That attribute is defined. It's a list of 2 versions. I don't know why.
I'm just creating this PR anyway, so someone else can pickup where I left off.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
lambda_info
ADDITIONAL INFORMATION
Fixes #1277
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-5 PR should be backported to the stable-5 branch bug This issue/PR relates to a bug community_review has_issue integration tests/integration mergeit Merge the PR (SoftwareFactory) plugins plugin (any type) tests tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

lambda integration tests - potential test flake
3 participants