Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parameters with aliases not being passed #92

Merged
merged 3 commits into from
Oct 12, 2021

Conversation

gravesm
Copy link
Member

@gravesm gravesm commented Oct 11, 2021

Depends-On: ansible/ansible-zuul-jobs#1176

SUMMARY

The recently added _is_an_alias() function should check whether the
current key is in the list of aliases.

Fixes: #91

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME
ADDITIONAL INFORMATION

The recently added _is_an_alias() function should check whether the
current key is in the list of aliases.
@tadeboro
Copy link
Contributor

tadeboro commented Oct 11, 2021

I did play a bit with this in xlab-steampunk@5b72fcb and once I wrote that last unit test, I knew things will not be as easy to solve as they looked like. We cannot just drop any aliased parameter because we can drop the only piece of information when doing so.

I did not work any further on this because I do not know why all that argument handling is there and would take me too long to figure things out. And the fact that the turbo code needs three fork calls to do the double-fork daemonization dance did not fill me with confidence ;)

Edit: PR I experimented in is #90

Copy link

@ansible-zuul ansible-zuul bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@goneri goneri closed this Oct 12, 2021
@goneri goneri reopened this Oct 12, 2021
@goneri goneri removed the gate label Oct 12, 2021
@goneri
Copy link
Member

goneri commented Oct 12, 2021

Closing/opening because I had to manually push a missing commit.

@gravesm
Copy link
Member Author

gravesm commented Oct 12, 2021

@goneri I think this one is going to fail until ansible-collections/kubernetes.core#261 gets merged.

@goneri
Copy link
Member

goneri commented Oct 12, 2021

@goneri I think this one is going to fail until ansible-collections/kubernetes.core#261 gets merged.

Should I disable ansible-test-molecule-kubernetes-core-with-turbo until we land this PR?

@gravesm
Copy link
Member Author

gravesm commented Oct 12, 2021

Yeah, let's do that.

@goneri
Copy link
Member

goneri commented Oct 12, 2021

recheck

@goneri goneri added the gate label Oct 12, 2021
ansible-zuul bot pushed a commit to ansible/ansible-zuul-jobs that referenced this pull request Oct 12, 2021
cloud.common: temp turn k8s check non voting

Temporarily disable ansible-test-molecule-kubernetes-core-with-turbo until
we land ansible-collections/cloud.common#92.

Reviewed-by: None <None>
@goneri goneri added gate and removed gate labels Oct 12, 2021
@tadeboro
Copy link
Contributor

If I understand things correctly, this fix is not OK. What if we only see one of the aliases being set? Test case I came up with when I was playing with this: https://github.com/xlab-steampunk/cloud.common/blob/5b72fcba614754bc759c2b85c86a2573f035a151/tests/unit/module_utils/test_turbo_module.py#L149-L155

I am pretty sure this test will fail since argument processing will throw away the bar value.

@gravesm
Copy link
Member Author

gravesm commented Oct 12, 2021

@tadeboro The arg spec in your example doesn't look like a legitimate arg spec. It is declaring bar as an alias to foo and then also declaring it as a param with itself as an alias. In fact, when I try to do this, ansible sanity complains:

ERROR: plugins/modules/k8s_log.py:0:0: parameter-alias-self: Argument 'bar' in argument_spec is specified as its own alias
ERROR: plugins/modules/k8s_log.py:0:0: parameter-documented-multiple-times: Argument 'foo' in argument_spec with aliases 'bar' is documented multiple times, namely as 'bar', 'foo'```

@ansible-zuul ansible-zuul bot merged commit 7bad786 into ansible-collections:main Oct 12, 2021
@tadeboro
Copy link
Contributor

@tadeboro The arg spec in your example doesn't look like a legitimate arg spec. It is declaring bar as an alias to foo and then also declaring it as a param with itself as an alias.

That is indeed not a valid argspec, but the prepare_args function does not take a regular argspec as an input. Instead, it takes an "expanded" argspec, produced by the expand_argument_specs_aliases. And those things do look like the test cases I prepared

When you updated the tests in this PR, you actually made them useless because they operate on data structures they will never encounter in the wild.

@gravesm
Copy link
Member Author

gravesm commented Oct 12, 2021

I think the issue is that the expand_argument_spec_aliases function is not necessary, unless I'm missing something. By the time it runs, the value for any alias that's been specified in the task will have been copied to the actual param, so any alias can be discarded. @goneri may know more about why it's there.

@tadeboro
Copy link
Contributor

Well, if you pass a regular argspec to the prepare_args, all calls to the _is_an_alias will return back False. So if the extended argument specification is not needed, we can also get rid of the alias check.

As I said before, I ran out of time before I was able to determine why exactly argument manipulation is needed, but my gut was telling me that the init_args call at

args = self.init_args()
could be replaced by a simple args = ansible.module_utils.basic._ANSIBLE_ARGS. This would also simplify the server implementation a lot, but I am assuming that I am missing some important pieces of information original author of the code considered when writing the code.

@goneri
Copy link
Member

goneri commented Oct 12, 2021

When I did that, the arguments AND their aliases were exposed in _ANSIBLE_ARGS. This was problematical when the value was used as argument_spec within the background process.
This _ANSIBLE_ARGS variable is not a public API and its behavior/content may change between two minor versions of Ansible. Also, there is maybe a better way to implement that.

@tadeboro
Copy link
Contributor

When I did that, the arguments AND their aliases were exposed in _ANSIBLE_ARGS.

The _ANSIBLE_ARGS global contains whatever was passed to module's stdin (or what came in the arguments file). If the arguments and aliases were both present there, this was an end-user error and ansible should throw a tantrum.

This was problematical when the value was used as argument_spec within the background process. This _ANSIBLE_ARGS variable is not a public API and its behavior/content may change between two minor versions of Ansible.

Well, while the _ANSIBLE_ARGS is not part of the public API, I do not expect it to be gone anytime soon because it is described in the docs as the official way of passing arguments to modules in unit tests. Removing it now would break a lot of existing code. And the same goes with the behavior change: since the official docs describe what should be there, I find it highly unlikely that the current behavior will change. Besides, the server code also relies on manipulating the _ANSIBLE_ARGS variable, which means that turbo modules already use that private API.

Also, there is maybe a better way to implement that.

I do not doubt this. But in my opinion, perfection should not stand in the way of progress and getting rid of argument mangling would be one large step forward in paying back some technical debt in the turbo module implementation.

ansible-zuul bot pushed a commit to ansible-collections/kubernetes.core that referenced this pull request Oct 13, 2021
Remove molecule dependencies

SUMMARY

Depends-on: ansible-collections/cloud.common#92
Molecule is overwriting the cloud.common dependency installed by zuul,
which is causing issues with the CI job for turbo mode. We still need to
find a way to test against the latest released version of cloud.common.

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Gonéri Le Bouder <goneri@lebouder.net>
Reviewed-by: None <None>
gravesm added a commit to gravesm/kubernetes.core that referenced this pull request Oct 13, 2021
Remove molecule dependencies

SUMMARY

Depends-on: ansible-collections/cloud.common#92
Molecule is overwriting the cloud.common dependency installed by zuul,
which is causing issues with the CI job for turbo mode. We still need to
find a way to test against the latest released version of cloud.common.

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Gonéri Le Bouder <goneri@lebouder.net>
Reviewed-by: None <None>
(cherry picked from commit ff43353)
gravesm added a commit to ansible-collections/kubernetes.core that referenced this pull request Oct 14, 2021
Remove molecule dependencies

SUMMARY

Depends-on: ansible-collections/cloud.common#92
Molecule is overwriting the cloud.common dependency installed by zuul,
which is causing issues with the CI job for turbo mode. We still need to
find a way to test against the latest released version of cloud.common.

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Gonéri Le Bouder <goneri@lebouder.net>
Reviewed-by: None <None>
(cherry picked from commit ff43353)
ansible-zuul bot pushed a commit to ansible/ansible-zuul-jobs that referenced this pull request Oct 18, 2021
Revert "cloud.common: temp turn k8s check non votin (#1176)"

Depends-On: ansible-collections/cloud.common#92
Depends-On: ansible-collections/kubernetes.core#261
This reverts commit 20f0173 introduced by #1176.

Reviewed-by: None <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Latest release broke parameters with aliases
3 participants