repository errors out when trying to update a repository with unsupported flags

[pcfe]

SUMMARY

Satellite 6.7.3 throws the following on the second run of a task that tries to ensure a repo is present

"param is missing or the value is empty: repository"

ISSUE TYPE

• Bug Report

ANSIBLE VERSION

pcfe@t3600 pcfe.net (master) $ ansible --version
ansible 2.9.13
  config file = /home/pcfe/work/git/HouseNet/ansible/pcfe.net/ansible.cfg
  configured module search path = ['/home/pcfe/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.8/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.5 (default, Aug 12 2020, 00:00:00) [GCC 10.2.1 20200723 (Red Hat 10.2.1-1)]

KATELLO/FOREMAN VERSION

pcfe@satellite ~ $ rpm -q tfm-rubygem-katello foreman
tfm-rubygem-katello-3.14.0.27-1.el7sat.noarch
foreman-1.24.1.25-1.el7sat.noarch

APYPIE VERSION

pcfe@t3600 pcfe.net (master) $ pip show apypie | grep Version
Version: 0.2.2

STEPS TO REPRODUCE

try to run a playbook containing this task twice (first run works)

    - name: "Ensure Repository for soe-ci generated RHEL 7 RPMs exists"
      theforeman.foreman.repository:
        username:     "{{ sat_username }}"
        password:     "{{ vaulted_sat6_pass }}"
        server_url:   "{{ sat_server_url }}"
        organization: "{{ sat_org }}"
        auto_enabled:       no
        checksum_type:      sha256
        content_type:       yum
        download_policy:    immediate
        mirror_on_sync:     yes
        label:              repo-soe-rpms-rhel7
        name:               repo-soe-rpms-rhel7
        product:            "prd-{{ cu_dept }}-soe"
        state:              present
        unprotected:        no
        url:                "{{ repo_url_rpms_rhel7 }}"
        verify_ssl_on_sync: "{{ repo_verify_ssl }}"

EXPECTED RESULTS

since the repo exists from first run, on subsequent runs I would like it to behave as defined by state:, so in this case override any manual changes someone might have done in the Satellite webUI.

Well and of course I want the playbook to be idempotent. :-)

ACTUAL RESULTS

on runs where the playbook would not need to change anything, I get with -vvv

TASK [Ensure Repository for soe-ci generated RHEL 7 RPMs exists] *********************************************************************************************************************************************************
task path: /home/pcfe/work/git/HouseNet/ansible/pcfe.net/satellite-FAM-create-soe-ci-CVs.yml:39
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: pcfe
<127.0.0.1> EXEC /bin/sh -c 'echo ~pcfe && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/pcfe/.ansible/tmp `"&& mkdir "` echo /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066 `" && echo ansible-tmp-1600868473.3831494-93673-81858291854066="` echo /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066 `" ) && sleep 0'
Using module file /home/pcfe/.ansible/collections/ansible_collections/theforeman/foreman/plugins/modules/repository.py
<127.0.0.1> PUT /home/pcfe/.ansible/tmp/ansible-local-93583j0frge0d/tmpx7pa72hn TO /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066/AnsiballZ_repository.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066/ /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066/AnsiballZ_repository.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066/AnsiballZ_repository.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/pcfe/.ansible/tmp/ansible-tmp-1600868473.3831494-93673-81858291854066/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_theforeman.foreman.repository_payload_8m06ix8s/ansible_theforeman.foreman.repository_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py", line 818, in resource_action
    result = self._resource_call(resource, action, resource_payload, options=options, data=data, files=files)
  File "/tmp/ansible_theforeman.foreman.repository_payload_8m06ix8s/ansible_theforeman.foreman.repository_payload.zip/ansible_collections/theforeman/foreman/plugins/module_utils/foreman_helper.py", line 446, in _resource_call
    return self._resource(resource).call(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/apypie/resource.py", line 30, in call
    return self.api.call(self.name, action, params, headers, options, data, files)
  File "/usr/lib/python3.8/site-packages/apypie/api.py", line 189, in call
    return self._call_action(action, params, headers, data, files)
  File "/usr/lib/python3.8/site-packages/apypie/api.py", line 194, in _call_action
    return self.http_call(
  File "/usr/lib/python3.8/site-packages/apypie/api.py", line 224, in http_call
    request.raise_for_status()
  File "/usr/lib/python3.8/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "error": {
        "displayMessage": "Missing values for repository.",
        "errors": [
            "param is missing or the value is empty: repository"
        ]
    },
    "invocation": {
        "module_args": {
            "ansible_collection_requirements": null,
            "auto_enabled": false,
            "checksum_type": "sha256",
            "content_type": "yum",
            "deb_architectures": null,
            "deb_components": null,
            "deb_errata_url": null,
            "deb_releases": null,
            "description": null,
            "docker_tags_whitelist": null,
            "docker_upstream_name": null,
            "download_policy": "immediate",
            "gpg_key": null,
            "http_proxy": null,
            "http_proxy_policy": null,
            "ignorable_content": null,
            "ignore_global_proxy": null,
            "label": "repo-soe-rpms-rhel7",
            "mirror_on_sync": true,
            "name": "repo-soe-rpms-rhel7",
            "organization": "Sat Test",
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "product": "prd-ACME-soe",
            "server_url": "https://satellite.internal.pcfe.net",
            "ssl_ca_cert": null,
            "ssl_client_cert": null,
            "ssl_client_key": null,
            "state": "present",
            "unprotected": false,
            "upstream_password": null,
            "upstream_username": null,
            "url": "http://jenkins.internal.pcfe.net/pub/soe-repo/rhel7/",
            "username": "admin",
            "validate_certs": true,
            "verify_ssl_on_sync": false
        }
    },
    "msg": "Error while performing update on repositories: 400 Client Error: Bad Request for url: https://satellite.internal.pcfe.net/katello/api/repositories/3824"
}

and with --check --diff I get

TASK [Ensure Repository for soe-ci generated RHEL 7 RPMs exists] *********************************************************************************************************************************************************
--- before
+++ after
@@ -1,10 +1,10 @@
 {
     "repositories": [
         {
+            "auto_enabled": false,
             "checksum_type": "sha256",
             "content_type": "yum",
             "download_policy": "immediate",
-            "http_proxy_id": null,
             "http_proxy_policy": "global_default_http_proxy",
             "id": 3824,
             "label": "repo-soe-rpms-rhel7",
@@ -12,9 +12,6 @@
             "name": "repo-soe-rpms-rhel7",
             "organization_id": 1,
             "product_id": 320,
-            "ssl_ca_cert_id": null,
-            "ssl_client_cert_id": null,
-            "ssl_client_key_id": null,
             "unprotected": false,
             "url": "http://jenkins.internal.pcfe.net/pub/soe-repo/rhel7/",
             "verify_ssl_on_sync": false

changed: [localhost]

additional info

here's the fulll playbook in case you wanna see it. Any comment welcome (just in case I do sth stupid in it)
this is part of my effort to modernise my soe-ci docs from "do this and that with hammer" to "run this or that playbook".

---
# c.f. https://github.com/theforeman/foreman-ansible-modules
# AND https://theforeman.org/plugins/foreman-ansible-modules/

# my Ansible control node connects to the Satellite server.
# Purely personal preference to run the tasks on another box.
- hosts: localhost

  # put your vars wherever is comforable for you, a Tower, an AWX, a file automatically picked up by Ansible, a file explicitly specified, in this playbook or a mix. Your call, choose what fits your environment best.
  vars_files:
    - "vars/satellite-secrets.yml"

  vars:
    sat_server_url: "https://satellite.internal.pcfe.net"
    sat_username: "admin"
    sat_password: "{{ vaulted_sat6_pass }}"
    sat_org: "Sat Test"
    cu_dept: "ACME"  # Do NOT use spaces, the content of this var is part of labels in Satellite
    repo_url_rpms_rhel7: "http://jenkins.internal.pcfe.net/pub/soe-repo/rhel7/"
    repo_url_rpms_rhel8: "http://jenkins.internal.pcfe.net/pub/soe-repo/rhel8/"
    repo_verify_ssl:     no

  tasks:
    - name: "Ensure Product used by soe-ci exists and is current in Satellite"
      theforeman.foreman.product:
        username:     "{{ sat_username }}"
        password:     "{{ vaulted_sat6_pass }}"
        server_url:   "{{ sat_server_url }}"
        organization: "{{ sat_org }}"
        name:         "prd-{{ cu_dept }}-soe"
        description:  "Used by soe-ci, managed by Ansible. Do not automatically sync, soe-ci handles syncing."
        state:        present

    - name: "Ensure Repository for soe-ci generated RHEL 7 RPMs exists"
      theforeman.foreman.repository:
        username:     "{{ sat_username }}"
        password:     "{{ vaulted_sat6_pass }}"
        server_url:   "{{ sat_server_url }}"
        organization: "{{ sat_org }}"
        auto_enabled:       no
        checksum_type:      sha256
        content_type:       yum
        download_policy:    immediate
        mirror_on_sync:     yes
        label:              repo-soe-rpms-rhel7
        name:               repo-soe-rpms-rhel7
        product:            "prd-{{ cu_dept }}-soe"
        state:              present
        unprotected:        no
        url:                "{{ repo_url_rpms_rhel7 }}"
        verify_ssl_on_sync: "{{ repo_verify_ssl }}"

    - name: "Ensure Repository for soe-ci generated RHEL 8 RPMs exists"
      theforeman.foreman.repository:
        username:     "{{ sat_username }}"
        password:     "{{ vaulted_sat6_pass }}"
        server_url:   "{{ sat_server_url }}"
        organization: "{{ sat_org }}"
        auto_enabled:     no
        checksum_type:    sha256
        content_type:     yum
        download_policy:  immediate
        mirror_on_sync:   yes
        label:            repo-soe-rpms-rhel8
        name:             repo-soe-rpms-rhel8
        product:          "prd-{{ cu_dept }}-soe"
        state:            present
        unprotected:      no
        url:              "{{ repo_url_rpms_rhel8 }}"
        verify_ssl_on_sync: "{{ repo_verify_ssl }}"

[evgeni]

as we discussed that before, here for everyone to read: the problem is that the module compares the existing repository to what you told it to create and sees a diff (as seen in the diff output of check mode), but those parameters aren't part of your katello version, so the api-parameter-cleanup step wipes them and we submit an empty json, which the API correctly rejects.

[evgeni]

@pcfe could you do me a favor and test if the issue still persists if you remove the auto_enabled: no line from your playbook?

[evgeni]

nvm, I can repro that on my sat 6.6 and removing auto_enabled (which 6.6 doesn't support) fixes it.

okay, now, how do I remove that automatically?

[pcfe]

confirmed, removing auto_enabled: no works around the issue. I was able to execute my playbook thrice in a row. Thanks!