gitlab_runner can't handle more than 20 runners

[hazzeh]

Summary

This looks like regression of ansible/ansible#57759

When register runners on a system with more than 20 runners an existing runner will not be updated, but a new will be registered.

It looks like other gitlab modules were updated with #4491

By updating the find_runner(self, description) function from
runners = self._runners_endpoint(as_list=False)
to
runners = self._runners_endpoint(as_list=False, all=True)
it solves my problem.

Issue Type

Bug Report

Component Name

gitlab_runner

Ansible Version

$ ansible --version
ansible [core 2.16.1]
  config file = None
  configured module search path = ['/<redacted>/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /<redacted>/venv/lib/python3.11/site-packages/ansible
  ansible collection location = /<redacted>/.ansible/collections:/usr/share/ansible/collections
  executable location = /<redacted>/venv/bin/ansible
  python version = 3.11.6 (main, Oct  2 2023, 13:45:54) [GCC 13.2.0] (/<redacted>/venv/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

Community.general Version

$ ansible-galaxy collection list community.general
# /<redacted>/venv/lib/python3.11/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 8.1.0

Configuration

$ ansible-config dump --only-changed
ANSIBLE_FORCE_COLOR(env: ANSIBLE_FORCE_COLOR) = True
CONFIG_FILE() = None
EDITOR(env: EDITOR) = vim
PAGER(env: PAGER) = less

OS / Environment

Ubuntu 20.04.6 LTS

Steps to Reproduce

1. Create a project in gitlab.com.
2. Do not allow shared runners
3. Create a personal access token with api and create_runner scopes.
4. Run the following play twice.

- hosts: localhost
  tasks:
   - community.general.gitlab_runner:
       api_url: https://gitlab.com/
       api_token: <api-token>
       description: "runner-{{ item }}"
       state: present
       project: <your project id>
     loop: "{{ range(0, 21) }}"

Expected Results

The second run should be idempotent.

Actual Results

The first 20 runners is working as expected, the 21th is newly registered causing duplicate runners in gitlab.

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/modules/gitlab_runner.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @Lunik @SamyCoenen @Shaps @lgatellier @marwatk @metanovii @nejch @scodeman @sh0shin @suukit @waheedi @zanssa
click here for bot help

[nejch]

This happens probably if you use python-gitlab v4.x.x because as_list=False was replaced byiterator=True.

The best way would be to handle this based on the python-gitlab version ( runners = self._runners_endpoint(as_list=False, all=True) means a no-op for as_list in v4, but no-op for all=True in v3). And setting per_page=100 on the gitlab instance for all modules would probably also help reduce API calls here.

[mikewadsten]

See this comment I left over in #7790 regarding using all=True.

tl;dr - as_list=False, all=True would work, but it would mean that on python-gitlab >= 3.6.0, we'd always be eagerly retrieving all the runners into a list. Unfortunately we can't just do as_list=False, iterator=True because from 3.6.0 to 3.15.0, python-gitlab errors out if you try to pass both.

Edit: I think @nejch and I are actually in agreement on the best way to solve this:

The best way would be to handle this based on the python-gitlab version

[nejch]

See this comment I left over in #7790 regarding using all=True.

tl;dr - as_list=False, all=True would work, but it would mean that on python-gitlab >= 3.6.0, we'd always be eagerly retrieving all the runners into a list. Unfortunately we can't just do as_list=False, iterator=True because from 3.6.0 to 3.15.0, python-gitlab errors out if you try to pass both.

Edit: I think @nejch and I are actually in agreement on the best way to solve this:

The best way would be to handle this based on the python-gitlab version

@mikewadsten yes, this is what I meant- provide the arguments based on the python-gitlab version (or only support v4+ if that's easier).

[lgatellier]

I think we can close this issue, since list() arguments have been fixed by #7790.

resolved_by_pr 7790