15.0.0 does not detect roles/requirements.yml

[ghost]

ISSUE TYPE

• Bug Report

SUMMARY

With the Upgrade to 15.0.0, AWX does not detect the roles/requirements.yml

TASK [detect requirements.yml] *************************************************
skipping: [localhost]
TASK [fetch galaxy roles from requirements.yml] ********************************
skipping: [localhost]
TASK [detect collections/requirements.yml] *************************************
skipping: [localhost]
TASK [fetch galaxy collections from collections/requirements.yml] **************
skipping: [localhost]

ENVIRONMENT

• AWX version: 15.0.0
• AWX install method: docker on linux
• Ansible version: 2.9.13
• Operating System: RHEL 7.8
• Web Browser: Tested with
  • Brave Version 1.14.84 Chromium: 85.0.4183.121
  • Google Chrome 86.0.4240.75
  • Firefox Version 81.0.1

STEPS TO REPRODUCE

Using AWX 15.0.0, sync a project with roles included in roles/requirements.yml

EXPECTED RESULTS

roles/requirements.yml will be included and roles will be pulled from its sources

ACTUAL RESULTS

roles/requirements.yml is skipped and roles will not be pulled from the sources which will result in the job template failing due to mussing roles.

ADDITIONAL INFORMATION

structure of the roles/requirements.yml

---
- src: git@gitlab.domain.tld:ansible/roles/common.git
  scm: git
  name: common
- src: https://github.com/RedHatInsights/insights-client-role.git
  scm: git
  name: RedHatInsights.insights-client
- src: https://github.com/linux-system-roles/timesync.git
  scm: git
  name: rhel-system-roles.timesync
...

Pulling the roles works with this setup in version 14.1.0 as expected.

[ryanpetrello]

@adgerth I'm wondering if this is a bug related to #7817; can you share the results of this API request?

/api/v2/settings/jobs/

...and also (for the Organization associated with the project update you ran):

/api/v2/organizations/X/galaxy_credentials/

[ghost]

@ryanpetrello this is the output from /api/v2/settings/jobs/

{
    "AD_HOC_COMMANDS": [
        "command",
        "shell",
        "yum",
        "apt",
        "apt_key",
        "apt_repository",
        "apt_rpm",
        "service",
        "group",
        "user",
        "mount",
        "ping",
        "selinux",
        "setup",
        "win_ping",
        "win_service",
        "win_updates",
        "win_group",
        "win_user"
    ],
    "ALLOW_JINJA_IN_EXTRA_VARS": "template",
    "AWX_PROOT_BASE_PATH": "/tmp",
    "AWX_PROOT_HIDE_PATHS": [],
    "AWX_PROOT_SHOW_PATHS": [],
    "AWX_ISOLATED_CHECK_INTERVAL": 30,
    "AWX_ISOLATED_LAUNCH_TIMEOUT": 600,
    "AWX_ISOLATED_CONNECTION_TIMEOUT": 10,
    "AWX_ISOLATED_HOST_KEY_CHECKING": false,
    "AWX_RESOURCE_PROFILING_ENABLED": false,
    "AWX_RESOURCE_PROFILING_CPU_POLL_INTERVAL": 0.25,
    "AWX_RESOURCE_PROFILING_MEMORY_POLL_INTERVAL": 0.25,
    "AWX_RESOURCE_PROFILING_PID_POLL_INTERVAL": 0.25,
    "AWX_TASK_ENV": {
        "HOME": "/var/lib/awx"
    },
    "PROJECT_UPDATE_VVV": false,
    "AWX_ROLES_ENABLED": true,
    "AWX_COLLECTIONS_ENABLED": true,
    "AWX_SHOW_PLAYBOOK_LINKS": false,
    "GALAXY_IGNORE_CERTS": false,
    "STDOUT_MAX_BYTES_DISPLAY": 1048576,
    "EVENT_STDOUT_MAX_BYTES_DISPLAY": 1024,
    "SCHEDULE_MAX_JOBS": 10,
    "AWX_ANSIBLE_CALLBACK_PLUGINS": [],
    "DEFAULT_JOB_TIMEOUT": 0,
    "DEFAULT_INVENTORY_UPDATE_TIMEOUT": 0,
    "DEFAULT_PROJECT_UPDATE_TIMEOUT": 0,
    "ANSIBLE_FACT_CACHE_TIMEOUT": 0,
    "MAX_FORKS": 200
}

And here's the output for the galaxy_credentials:

{
    "id": null
}

Let me know if you need any more info on this.

[ryanpetrello]

@adgerth how about this?

/api/v2/credentials/?credential_type__kind=galaxy

One of the things we changed in 15.0.0 was the ability to specify rules per-Organization about where to pull Galaxy-like content from. By default, on upgrade, each org is supposed to have a credential associated with it that represents galaxy.ansible.com, but your install doesn't seem to have that.

[ryanpetrello]

Also, can you exec into one of your containers, and run awx-manage dbshell. You'll get a postgres CLI prompt, and I'd like to know what this returns:

SELECT * FROM conf_setting WHERE key='PUBLIC_GALAXY_ENABLED';

[ghost]

@ryanpetrello here's the output with credential_type__kind=galaxy:

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 2,
            "type": "credential",
            "url": "/api/v2/credentials/2/",
            "related": {
                "created_by": "/api/v2/users/1/",
                "modified_by": "/api/v2/users/1/",
                "activity_stream": "/api/v2/credentials/2/activity_stream/",
                "access_list": "/api/v2/credentials/2/access_list/",
                "object_roles": "/api/v2/credentials/2/object_roles/",
                "owner_users": "/api/v2/credentials/2/owner_users/",
                "owner_teams": "/api/v2/credentials/2/owner_teams/",
                "copy": "/api/v2/credentials/2/copy/",
                "input_sources": "/api/v2/credentials/2/input_sources/",
                "credential_type": "/api/v2/credential_types/18/"
            },
            "summary_fields": {
                "credential_type": {
                    "id": 18,
                    "name": "Ansible Galaxy/Automation Hub API Token",
                    "description": ""
                },
                "created_by": {
                    "id": 1,
                    "username": "root",
                    "first_name": "",
                    "last_name": ""
                },
                "modified_by": {
                    "id": 1,
                    "username": "root",
                    "first_name": "",
                    "last_name": ""
                },
                "object_roles": {
                    "admin_role": {
                        "description": "Can manage all aspects of the credential",
                        "name": "Admin",
                        "id": 22
                    },
                    "use_role": {
                        "description": "Can use the credential in a job template",
                        "name": "Use",
                        "id": 23
                    },
                    "read_role": {
                        "description": "May view settings for the credential",
                        "name": "Read",
                        "id": 24
                    }
                },
                "user_capabilities": {
                    "edit": false,
                    "delete": false,
                    "copy": true,
                    "use": true
                },
                "owners": []
            },
            "created": "2020-10-05T13:38:59.625495Z",
            "modified": "2020-10-05T13:38:59.625524Z",
            "name": "Ansible Galaxy",
            "description": "",
            "organization": null,
            "credential_type": 18,
            "managed_by_tower": true,
            "inputs": {
                "url": "https://galaxy.ansible.com/"
            },
            "kind": "galaxy_api_token",
            "cloud": false,
            "kubernetes": false
        }
    ]
}

And here's the output of the select statement:

awx=# SELECT * FROM conf_setting WHERE key='PUBLIC_GALAXY_ENABLED';
 id | created | modified | key | value | user_id 
----+---------+----------+-----+-------+---------
(0 rows)

[ryanpetrello]

What's odd is that the galaxy credential got created, but was never added to your organization. I'm reading the migration code the see if I can figure out why.

[ryanpetrello]

@adgerth this Organization in question, is it one you've created since upgrading to AWX 15.0.0?

[ghost]

@ryanpetrello yes, this is a testing instance that has been nuked and recreated from scratch. All content in it (organizations, projects ...) has been configured using the awx ansible modules.

Also, I can always re-create this instance if necessary since this host is in a test environment.

[ryanpetrello]

Gotcha. So one of the changes in 15.0.0 is that organizations now have this new endpoint that requires you to configure where content should be pulled from by default e.g.,

POST /api/v2/organizations/N/galaxy_credentials/
{
    "id": 2,
    "associate": true
}

Because we moved this from a global configuration to a per-Org configuration, there's no longer a default global behavior here. Additionally, it's now possible to run your own on-premise Galaxy-like service for synchronizing and maintaining your own content, and we're beginning to nudge users towards this for their usage (and thus, away from the global default of automatically pulling community content from galaxy.ansible.com without first opting in):

https://github.com/ansible/galaxy_ng

Given these changes, you'll want to fold this into your process for automating the creation of organizations.

[ryanpetrello]

Also, I should mention that the latest collection has support for this, and it's easy to get the old behavior out-of-the-box:

https://github.com/ansible/awx/blob/devel/awx_collection/plugins/modules/tower_organization.py#L99-L100

[ryanpetrello]

@adgerth here's a PR that should help clarify this in the release notes; thanks for pointing it out:

#8376