requirements.yml is ignored on project sync

[al-lac]

ISSUE TYPE

• Bug Report

SUMMARY

ENVIRONMENT

• AWX version: 16.0.0
• AWX install method: docker on linux
• Ansible version: 2.9
• Operating System: CentOS 8
• Web Browser: -

STEPS TO REPRODUCE

• Create a repository with a roles/requirements.yml
• Add the Repository as a Git Project on AWX
• Sync the Project

EXPECTED RESULTS

• AWX should sync the dependencies from the requirements.yml file

ACTUAL RESULTS

• requirements.yml sync is skipped

TASK [detect roles/requirements.(yml/yaml)] ************************************
skipping: [localhost] => (item={'ext': '.yml'}) 
skipping: [localhost] => (item={'ext': '.yaml'}) 

ADDITIONAL INFORMATION

I tried to sync a repository with a requirements.yml and it is always skipped. Tried it with different AWX Versions (15.0.1, 16.0) and different git repos (Bitbucket, Gitlab, GitHub). Maybe the format of my requirements.yml is wrong, but doing a ansible-galaxy command on the awx server works just fine.

requirements.yml:

---
- src: git+https://github.com/al-lac/ansible-role-chrony.git
  version: master

or:

---
- src: git+https://oauth:<key>@gitlab.local/al-lac/ansible-role-chrony.git
  version: master

[supaflyster]

just my 5cents, but as far as i remember requirements are synced only on job execution. Sio you have to trigger a job from a template, and then awx will pull the requirements. Syncing project skips it.

[al-lac]

I think it needs to be synced with the project, otherwise i get the following error when trying to run templates:

ERROR! the role 'ansible-role-chrony' was not found in /tmp/awx_2_wihiv2zf/project/roles:/tmp/awx_2_wihiv2zf/requirements_roles:/var/lib/awx/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/tmp/awx_2_wihiv2zf/project
The error appears to be in '/tmp/awx_2_wihiv2zf/project/chrony.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
  roles:
    - ansible-role-chrony
      ^ here

[supaflyster]

You have in your role meta https://github.com/al-lac/ansible-role-chrony/blob/master/meta/main.yml
role_name: chrony
and you use it as
roles:
- ansible-role-chrony

So it can't obviously find it.
You either put ansible-role-chrony in meta or specify name: ansible-role-chrony in requirements.yml

[al-lac]

Ah yeah i am not using that repository. I am using my gitlab server for that. I changed both repositories to use my gitlab server now, changed the name in meta and requirements.yml. But still get the same error.
Both repositories available here:
https://gitlab.iamroot.at/al-lac/ansible-role-chrony
https://gitlab.iamroot.at/root/ansible-base-roles

[KornKalle]

Can confirm this behaviour.
Did a clean 16.0.0 install today and both requirements.yml in collections/ and roles/ getting skipped.

It gets skipped on project sync and when starting a job it also will fail with the same error message as shown above.
This leads to an unusable behaviour for us, as we can't even use community.general collection.

[al-lac]

Hi @KornKalle, what is the last version it worked for you? I also tried it with version 15 and got the same behaviour.

[KornKalle]

Last working Version for me was 11.1.0

[supaflyster]

We are using 15.0.1 (started with 1.0) and it was always working. I've just spun up clean 15.0.1 in vagrant, and added https://gitlab.iamroot.at/root/ansible-base-roles/ as a project, and run a template with ansible-role-chrony.yml playbook, targeting localhost and it downloaded chrony role and started executing it.

[al-lac]

I just added the project and template exactly like you did on one of our 15.0.1 instances. I still get the error that the role was not found when starting the template. We used the docker compose install method for this server.

[KornKalle]

Same still for me. It was still working pre-upgrade on 11.1.0, so I would rather like to get some information how we can reproduce this behavior.
I also used the docker compose install method on Debian Buster with docker 19.03.12

[ghost]

From 15.0 onwards, there has been a change made to how the sync works (instead of global to per-org), by default an org has no credentials hence it does not honor requirements.yml, you might want to try creating a dummy galaxy credential (it's required even if you don't use galaxy) and associating it to your org via the API as mentioned here #8341

[al-lac]

@constreference thanks a lot! That solved it for me.

I had to create a new (dummy) galaxy credential and associate it with my org via the api.

Get galaxy credential:

GET /api/v2/credentials/?credential_type__kind=galaxy
{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 11,
            "type": "credential",
            "url": "/api/v2/credentials/11/",
            "related": {
                "created_by": "/api/v2/users/8/",
                "modified_by": "/api/v2/users/8/",
                "organization": "/api/v2/organizations/2/",

Associate galaxy credential to org (returns 204 on success):

POST /api/v2/organizations/2/galaxy_credentials/
{
    "id": 11,
    "associate": true
}

This change should really be better documented somewhere. An option to do the association via the UI should be implemented soon as well.

[AlanCoding]

@ryanpetrello can you review the solution here and triage this issue? This could go in a couple directions ui_next/installer/etc.

[ryanpetrello]

@AlanCoding yep, the need to explicitly associate a public Galaxy credential is a change we made in a recent version of awx - and @al-lac's comment is an example of how to do that correctly.

This change should really be better documented somewhere.

It's explicitly called out as the top line item in the awx changelog for the version where this changed:

faf0fa9
#8341 (comment)

[ryanpetrello]

Duplicate of #8341, see #8341 (comment)