awx.awx.tower_export missing Galaxy Credential in organizations #9342

metabsd · 2021-02-17T19:17:05Z

ISSUE TYPE

Bug Report

SUMMARY

awx.awx.tower_export is not exporting Galaxy Credential with the Organization

ENVIRONMENT

AWX version: 15.0.0, 15.0.1, 17.0.1
AWX install method: AWX Operator
Ansible version: 2.9.17
Operating System: docker.io/ansible/awx:{VERSIONS}
Web Browser: Edge, Chrome, Firefox

STEPS TO REPRODUCE

---
- name: awx assets export
  hosts: localhost
  debugger: on_failed
  gather_facts : false
  become: false
  tasks:
    - name: export job templates
      awx.awx.tower_export:
        all: True
        tower_host: awx.mydomain
        tower_username: admin
        tower_password: "{{ my_admin_password }}"
        validate_certs: no
      register: export_results

    - copy: content="{{ export_results.assets }}" dest=./awx_export.json

My organization have a Galaxy Credential but the export don't export it. So I finally figure out by redoing it manually and comparing.

An example of the Organization Export.

  "organizations": [
    {
      "name": "MyOrg",
      "description": "",
      "max_hosts": 0,
      "custom_virtualenv": null,
      "related": {
        "notification_templates": [],
        "notification_templates_started": [],
        "notification_templates_success": [],
        "notification_templates_error": [],
        "notification_templates_approvals": []
      },
      "natural_key": {
        "name": "MyOrg",
        "type": "organization"
      }
    }
  ]

EXPECTED RESULTS

I expect that the export will also export the organization with the galaxy credential to be able to download and install collection automatically in a project sync.

ACTUAL RESULTS

Missing Galaxy Credential in my Organization.

The text was updated successfully, but these errors were encountered:

sean-m-sullivan · 2021-02-17T22:18:42Z

I just tested this using awxkit 17.0.1 , awx, 17.0.1 and tower 3.8.1. and the galaxy credential does not show up.
It should show up under related, as it does on the awx api.

{
    "all_assets": {
        "assets": {
            "organizations": [
                {
                    "custom_virtualenv": null,
                    "description": "",
                    "max_hosts": 0,
                    "name": "Default",
                    "natural_key": {
                        "name": "Default",
                        "type": "organization"
                    },
                    "related": {
                        "notification_templates": [],
                        "notification_templates_approvals": [],
                        "notification_templates_error": [],
                        "notification_templates_started": [],
                        "notification_templates_success": []
                    }
                }
            ]
        },
        "changed": false,
        "failed": false
    }
}

jainnikhil30 · 2021-03-16T07:50:52Z

@metabsd @sean-m-sullivan

I think this is expected behavior. You are probably looking to export the default Galaxy credential which gets created when you install awx/tower. Tower_export module will not export the "default" galaxy credential.

awx/awxkit/awxkit/api/pages/api.py

Line 85 in 6aab882

if _page.json.get('managed_by_tower'):

The default galaxy credential is "managed_by_tower" and hence it wont be exported.

But if you create another galaxy credential and try to do the export you should see it. I tested it and I do see it getting exported.

{
                "credential_type": {
                    "kind": "galaxy",
                    "name": "Ansible Galaxy/Automation Hub API Token",
                    "type": "credential_type"
                },
                "description": "",
                "inputs": {
                    "url": "ansible.galaxy.com"
                },
                "name": "Galaxy",
                "natural_key": {
                    "credential_type": {
                        "kind": "galaxy",
                        "name": "Ansible Galaxy/Automation Hub API Token",
                        "type": "credential_type"
                    },
                    "name": "Test Galaxy Credential",
                    "organization": {
                        "name": "Default",
                        "type": "organization"
                    },
                    "type": "credential"
                },
                "organization": {
                    "name": "Default",
                    "type": "organization"
                }
            },

sean-m-sullivan · 2021-03-16T19:19:37Z

@jainnikhil30 we know it comes out under credentials as a credential, we were referring to Organizations,
it should show up here

{
"organizations": [
    {
      "name": "Foo_cred",
      "description": "",
      "max_hosts": 0,
      "custom_virtualenv": null,
      "related": {
        "notification_templates": [
          
        ],
        "notification_templates_started": [
          
        ],
        "notification_templates_success": [
          
        ],
        "notification_templates_error": [
          
        ],
        "notification_templates_approvals": [
          
        ]
      },
      "natural_key": {
        "name": "Foo_cred",
        "type": "organization"
      }
    },
  ],

and in credentials it would be:

    "credentials": [
      {
        "name": "foo galaxy",
        "description": "",
        "inputs": {
          "url": "http://localhost",
          "token": ""
        },
        "organization": {
          "name": "Foo_cred",
          "type": "organization"
        },
        "credential_type": {
          "name": "Ansible Galaxy/Automation Hub API Token",
          "kind": "galaxy",
          "type": "credential_type"
        },
        "natural_key": {
          "organization": {
            "name": "Foo_cred",
            "type": "organization"
          },
          "name": "foo galaxy",
          "credential_type": {
            "name": "Ansible Galaxy/Automation Hub API Token",
            "kind": "galaxy",
            "type": "credential_type"
          },
          "type": "credential"
        }
      },
}

Notice that org Foo_cred doesn't have foo galaxy under it.
however on page
https://localhost/api/v2/organizations/2/

{
    "id": 2,
    "type": "organization",
    "url": "/api/v2/organizations/2/",
    "related": {
        "notification_templates": "/api/v2/organizations/2/notification_templates/",
        "notification_templates_started": "/api/v2/organizations/2/notification_templates_started/",
        "notification_templates_success": "/api/v2/organizations/2/notification_templates_success/",
        "notification_templates_error": "/api/v2/organizations/2/notification_templates_error/",
        "notification_templates_approvals": "/api/v2/organizations/2/notification_templates_approvals/",
        "galaxy_credentials": "/api/v2/organizations/2/galaxy_credentials/"
     }
}

is more of this on the output of tower export

{
    "organizations": [
        {
          "name": "Foo_cred",
          "description": "",
          "max_hosts": 0,
          "custom_virtualenv": null,
          "related": {
            "notification_templates": [
              
            ],
            "notification_templates_started": [
              
            ],
            "notification_templates_success": [
              
            ],
            "notification_templates_error": [
              
            ],
            "notification_templates_approvals": [
              
            ],
            "galaxy_credentials": [
                {
                    "name": "foo galaxy"
                  }
            ]
          },
          "natural_key": {
            "name": "Foo_cred",
            "type": "organization"
          }
        },
      ],
 }

This last example compared to the first one I showed has galaxy_credentials as a related field, which is what is being advocated for.

jainnikhil30 · 2021-03-17T04:19:45Z

@sean-m-sullivan

My bad, thanks for the clarification. I didn't notice it was organization. Yeah this seems like a bug !

jainnikhil30 · 2021-03-17T07:27:39Z

@sean-m-sullivan

I think I have a potential fix. With the fix:

ok: [localhost] => {
    "assets": {
        "organizations": [
            {
                "custom_virtualenv": null,
                "description": "",
                "max_hosts": 0,
                "name": "Default",
                "natural_key": {
                    "name": "Default",
                    "type": "organization"
                },
                "related": {
                    "credentials": [
                        {
                            "credential_type": {
                                "kind": "cloud",
                                "name": "Amazon Web Services",
                                "type": "credential_type"
                            },
                            "name": "aws",
                            "organization": {
                                "name": "Default",
                                "type": "organization"
                            },
                            "type": "credential"
                        },
                        {
                            "credential_type": {
                                "kind": "galaxy",
                                "name": "Ansible Galaxy/Automation Hub API Token",
                                "type": "credential_type"
                            },
                            "name": "Galaxy",
                            "organization": {
                                "name": "Default",
                                "type": "organization"
                            },
                            "type": "credential"
                        }
                    ],
                    "notification_templates": [],
                    "notification_templates_approvals": [],
                    "notification_templates_error": [],
                    "notification_templates_started": [],
                    "notification_templates_success": []
                }
            }
        ]
    },

sean-m-sullivan · 2021-03-17T07:32:35Z

Good start, but think it should be its own related. Because on post back in api, tower import, for organizations, galaxy credentials is its own field, like in tower_organization module

Credentials as you listed wouldn't be associated as "the" galaxy credential in the org, just related Credentials in general

If you can point me to your code change for above, I'd be glad to digg into it and see about doing as I said, personally just don't know where to start

shanemcd · 2021-04-10T14:48:05Z

@sean-m-sullivan @jainnikhil30 Any updates on this one?

sean-m-sullivan · 2021-04-10T16:00:12Z

I unfortunately have not been able to decode awxkit to figure out how to make changes, though if I could see the diff on @jainnikhil30 fix above, I might possibly be able to do so. I will be away for most all of next week and may revisit it once I return.

sean-m-sullivan · 2021-04-27T20:13:32Z

@john-westcott-iv Do you have any movement on this, or point me to where we define these things in the files so I could take a crack at it.

john-westcott-iv · 2021-04-28T12:30:07Z

@sean-m-sullivan I just took a quick peak at the code but check out the EXPORTABLE_RELATIONS.

awx/awxkit/awxkit/api/pages/api.py

Line 30 in 6aab882

EXPORTABLE_RELATIONS = [

It looks like it should be exporting credential types but maybe the galaxy credential is a slightly different class?

awx/awxkit/awxkit/api/pages/api.py

Line 133 in 6aab882

is_relation = rel.__class__.__name__ in EXPORTABLE_RELATIONS

sean-m-sullivan · 2021-05-04T15:56:44Z

So have been digging around, going to post my notes here, I am not quite to the bottom of why this is, but while evaluating line 133, the _create is not finding the related field like it normally would, because galaxy_credentials doesn't exist on the main page, as its under credentials.

126
log.error("key %r is_relation %s.", key, rel_endpoint)
key 'instance_groups' is_relation /api/v2/organizations/1/instance_groups/.
133
log.error("class %r export %s.", rel.__class__, EXPORTABLE_RELATIONS)
class <class 'awxkit.api.pages.instance_groups.InstanceGroups'> export ['Roles', 'NotificationTemplates', 'WorkflowJobTemplateNodes', 'Credentials', 'GalaxyCredentials', 'Hosts', 'Groups', 'ExecutionEnvironments'].

126
log.error("key %r is_relation %s.", key, rel_endpoint)
key 'galaxy_credentials' is_relation /api/v2/organizations/1/galaxy_credentials/.
133
log.error("class %r export %s.", rel.__class__, EXPORTABLE_RELATIONS)
class <class 'awxkit.api.pages.base.Base'> export ['Roles', 'NotificationTemplates', 'WorkflowJobTemplateNodes', 'Credentials', 'GalaxyCredentials', 'Hosts', 'Groups', 'ExecutionEnvironments'].

The difference is that awxkit.api.pages, while its defined for all the other exportable relations, like it is for instance groups, its not for galaxy credential.

jbradberry · 2021-05-25T16:09:34Z

@john-westcott-iv @sean-m-sullivan so the downside of the current scheme is that it relies upon the registry being fully up to date and making use of the right Page subclass for each API endpoint. Galaxy credentials should absolutely get a Credential page type, but since the endpoint isn't properly registered, it falls back to Page. I did some quick tinkering and put together a patch that should address the issue which I'll work with @beeankha to turn into a PR today.

Enable Export of Galaxy Credentials Associated to Organizations SUMMARY Addresses #9342 ISSUE TYPE Bugfix Pull Request COMPONENT NAME API AWX VERSION awx: 19.1.0 Reviewed-by: Shane McDonald <me@shanemcd.com> Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>

beeankha · 2021-05-26T18:20:49Z

Hi @metabsd, with the merging of PR #10271, Galaxy Creds which are associated with organizations should be able to be exported correctly. I'm closing this issue, please let us know if you have any further problems related to this.

wenottingham added component:awx_collection issues related to the collection for controlling AWX state:needs_devel type:bug labels Feb 19, 2021

beeankha assigned jbradberry and beeankha May 25, 2021

beeankha mentioned this issue May 25, 2021

Enable Export of Galaxy Credentials Associated to Organizations #10271

Merged

AlanCoding added state:needs_test and removed state:needs_devel labels May 26, 2021

beeankha removed the state:needs_test label May 26, 2021

beeankha closed this as completed May 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

awx.awx.tower_export missing Galaxy Credential in organizations #9342

awx.awx.tower_export missing Galaxy Credential in organizations #9342

metabsd commented Feb 17, 2021

sean-m-sullivan commented Feb 17, 2021

jainnikhil30 commented Mar 16, 2021

sean-m-sullivan commented Mar 16, 2021 •

edited

Loading

jainnikhil30 commented Mar 17, 2021 •

edited

Loading

jainnikhil30 commented Mar 17, 2021

sean-m-sullivan commented Mar 17, 2021 •

edited

Loading

shanemcd commented Apr 10, 2021

sean-m-sullivan commented Apr 10, 2021

sean-m-sullivan commented Apr 27, 2021

john-westcott-iv commented Apr 28, 2021

sean-m-sullivan commented May 4, 2021 •

edited

Loading

jbradberry commented May 25, 2021

beeankha commented May 26, 2021

awx.awx.tower_export missing Galaxy Credential in organizations #9342

awx.awx.tower_export missing Galaxy Credential in organizations #9342

Comments

metabsd commented Feb 17, 2021

ISSUE TYPE

SUMMARY

ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

sean-m-sullivan commented Feb 17, 2021

jainnikhil30 commented Mar 16, 2021

sean-m-sullivan commented Mar 16, 2021 • edited Loading

jainnikhil30 commented Mar 17, 2021 • edited Loading

jainnikhil30 commented Mar 17, 2021

sean-m-sullivan commented Mar 17, 2021 • edited Loading

shanemcd commented Apr 10, 2021

sean-m-sullivan commented Apr 10, 2021

sean-m-sullivan commented Apr 27, 2021

john-westcott-iv commented Apr 28, 2021

sean-m-sullivan commented May 4, 2021 • edited Loading

jbradberry commented May 25, 2021

beeankha commented May 26, 2021

sean-m-sullivan commented Mar 16, 2021 •

edited

Loading

jainnikhil30 commented Mar 17, 2021 •

edited

Loading

sean-m-sullivan commented Mar 17, 2021 •

edited

Loading

sean-m-sullivan commented May 4, 2021 •

edited

Loading