Implement AWX support for a data model and APIs to represent and manage Execution Environments

[ryanpetrello]

This is a subtask of a larger enhancement/initiative surrounding execution environments: #5157

Implement APIs for defining execution environment and their sources (registries).

• A model and API for defining, interacting with, and authenticating to one or more execution environments (e.g., name, URL, any necessary credential data).
  • A name
  • Credentials specific to registry authentication
  • A URL e.g., quay.io/redhat/someimage:latest
• An interface for attaching execution environments to Organization, Inventories, and Job Templates.
• RBAC for execution environments
• A documented process for registering new execution environments, and notifying AWX of their existance.

This work will be consumed by #7060

[ryanpetrello]

Additionally, we'll want to support this in our various SDK/CLI toolkit i.e., adding support creating and associating these in both the AWX collection and CLI.

[jlmitch5]

UI work for execution environments:

• There will be a new credential type (container registry credential type). Hopefully this will just work based on the credential type coming from the api once we build that (no UI config)
• New CRUD view for execution environments (i.e. images). Detail view will have RBAC...would also need to add to the RBAC adder at the user level. Fields are: Fields are
  • name (text)
  • registry url (text)
  • credential selector (lookup, optional)
• Maybe: Execution environment detail view might have test button which shows success/failure notification when:
  • the image is accessible with the provided credentials
  • the test may also validate that containers built from the image provides access to runner and ansible
• We need fields to set execution environment where venv selectors are now (replaces the venv selectors)...in Job Template, Inventory sync, etc.

[jbradberry]

Note that I've been convinced that having separate name and image url fields is redundant, so I've reduced it to a single field called image.

[jbradberry]

Additional items and potential items that were left out of PR #7455

• Add a new CredentialType
• Add a configure-Tower-in-Tower setting for the global default EE
• Provide a way to install new bundled EEs
• Create a new Org-level EE admin RBAC role
  • Update the access class(es) to allow this to work
• Add a new Project.default_environment field, allowing a level of default EE between a full Org-level default and an explicit job template configured EE
  • Update the Project serializer to deal with this new field
• Add a new api sublist view for EE <- Unified Jobs
• Add an access_list to the EE api endpoint (Add access view for Execution Environments #8198)
• Add user_capabilities to the EE api endpoint (EE user capabilities #8208)
• Add a create() method for EEs under awxkit (Changes to get execution environments factories working #8126)
• Add a factory or factories for creating EEs for unit tests

cc @shanemcd

[one-t]

Just mentioning it here, there's currently no create method in awxkit for EEs

[AlanCoding]

Most of those look good, a few questions:

Provide a way to install new bundled EEs

What does this mean? I thought the user is expected to use an image registry. Yes, there is some time required to download the images on a new machine (somewhat similar to how project syncs work), and timeout in some tests have to be relaxed for this reason, particularly as image size grows. But I don't know where you're going with this idea of bundled EEs. I don't know of any remaining work that looks quite like that.

Add a new api sublist view for EE <- Unified Jobs

What does this mean? A single unified job should have exactly 1 EE associated with it, similar to instance & instance group & custom virtualenv.

[jbradberry]

Add a new api sublist view for EE <- Unified Jobs

What does this mean? A single unified job should have exactly 1 EE associated with it, similar to instance & instance group & custom virtualenv.

Sure, but the reverse is not true. Any given EE will wind up with many jobs that were run using it.

To be clear, I'm squishy as to whether this is an actually useful API endpoint, which is why I kicked the can down the road.

[jbradberry]

Provide a way to install new bundled EEs

What does this mean? I thought the user is expected to use an image registry. Yes, there is some time required to download the images on a new machine (somewhat similar to how project syncs work), and timeout in some tests have to be relaxed for this reason, particularly as image size grows. But I don't know where you're going with this idea of bundled EEs. I don't know of any remaining work that looks quite like that.

There is supposed to be an EE model instance that will be associated with and installed by any given version of AWX, that will have the reference to the image in the registry that is the expected default container EE that will be used.

[AlanCoding]

Any given EE will wind up with many jobs that were run using it.

Oh right, carry on. Something like /api/v2/execution_environments/4/unified_jobs/. I don't have a use, but it's a coherent idea.

There is supposed to be an EE model instance that will be associated with and installed by any given version of AWX, that will have the reference to the image in the registry that is the expected default container EE that will be used.

I want to know a little more about the reasons here. @kdelee has done a lot to try and clarification expectations for the image version. Yes, we need to produce tags specific to AWX version, so you might have :15.0.0 at the end of the image URL. But you could also have :devel or :latest.

Just a little bit above you mentioned:

Add a configure-Tower-in-Tower setting for the global default EE

If we have this, why would we need an EE model instance for the same thing?

Even if you need to pin the exact version, you already have the current AWX version, so you could just append that onto the default value, defined in settings/defaults.py. This is already in code:

awx/awx/settings/defaults.py

Line 67 in c72d55f

AWX_EXECUTION_ENVIRONMENT_DEFAULT_IMAGE = 'quay.io/ansible/ansible-runner:devel'

Let's say the default image gets named quay.io/ansible/awx-default-execution-environment, you could put in

AWX_EXECUTION_ENVIRONMENT_DEFAULT_IMAGE = 'quay.io/ansible/awx-default-execution-environment:' + get_awx_version() 

And again, I don't even know if you need to do that, because we can just make :latest the default and give users instructions to pin it if they so choose.

[kdelee]

Closed due to extensive testing from @elyezer @yagomarques and @shebangbash , any follow up will be tracked in new issues