Replace custom virtual environments with custom execution environments

[ryanpetrello]

To support custom SDKs/libraries in playbook execution, AWX supports the notion of "custom virtual environments" today, whereby users:

1. Build and deploy Python virtualenvs across their AWX infrastructure (in particular, on all awx_task nodes)
2. AWX code discovers these virtualenvs on the file system and exposes them as "options" for selection.
3. Users associate these virtualenvs at the Organization, Inventory, or Job Template level so that they will be utilized for playbook runs or inventory updates.

In a future version of AWX, we'd like to replace this functionality with the custom execution environments described in #7059

This has a few implications for migration:

1. The process for defining these custom execution environments will be different (the thing you're producing is a container image, not a Python virtualenv), and as such it is possible to incorporate system-level dependencies and collection-based content. We'll need good documentation on the new way of doing things, and will need to point users looking to migrate in the right direction.

2. We may want to consider a process for aiding users in migration from "old style" virtual environments (perhaps via pip freeze and a generated requirements.txt) to the new execution environments process.

3. In newer versions of AWX, we'll likely need a way to detect that you have old virtual environments associated with Organizations/Inventory/Job Templates and warn you that you'll need to migrate to the new custom execution enviroment approach described in Produce and document a process for AWX users to produce custom execution environments #7059

   • In this scenario, what should the behavior be? If you have an "old style" virtual environment, should we just use the default execution environment, or should AWX refuse to run the job until you've migrated away?

[kdelee]

@rebeccahhh is this what you are planning on working on this first sprint?

[rebeccahhh]

@kdelee yep!

[rebeccahhh]

update, this is moving to a later sprint

[kdelee]

@rebeccahhh where are we on this? When are we thinking we will remove it from the API? Is this feature the one that is tracking the tool that will do a pip freeze on your venvs?

[rebeccahhh]

@kdelee good question, this is kind of an umbrella issue but the next thing on the list is doing the pip freeze tool. The tool is also one of the last things we have to do for this prior to this being completed for release 🎊
Currently I have created a new API endpoint that spits out the pip freeze data for all the virtualenvs for the user to see. Still a bit buggy at the moment, but I'll have it up in a branch or draft PR today so it can be visible and others can poke it.

[AlanCoding]

#9498 (merged) made custom_virtualenv into a read-only field in the API.

The UI issue #9846 also appears to be relevant (I didn't see any links)

Right now we are displaying a warning message if a previous resource had a custom virtual env, that was not replace by an EE. It would nice to have a link to the doc in how to do that as part of the warning.

That makes it sound like the UI is already making use of the read-only field.

[rebeccahhh]

smol update, we're shifting from an API endpoint to an awx-manage command instead, most of the logic from the existing PR will remain the same, I'll be closing the PR and opening an new one for ease and clarity.

[kdelee]

@AlanCoding @rebeccahhh so we have left custom_virtualenv as a read only attribute on the objects, and we have now an awx-manage command that helps us list and export custom virutalenvs present on the system.

What seems to be missing is some way to export a mapping of custom venvs to objects. The way it stands the user is going to have to crawl over all the orgs and JTs etc in the API to find out what custom venv they used. This may be practical for small numbers of job templates or organizations, but we know some Tower deployments have hundreds of orgs and thousands of Job Templates and the book keeping could get pretty nasty/slow to crawl over the API and build the mapping in that way. Seems like doing that mapping in awx-manage talking directly to the DB would be faster and nicer than making the user crawl over the API.

[kdelee]

@rebeccahhh added to #10090 a command to list associations where the venv is being used. As soon as that merges, I think we can close this issue

[wenottingham]

Merged! closing.