Cache the roles directory based on requirements.yml

[mrmibrown]

ISSUE TYPE

• Feature Idea

SUMMARY

Cache the roles directory based on requirements.yml to expedite jobs with larger repositories.

[wenottingham]

@AlanCoding do you have idea on how this would be done?

[AlanCoding]

Yes, I really wanted to do this in spare time, but I'm too far behind at the moment.

To correctly handle different branches, it makes sense to organize by the commit hash. Either project updates or project syncs could run the command, and then save the role contents in a folder specific to the project and specific to the commit (not in the project source tree).

This wouldn't change the job folder structure that we have now, but it would change user experience with updating projects and debugging Galaxy-related errors (hopefully for the better).

There are certainly more details to nit pick over, but in general terms I don't think there should be any problem with this.

[wenottingham]

Well, I'd point immediately at the cache invalidation/cleanup issue, especially when it comes to promptable branches.

[AlanCoding]

that's a fair point, and because of that it might not be realistic to cache anything other than default branches.

[n2aws]

I see this is labelled as an enhancement, but I think we should consider the current behavior in 9.1.0, 9.1.1, and 9.2.0 to be a "bug" as the change broke how roles/requirements.yml works, and several people have entire jobs that broke with this.

[AlanCoding]

support for collections from SCM just merged

ansible/ansible#69154

trying to think of how that could impact any solution here.

[AlanCoding]

This is being merged, I'm leaving the issue open for @elyezer to close when ready

[elyezer]

This has been verified and works as expected.