Add GHA action to initialize pantsbuild env

[cognifloyd]

Background

This is another part of introducing pants, as discussed in the TSC Meetings on 12 July 2022, 02 Aug 2022 and 06 Sept 2022. Pants has fine-grained per-file caching of results for lint, fmt (like black), test, etc. It also has lockfiles that work well for monorepos that have multiple python packages. With these lockfiles CI should not break when any of our dependencies or our transitive dependencies release new versions, because CI will continue to use the locked version until we explicitly relock with updates.

To keep PRs as manageable/reviewable as possible, introducing pants will take a series of PRs. I do not know yet how many PRs; I will break this up into logical steps with these goals:

• introduce pants to the st2 repo, and
• teach some (but not all) TSC members about pants step-by-step.

Previous pants PRs include:

• Initialize pantsbuild #5713
• Configure pantsbuild source roots #5724
• Enable pantsbuild language backends #5725
• Adjust .gitignore to work better with pantsbuild #5726

Overview of this PR

It's helpful to review this PR one commit at a time.

This PR prepares us to run pants in GitHub Actions (GHA). There are 2 parts to this:

1. A reusable composite action to manage installing pants and restoring the pants caches
2. Add a CI-specific pants config file: pants.ci.toml (used in addition to pants.toml)

pants-init GHA composite action

We are going to use pants in a lot of different workflows. I developed the steps that make up this action in the pants PoC:
https://github.com/st2sandbox/st2/blob/pants/.github/workflows/lint.yaml

That PoC workflow was based on this (but following some of the idioms from our st2 workflows):
https://github.com/pantsbuild/example-python/blob/main/.github/workflows/pants.yaml

Using this action will look like this:

      - name: Initialize Pants
        uses: StackStorm/st2/.github/actions/pants-init@master
        with:
          # cache0 makes it easy to bust the cache if needed
          gha-cache-key: cache0-py${{ matrix.python-version }}

It also allows changing the python version that pants uses (the version pants uses can be different from the versions we target for our code - that is OK and expected) and the base branch (eg to switch from master to main) like this:

      - ...
        with:
          ...
          python-version: "3.9"
          base-branch: master

I've added some comments to point things out in this workflow.

CI-specific pants config

The pants docs recommend adding a CI-specific pants config file called pants.ci.toml:
https://www.pantsbuild.org/docs/using-pants-in-ci#configuring-pants-for-ci-pantscitoml-optional

Please note this quote from the docs:

set the environment variable PANTS_CONFIG_FILES=pants.ci.toml to use this new config file, in addition to pants.toml.

So, pants.ci.toml does NOT replace pants.toml, it extends it. So, pants will still use the pants.toml, but we can tune settings specifically for GHA, and possibly CircleCI (if we need to use pants there).

To make this work, the GHA action handles adding the PANTS_CONFIG_FILES env var. This way, authors that use our composite action will not have to remember to add that env var in their workflow.

[cognifloyd]

In the pants PoC workflow, this was here:

st2/.github/workflows/lint.yaml

Lines 77 to 80 in 9cf11f4

	- name: 'Set up Python for Pants (3.9)'
	uses: actions/setup-python@v2
	with:
	python-version: '3.9'

I made the python-version configurable to facilitate upgrading the python version used.

[cognifloyd]

NB: this composite action has to be available on the master branch before it can be used because you have to provide a version, which we will set to @master. So, you cannot modify the action in the same PR where you use the action.

[cognifloyd]

These steps make it so that we always use the cache from the last merge to master.

In the pants PoC workflow, this was here:

st2/.github/workflows/lint.yaml

Lines 86 to 104 in 9cf11f4

	- name: Get Pants Cache Merge Base Commit (base branch commit to pull cache from)
	id: pants-cache-commit
	# TODO: switch default base branch from pants to main/master
	run: |
	COMMIT=$(git merge-base ${GITHUB_BASE_REF:-pants} HEAD | head -n1)
	echo MERGEBASE=${COMMIT}
	echo "::set-output name=MERGEBASE::${COMMIT}"
	- name: Cache Pants Caches (Python ${{ matrix.python-version }})
	uses: actions/cache@v2
	id: pants-cache
	with:
	path: |
	~/.cache/pants/setup
	~/.cache/pants/lmdb_store
	~/.cache/pants/named_caches
	key: ${{ runner.os }}-pants-py${{ matrix.python-version }}-${{ github.sha }}
	restore-keys: |
	${{ runner.os }}-pants-py${{ matrix.python-version }}-${{ steps.pants-cache-commit.outputs.MERGEBASE }}
	${{ runner.os }}-pants-py${{ matrix.python-version }}-

I also made the base branch (master) configurable so that we can easily switch to main at some point (if we choose to do that).

[cognifloyd]

In the pants PoC workflow, this was here:

st2/.github/workflows/lint.yaml

Lines 132 to 134 in 9cf11f4

	- name: Bootstrap Pants
	run: |
	./pants --version

[cognifloyd]

Adding env vars like this to ${GITHUB_ENV} makes the vars show up for all subsequent steps in the workflow (in the workflow that uses this composite action).
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-environment-variable

[cognifloyd]

The scope of this PR changed as the action was moved under the pantsbuild umbrella. I'll open a different one.