[airflow] upgrade to composer-2.7.1-airflow-2.7.3, python 3.11 (Recid…

…iviz/recidiviz-data#29712)

## Description of the change

Recidiviz/recidiviz-data#27852 upgraded to `composer-2.6.2-airflow-2.6.3` which failed during
the tf step and was reverted in Recidiviz/recidiviz-data#29696

The commits in this PR are as follows --
1. un-reverts Recidiviz/recidiviz-data#27852
2. upgrades to `composer-2.7.1-airflow-2.7.3`, copying requirements over
from [version
page](https://cloud.google.com/composer/docs/concepts/versioning/composer-versions#images)
3.  airflow pipenv lock workflow
4. removes a few imports and checks since
[Recidiviz/recidiviz-data#32731](apache/airflow#32731) opened by our
very own @ohaibbq is included in 2.7.1

read through the [release
notes](https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html#airflow-2-7-0-2023-08-18),
nothing in our usage seems to be deprecated; there are a few things that
might be of interest to folks:
- _Create metrics to track Scheduled->Queued->Running task state
transition times
([Recidiviz/recidiviz-data#30612](apache/airflow#30612 will be nice
for us to be able to see this re: k8s spin up time
- _Add OpenTelemetry to Airflow
([AIP-49](https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-49+milestone%3A%22Airflow+2.7.0%22))_
- _The trigger UI form is skipped in web UI if no parameters are defined
in a DAG ([Recidiviz/recidiviz-data#33351](apache/airflow#33351: not
sure if folks run ad-hoc dags using the UI instead of
`trigger_state_specific_calculation/ingest_dag` but we need to add
`show_trigger_form_if_no_params` if we want to still be able to trigger
via the UI

would love a second set of eyes around these warning logs that occur
after tests run, see
[here](https://github.com/Recidiviz/recidiviz-data/actions/runs/9071731702/job/24926024866?pr=29712#step:6:53).
also occurred locally but i couldn't isolate the cause of the issue
```
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/logging/__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 509, in <lambda>
    and _finalize_fairy(
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 791, in _finalize_fairy
    pool.logger.error(
Message: 'Exception during reset or similar'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 763, in _finalize_fairy
    fairy._reset(pool, transaction_was_reset)
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1038, in _reset
    pool._dialect.do_rollback(self)
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 683, in do_rollback
    dbapi_connection.rollback()
psycopg2.OperationalError: server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/logging/__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 509, in <lambda>
    and _finalize_fairy(
  File "/home/runner/.local/share/virtualenvs/airflow-jCIRjM3K/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 791, in _finalize_fairy
    pool.logger.error(
Message: 'Exception during reset or similar'
Arguments: ()
```

## Type of change

> All pull requests must have at least one of the following labels
applied (otherwise the PR will fail):

| Label | Description |
|-----------------------------
|-----------------------------------------------------------------------------------------------------------
|
| Type: Bug | non-breaking change that fixes an issue |
| Type: Feature | non-breaking change that adds functionality |
| Type: Breaking Change | fix or feature that would cause existing
functionality to not work as expected |
| Type: Non-breaking refactor | change addresses some tech debt item or
prepares for a later change, but does not change functionality |
| Type: Configuration Change | adjusts configuration to achieve some end
related to functionality, development, performance, or security |
| Type: Dependency Upgrade | upgrades a project dependency - these
changes are not included in release notes |

## Related issues

Closes Recidiviz/recidiviz-data#29554

## Checklists

### Development

**This box MUST be checked by the submitter prior to merging**:
- [x] **Double- and triple-checked that there is no Personally
Identifiable Information (PII) being mistakenly added in this pull
request**

These boxes should be checked by the submitter prior to merging:
- [ ] Tests have been written to cover the code changed/added as part of
this pull request

### Code review

These boxes should be checked by reviewers prior to merging:

- [x] This pull request has a descriptive title and information useful
to a reviewer
- [x] Potential security implications or infrastructural changes have
been considered, if relevant

---------

Co-authored-by: Helper Bot <helperbot@recidiviz.org>
GitOrigin-RevId: 355d050265f490df1779ff773533f5aefd41de6b

.github/actions/airflow-python-setup/action.yml

@@ -4,7 +4,7 @@ runs:
   steps:
     - uses: actions/setup-python@v2
       with:
-        python-version: "3.8.12"
+        python-version: "3.11.8"
     - name: Install pipenv
       run: pip install pipenv
       shell: bash
@@ -13,13 +13,13 @@ runs:
       id: pipenv-caching
       with:
         path: ~/.local/share/virtualenvs/
-        key: ${{ runner.os }}-python-${{ steps.setup-python.outputs.python-version }}-pipenv-airflow-v3-${{ hashFiles('**/recidiviz/airflow/Pipfile.lock') }}
+        key: ${{ runner.os }}-python-${{ steps.setup-python.outputs.python-version }}-pipenv-airflow-v5-${{ hashFiles('**/recidiviz/airflow/Pipfile.lock') }}
         # If we haven't cached a pipenv for the new version of the lock file this will
         # restore the cache for the most recent version. Since most of the time our
         # lock file updates only change a small number of packages, this speeds up
         # the sync step significantly.
         restore-keys: |
-          ${{ runner.os }}-python-${{ steps.setup-python.outputs.python-version }}-pipenv-airflow-v3-
+          ${{ runner.os }}-python-${{ steps.setup-python.outputs.python-version }}-pipenv-airflow-v5-
     - name: Sync pipenv
       if: steps.pipenv-caching.outputs.cache-hit != 'true'
       # If we didn't find an exact pipenv match, we have to `sync.` Because we may have