Compile Python requirement files for MacOs, Linux or Windows.
- Compile Python requirement files (from requirements.in or pyproject.toml) for MacOs, Linux or Windows.
- Create PRs with the changes.
- Create Artifacts with compiled requirements.
This project will be irrelevant for you if:
- Your entire team uses Docker for development.
- Or you are the sole developer of a project.
- Or your entire team uses the same operating system (MacOs, Linux or Windows).
If none of the statements above qualifies, read further.
When a single project is being worked on by multiple developers, it's useful to streamline the installation process and ensure everyone is using exactly the same package versions.
It's a good practice to compile your input requirements and build your production environment from compiled requirements to prevent unpleasant surprises.
In many cases, pip-tools is your best friend, however, what pip-tools doesn't do is compiling requirements for multiple platforms. It can only compile requirements for the platform it's being executed on.
Software engineering is multi-diverse. Some may find Docker an essential tool, some others won't even bother using it and would prefer to work in a virtual environment.
And then it's your job to make sure everyone is working efficiently and does not accidentally break the project. If a Linux user makes a change in input requirements, you want to make sure requirements are properly compiled for all designated systems so that when a MacOs user pulls the changes and runs the installation script, he immediately has the project up-and-running with properly synchronized package versions.
Required The path to the input file (example: requirements.in).
Required Operating system name (platform slug).
GitHub Token for creating pull requests.
The path to directory to place the compiled requirements (default value: requirements/).
Branch to make a pull request to.
Prefix for the destination compiled filename.
Flag to determine if an artifact should be created (default value: false).
Selectively pick which workflow to run based on what has changed.
- Monitor your input requirement files using
pathsdirective. - Do not trigger your main tests on changes in input requirements.
- Trigger your main tests if compiled requirements have changed.
To create a Personal access token (PAT), go to GitHub Settings -> Developer Settings -> Fine-grained tokens and create a new token on your repository with the following permissions:
Actions: Read and write.Contents: Read and write.Environments: Read-only.Pull Requests: Read and write.Secrets: Read-only.
Additionally, it might be useful to allow the following too:
Commit statuses: Read-only.Metadata: Read-only.
Finally, create a New repository secret from Secrets and variables
section of your repository Settings, specify PAT_TOKEN as Name and
paste the content of newly created PAT as Secret.
Makefile example
# Define the name of the virtual environment directory
VENV := venv
# Detect the operating system
ifeq ($(OS),Windows_NT)
detected_OS := Windows
PYTHON := python
VENV_BIN := $(VENV)/Scripts
else
detected_OS := $(shell uname)
PYTHON := python3
VENV_BIN := $(VENV)/bin
endif
# Define the requirement file based on the operating system
ifeq ($(detected_OS),Windows)
REQUIREMENTS_FILE := requirements/windows-latest.txt
else ifeq ($(detected_OS),Darwin)
REQUIREMENTS_FILE := requirements/macos-latest.txt
else ifeq ($(detected_OS),Linux)
REQUIREMENTS_FILE := requirements/ubuntu-latest.txt
endif
# Default target
all: install
# Create a virtual environment
venv: $(VENV_BIN)/activate
# Virtual environment creation
$(VENV_BIN)/activate:
$(PYTHON) -m venv $(VENV)
# Install requirements into the virtual environment
install: venv
$(VENV_BIN)/pip install -r $(REQUIREMENTS_FILE)
# Enter virtual environment shell
shell: venv
$(VENV_BIN)/python
pip-list: venv
$(VENV_BIN)/pip list
# Clean the virtual environment
clean:
rm -rf $(VENV)
.PHONY: install venv clean
.github/workflows/test-action.yml
name: Test Compile Requirements Action
on:
push:
paths:
- 'examples/requirements.in'
- 'examples/pyproject.toml'
- '.github/workflows/test-action.yml'
- 'action.yml'
permissions:
contents: write
pull-requests: write
jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ # See this as an example
ubuntu-latest,
ubuntu-22.04,
ubuntu-20.04,
windows-latest,
windows-2022,
windows-2019,
macos-latest,
macos-13,
macos-12,
macos-11,
]
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Set up platform-specific variables
id: vars
shell: bash
run: |
OS_NAME=$(echo ${{ matrix.os }} | tr '[:upper:]' '[:lower:]' | sed -e 's/[^a-zA-Z0-9]+/-/g')
echo "PLATFORM_SLUG=${OS_NAME%}" >> $GITHUB_ENV
echo "TARGET_BRANCH=$(echo ${GITHUB_REF#refs/heads/})" >> $GITHUB_ENV
- name: Run Compile and PR Requirements Action
uses: barseghyanartur/compile-python-requirements-action@0.1
with:
input-file: 'examples/requirements.in'
os-name: ${{ env.PLATFORM_SLUG }}
github-token: ${{ secrets.PAT_SECRET }}
output-directory: 'examples/requirements' # Optional
prefix: '' # Optional
# Optional. Pass the target branch to the action
target-branch: ${{ env.TARGET_BRANCH }}
create-artifact: 'true' # Optional
- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: requirements-${{ env.PLATFORM_SLUG }}
path: examples/requirements/requirements.tar.gz
if-no-files-found: 'warn'MIT
For security issues contact me at the e-mail given in the Author section.
For overall issues, go to GitHub issues.
Artur Barseghyan