forked from llvm/torch-mlir
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
build: create RollPyTorch to update PyTorch version in Torch-MLIR (ll…
…vm#1419) This patch fetches the most recent nightly (binary) build of PyTorch, before pinning it in pytorch-requirements.txt, which is referenced in the top-level requirements.txt file. This way, end users will continue to be able to run `pip -r requirements.txt` without worrying whether doing so will break their Torch-MLIR build. This patch also fetches the git commit hash that corresponds to the nightly release, and this hash is passed to the out-of-tree build so that it can build PyTorch from source. If we were to sort the torch versions as numbers (in the usual descending order), then 1.9 appears before 1.13. To fix this problem, we use the `--version-sort` flag (along with `--reverse` for specifying a descending order). We also filter out lines that don't contain version numbers by only considering lines that start with a digit. As a matter of slight clarity, this patch renames the variable `torch_from_src` to `torch_from_bin`, since that variable is initialized to `TM_USE_PYTORCH_BINARY`. Co-authored-by: powderluv <powderluv@users.noreply.github.com>
- Loading branch information
Showing
6 changed files
with
105 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| name: Roll PyTorch | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| build_linux: | ||
| name: Manylinux Build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Get torch-mlir | ||
| uses: actions/checkout@v2 | ||
| with: | ||
| submodules: 'true' | ||
| - name: Setup ccache | ||
| uses: ./.github/actions/setup-build | ||
| with: | ||
| cache-suffix: x86_64-out-of-tree-OFF | ||
| - name: Determine nightly PyTorch version | ||
| run: | | ||
| cd ${GITHUB_WORKSPACE} | ||
| python -m pip install wheel | ||
| # Fetch the most recent nightly PyTorch release | ||
| PT_RELEASE=$(python -m pip index versions -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html --pre torch | grep "Available versions" | tr ' ' '\n' | grep "^[0-9]" | sort --version-sort --reverse | head -n1 | tr -d ',' | sed 's/\([^+]*\).*/\1/') | ||
| printf -- "-f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html\n--pre\ntorch==%s\n" "${PT_RELEASE}" > pytorch-requirements.txt | ||
| # Fetch the whl file associated with the nightly release | ||
| rm -f torch-"${PT_RELEASE}"*.whl | ||
| python -m pip download -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html --pre "torch==${PT_RELEASE}" | ||
| # Read the commit hash from the downloaded whl file without extracting it | ||
| PT_HASH=$(unzip -p torch-"${PT_RELEASE}"*.whl torch/version.py | grep git_version | awk '{ print $3 }' | tr -d "'") | ||
| echo "${PT_HASH}" > pytorch-version.txt | ||
| rm torch-"${PT_RELEASE}"*.whl | ||
| # Write the release and hash to the environment file so that we can | ||
| # retrieve them when creating a PR | ||
| echo "PT_HASH=${PT_HASH}" >> ${GITHUB_ENV} | ||
| echo "PT_RELEASE=${PT_RELEASE}" >> ${GITHUB_ENV} | ||
| - name: Build and test | ||
| run: | | ||
| cd ${GITHUB_WORKSPACE} | ||
| TM_PACKAGES="out-of-tree" TM_USE_PYTORCH_BINARY="OFF" \ | ||
| TORCH_MLIR_SRC_PYTORCH_BRANCH="${{ env.PT_HASH }}" \ | ||
| TORCH_MLIR_SRC_PYTORCH_RELEASE="${{ env.PT_RELEASE }}" \ | ||
| ./build_tools/python_deploy/build_linux_packages.sh | ||
| - name: Push changes to new branch | ||
| run: | | ||
| BRANCH="merge/pytorch-update-${{ env.PT_RELEASE }}" | ||
| TITLE="update PyTorch version to ${{ env.PT_RELEASE }}" | ||
| echo "BRANCH=${BRANCH}" >> ${GITHUB_ENV} | ||
| echo "TITLE=${TITLE}" >> ${GITHUB_ENV} | ||
| cd ${GITHUB_WORKSPACE} | ||
| git config user.email "torch-mlir@users.noreply.github.com" | ||
| git config user.name "Roll PyTorch Action" | ||
| git checkout -b "${BRANCH}" | ||
| git add pytorch-version.txt pytorch-requirements.txt | ||
| git commit -m "${TITLE}" | ||
| git push --set-upstream origin "${BRANCH}" | ||
| - name: Create PR to push new PyTorch version | ||
| run: | | ||
| URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" | ||
| BODY="PyTorch commit hash: \`${{ env.PT_HASH }}\` -- CI link: ${URL}" | ||
| cd ${GITHUB_WORKSPACE} | ||
| gh pr create -H "${{ env.BRANCH }}" -B main --title "${{ env.TITLE }}" \ | ||
| --body "${BODY}" --reviewer powderluv | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.WORKFLOW_INVOCATION_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html | ||
| --pre | ||
| torch==1.13.0.dev20220927 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 04bb9533d516919190b80a0682e93dd44da9ef6d |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters