Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_ci_ Refactor release pipeline to better support m1 builds #9600

Merged
merged 6 commits into from
Nov 8, 2022

Conversation

ianconsolata
Copy link
Contributor

If possible, I would like to merge this in before the 1.18.0 release, otherwise I will have to manually build and update the binaries for that release myself. I know it's kinda big and last minute though, so I understand if you want to wait until 1.18.1-rc1 to put this in action.

Related Issues

Right now, our m1 builds are broken, and don't generate working universal binaries during releases. On pre-releases, they aren't built at all.

Proposed Changes

This does a few things:

  • Uses a custom m1 mac runner running in aws to build darwin-arm64 binaries
  • Prebuilds binaries for darwin-arm64, darwin-amd64, and linux-amd64 in three separate build jobs
  • Pull those binaries together in a release job which makes a release using goreleaser, pubishing that to the Github Release
  • It will now successfully publish the binaries on all releases, including pre-releases (i.e. *-rc) , but will only publish to homebrew on final releases.
  • removes the build-ntwk-calibration and build-ntwk-butterfly jobs, as no one is using / relying on those anymore. In a future PR I will be tweaking our docker publishing flow to include docker builds for those networks.
  • Removed a bunch of seemingly unnecessary cruft in our build process (like caching the cargo dependencies on jobs that we don't use rust)

Testing

This was extensively tested on my test branch, but given the nature of the release process it can't be fully tested until it's run in a release environment. Please let me know when the next release is that will ship these changes, and I will make sure I am on call to fix any issues during the release.

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

This builds three separate binaries (darwin/amd64, darwin/arm64,
linux/amd64), and then combines them into single release (including a
universal darwin binary) using goreleaser.

Also removes build-ntwk-{calibration,butterfly}
- Set CPATH and LIBRARY_PATH for lotus m1 build
- support dry-run on new release job
- Removes cargo caching (since we don't build FFI from source, this
isn't used)
- Removes npm (this isn't a build dependency, so not sure why it was
  being installed)
@ianconsolata ianconsolata requested a review from a team as a code owner November 7, 2022 16:31
@ianconsolata ianconsolata changed the title Refactor release pipeline to better support m1 builds _ci_ Refactor release pipeline to better support m1 builds Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants