Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-40749: [Python][Packaging] Strip unnecessary symbols when building wheels #42028

Merged
merged 7 commits into from
Jun 11, 2024

Conversation

raulcd
Copy link
Member

@raulcd raulcd commented Jun 7, 2024

Rationale for this change

Removing unnecessary symbols for wheels will allow us to reduce the size of the wheels considerably.

What changes are included in this PR?

Running strip --strip-debug on Linux wheels for all *.so files.

Are these changes tested?

Yes, via Archery.

Are there any user-facing changes?

No

@github-actions github-actions bot added the awaiting committer review Awaiting committer review label Jun 7, 2024
@raulcd
Copy link
Member Author

raulcd commented Jun 7, 2024

@github-actions crossbow submit wheel-manylinux-*

This comment was marked as outdated.

@raulcd
Copy link
Member Author

raulcd commented Jun 7, 2024

This change reduces from 131MB locally unzipped wheel from the previous nightly build to 101MB locally unzipped wheel the wheel on wheel-manylinux-2-28-cp310-amd64.

@raulcd
Copy link
Member Author

raulcd commented Jun 7, 2024

@jorisvandenbossche @pitrou would you prefer to:

@pitrou
Copy link
Member

pitrou commented Jun 7, 2024

@jorisvandenbossche @pitrou would you prefer to:

  • [1] Go with the current change just using --strip on auditwheel repair

-1

  • [2] Run strip --strip-debug manually

+1

+0

@raulcd
Copy link
Member Author

raulcd commented Jun 7, 2024

@github-actions crossbow submit wheel-manylinux-*

Copy link

github-actions bot commented Jun 7, 2024

Failed to push updated references, potentially because of credential issues: ['refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp311-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp311-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp312-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp312-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp311-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp311-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp310-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp310-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp39-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp39-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp310-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp310-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp312-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp312-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp312-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp312-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp311-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp311-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp310-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp310-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp312-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp312-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp310-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp310-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp39-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp39-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp38-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp38-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2014-cp38-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2014-cp38-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp38-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp38-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp39-amd64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp39-amd64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp38-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp38-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp311-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp311-arm64', 'refs/heads/actions-3236717c81-github-wheel-manylinux-2-28-cp39-arm64', 'refs/tags/actions-3236717c81-github-wheel-manylinux-2-28-cp39-arm64', 'refs/heads/actions-3236717c81']
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/9420748198

@raulcd
Copy link
Member Author

raulcd commented Jun 7, 2024

@github-actions crossbow submit wheel-manylinux-*

Copy link

github-actions bot commented Jun 7, 2024

Revision: 7dd1e28

Submitted crossbow builds: ursacomputing/crossbow @ actions-f4cd718f1d

Task Status
wheel-manylinux-2-28-cp310-amd64 GitHub Actions
wheel-manylinux-2-28-cp310-arm64 GitHub Actions
wheel-manylinux-2-28-cp311-amd64 GitHub Actions
wheel-manylinux-2-28-cp311-arm64 GitHub Actions
wheel-manylinux-2-28-cp312-amd64 GitHub Actions
wheel-manylinux-2-28-cp312-arm64 GitHub Actions
wheel-manylinux-2-28-cp38-amd64 GitHub Actions
wheel-manylinux-2-28-cp38-arm64 GitHub Actions
wheel-manylinux-2-28-cp39-amd64 GitHub Actions
wheel-manylinux-2-28-cp39-arm64 GitHub Actions
wheel-manylinux-2014-cp310-amd64 GitHub Actions
wheel-manylinux-2014-cp310-arm64 GitHub Actions
wheel-manylinux-2014-cp311-amd64 GitHub Actions
wheel-manylinux-2014-cp311-arm64 GitHub Actions
wheel-manylinux-2014-cp312-amd64 GitHub Actions
wheel-manylinux-2014-cp312-arm64 GitHub Actions
wheel-manylinux-2014-cp38-amd64 GitHub Actions
wheel-manylinux-2014-cp38-arm64 GitHub Actions
wheel-manylinux-2014-cp39-amd64 GitHub Actions
wheel-manylinux-2014-cp39-arm64 GitHub Actions

@raulcd raulcd marked this pull request as ready for review June 10, 2024 09:07
@raulcd raulcd requested review from assignUser and kou as code owners June 10, 2024 09:07
@raulcd raulcd marked this pull request as draft June 10, 2024 09:08
@raulcd
Copy link
Member Author

raulcd commented Jun 10, 2024

Running strip --strip-debug manually seems to reduce a locally unzipped wheel from 131MB to 124MB

@raulcd
Copy link
Member Author

raulcd commented Jun 10, 2024

@github-actions crossbow submit wheel-macos-*

Copy link

Revision: aeab32e

Submitted crossbow builds: ursacomputing/crossbow @ actions-8a67ea442e

Task Status
wheel-macos-big-sur-cp310-arm64 GitHub Actions
wheel-macos-big-sur-cp311-arm64 GitHub Actions
wheel-macos-big-sur-cp312-arm64 GitHub Actions
wheel-macos-big-sur-cp38-arm64 GitHub Actions
wheel-macos-big-sur-cp39-arm64 GitHub Actions
wheel-macos-catalina-cp310-amd64 GitHub Actions
wheel-macos-catalina-cp311-amd64 GitHub Actions
wheel-macos-catalina-cp312-amd64 GitHub Actions
wheel-macos-catalina-cp38-amd64 GitHub Actions
wheel-macos-catalina-cp39-amd64 GitHub Actions

@raulcd
Copy link
Member Author

raulcd commented Jun 10, 2024

@github-actions crossbow submit wheel-macos-catalina-cp310-amd64

Copy link

Revision: e41c654

Submitted crossbow builds: ursacomputing/crossbow @ actions-9df43f583e

Task Status
wheel-macos-catalina-cp310-amd64 GitHub Actions

@raulcd
Copy link
Member Author

raulcd commented Jun 10, 2024

@github-actions crossbow submit wheel-macos-catalina-cp310-amd64

Copy link

Revision: 837d114

Submitted crossbow builds: ursacomputing/crossbow @ actions-1d686e8f7f

Task Status
wheel-macos-catalina-cp310-amd64 GitHub Actions

@raulcd
Copy link
Member Author

raulcd commented Jun 10, 2024

strip -S $filename doesn't seem to have any effect on macOS, the size of the wheels and the unpacked wheel is exactly the same with and without executing the command

@raulcd raulcd marked this pull request as ready for review June 10, 2024 11:32
@raulcd raulcd requested a review from pitrou June 10, 2024 11:32
@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Jun 10, 2024
Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 except for a nit

Co-authored-by: Antoine Pitrou <pitrou@free.fr>
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jun 10, 2024
@raulcd
Copy link
Member Author

raulcd commented Jun 11, 2024

👍 merging

@raulcd raulcd merged commit baf4089 into apache:main Jun 11, 2024
12 checks passed
@raulcd raulcd removed the awaiting change review Awaiting change review label Jun 11, 2024
@raulcd raulcd deleted the GH-40749 branch June 11, 2024 09:41
Copy link

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit baf4089.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants