setup-hooks/strip: resolve/uniq symlinks before stripping #246164
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 30, 2023
ncfavier
reviewed
Jul 30, 2023
Before the change the hook had a chance to run `strip` against the same file using multiple link paths. In case of `gcc` `libgcc.a` was stripped multiple times in parallel and produces corrupted archive. The change runs inputs via `realpath | uniq` to make sure we don't attempt to strip the same files multiple times.
vcunat
added a commit
that referenced
this pull request
Jul 30, 2023
Adapted from PR #246164 TODO: clean up / use it everywhere on the next rebuild.
|
Good shout! This makes me wonder though if we should also filter to avoid even trying to strip files outside the outputs of the current derivation -- currently, if we have a symlink pointing to another store path, stripping it will be attempted but of course fail because it's read-only. |
|
I sounds reasonable to fix those cases as well. I'm a bit worried to add bigger changes to this specific PR. |
lheckemann
approved these changes
Jul 31, 2023
Wow, nice find. |
12 tasks
13 tasks
tpwrules
added a commit
to tpwrules/nixpkgs
that referenced
this pull request
May 24, 2024
NixOS#246164 but for hardlinks. Mesa, among other packages, has binaries that are linked together and can end up corrupted when the same binary is stripped through two different names. To resolve this, print out the device and inode number before each file name, sort/uniq based on that, then cut it back out before stripping. The symlink resolution logic is removed as the same file accessed through two different links in `$paths` will necessarily have the same numbers. File/directory within the paths listed in `$paths` are correctly not (and were never) processed due to the `-type f` predicate and (implied) `-P` option to `find`.
github-actions bot
pushed a commit
that referenced
this pull request
May 25, 2024
#246164 but for hardlinks. Mesa, among other packages, has binaries that are linked together and can end up corrupted when the same binary is stripped through two different names. To resolve this, print out the device and inode number before each file name, sort/uniq based on that, then cut it back out before stripping. The symlink resolution logic is removed as the same file accessed through two different links in `$paths` will necessarily have the same numbers. File/directory within the paths listed in `$paths` are correctly not (and were never) processed due to the `-type f` predicate and (implied) `-P` option to `find`. (cherry picked from commit 4d6d293)
winterqt
pushed a commit
that referenced
this pull request
May 26, 2024
#246164 but for hardlinks. Mesa, among other packages, has binaries that are linked together and can end up corrupted when the same binary is stripped through two different names. To resolve this, print out the device and inode number before each file name, sort/uniq based on that, then cut it back out before stripping. The symlink resolution logic is removed as the same file accessed through two different links in `$paths` will necessarily have the same numbers. File/directory within the paths listed in `$paths` are correctly not (and were never) processed due to the `-type f` predicate and (implied) `-P` option to `find`. (cherry picked from commit 4d6d293)
github-actions bot
pushed a commit
that referenced
this pull request
May 26, 2024
#246164 but for hardlinks. Mesa, among other packages, has binaries that are linked together and can end up corrupted when the same binary is stripped through two different names. To resolve this, print out the device and inode number before each file name, sort/uniq based on that, then cut it back out before stripping. The symlink resolution logic is removed as the same file accessed through two different links in `$paths` will necessarily have the same numbers. File/directory within the paths listed in `$paths` are correctly not (and were never) processed due to the `-type f` predicate and (implied) `-P` option to `find`. (cherry picked from commit 4d6d293)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before the change the hook had a chance to run
stripagainst the same file using multiple link paths. In case ofgcclibgcc.awas stripped multiple times in parallel and produces corrupted archive.The change runs inputs via
realpath | uniqto make sure we don't attempt to strip the same files multiple times.Description of changes
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)