Split segment size fix #447

otherjason · 2022-12-02T15:20:44Z

Fix for #446

When a section in the file needs to be enlarged (e.g. to accommodate setting a larger RPATH), shiftFile() is used to shift all content following the growing section to a later position in the file.

Commit 109b771 introduced logic to ensure that, after the segment split, no sections span multiple segments. This is done by sliding the portion of the segment after the split point later in the file, then adding a new PT_LOAD segment that contains the preceding data plus the extra room that is being added. The existing implementation does this by simply adding extraPages*getPageSize() bytes to the number of bytes ahead of the split point in the segment.

However, this approach can result in two PT_LOAD segments that overlap when page boundaries are taken into account. As an example, this PT_LOAD section (taken from a Python 3.10 binary):

LOAD           0x0000000000000000 0x0000000000400000 0x0000000000400000
               0x0000000000000948 0x0000000000000948  R E    0x200000

is split into the following two sections:

LOAD           0x0000000000000000 0x00000000003ff000 0x00000000003ff000
               0x0000000000001594 0x0000000000001594  R E    0x1000
LOAD           0x0000000000001594 0x0000000000400594 0x0000000000400594
               0x00000000000003b4 0x00000000000003b4  R E    0x1000

Note that the two PT_LOAD sections both contain the memory page at address 0x400000. The Linux kernel's ELF loader (at least as of v4.18) does not accept this as a valid ELF executable, triggering a segfault with si_code=SI_KERNEL immediately when the binary is executed.

The fix here is to set the length of the segment that comes before the split point more carefully; instead of adding extraPages*getPageSize() bytes to the portion of the segment that came before the split, the actual number of padding bytes that were needed (before rounding up to the next multiple of the page size) are used. This avoids the overlap in the PT_LOAD segments and makes the output files executable again.

Fix for incorrect segment flags

Another thing I fixed while in here is the unconditional setting of the flags in the newly-generated segment to RW. Since this segment is formed by splitting one of the segments from the input file into two, it seems most accurate to preserve that section's flags for both of the two post-split segments.

…ginal flags in both

When a section in the file needs to be enlarged (e.g. to accommodate setting a larger RPATH), shiftFile() is used to shift all content following the growing section to a later position in the file. Commit 109b771 introduced logic to ensure that, after the segment split, no sections span multiple segments. This is done by sliding the portion of the segment after the split point later in the file, then adding a new PT_LOAD segment that contains the preceding data plus the extra room that is being added. The existing implementation does this by simply adding `extraPages*getPageSize()` bytes to the number of bytes ahead of the split point in the segment. However, this approach can result in two PT_LOAD segments that overlap when page boundaries are taken into account. As an example, this PT_LOAD section (taken from a Python 3.10 binary): LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x0000000000000948 0x0000000000000948 R E 0x200000 is split into the following two sections: LOAD 0x0000000000000000 0x00000000003ff000 0x00000000003ff000 0x0000000000001594 0x0000000000001594 R E 0x1000 LOAD 0x0000000000001594 0x0000000000400594 0x0000000000400594 0x00000000000003b4 0x00000000000003b4 R E 0x1000 Note that the two PT_LOAD sections both contain the memory page at address 0x400000. The Linux kernel's ELF loader (at least as of v4.18) does not accept this as a valid ELF executable, triggering a segfault with si_code=SI_KERNEL immediately when the binary is executed. The fix here is to set the length of the segment that comes before the split point more carefully; instead of adding `extraPages*getPageSize()` bytes to the portion of the segment that came before the split, the actual number of padding bytes that were needed (before rounding up to the next multiple of the page size) are used. This avoids the overlap in the PT_LOAD segments and makes the output files executable again.

Mic92 · 2022-12-14T12:01:33Z

Sorry, missed this pr.

Mic92 · 2022-12-14T12:08:46Z

This PR seems to break a number of tests. Can you have a look by running make check on them? Let me know if you need help to reproduce them.

otherjason · 2022-12-23T20:30:14Z

I haven't been able to reproduce them on my system; all of the tests pass successfully. I guess that is to be expected, though, as I'm on Ubuntu 18.04 and all of the Ubuntu tests in CI seem to have passed as well; only the nix tests show failures.

Mic92 · 2022-12-24T07:21:30Z

Here is the full log:

=================================================
   patchelf 0.17.0-alpha: tests/test-suite.log
=================================================

# TOTAL: 48
# PASS:  41
# SKIP:  0
# XFAIL: 0
# FAIL:  7
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: set-rpath.sh
==================

patching ELF file 'scratch/set-rpath/main'
new rpath is '/oops:/home/joerg/git/patchelf/tests/scratch/set-rpath/libsA:/home/joerg/git/patchelf/tests/scratch/set-rpath/libsB'
rpath is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 544
replacing section '.dynstr' with size 227
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x520/0x400520
first page is 0x400000
needed space is 1976
needed space is 2032
needed pages is 1
clearing first 4560 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 44) to offset 0x408 (size 44)
rewriting section '.gnu.hash' from offset 0x400 (size 28) to offset 0x438 (size 28)
rewriting section '.dynsym' from offset 0x420 (size 144) to offset 0x458 (size 144)
rewriting section '.dynstr' from offset 0x4b0 (size 111) to offset 0x4e8 (size 227)
rewriting section '.dynamic' from offset 0x3db0 (size 528) to offset 0x5d0 (size 544)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/set-rpath/main
./set-rpath.sh: line 22: 2780815 Segmentation fault      (core dumped) ./main
bad exit code!
FAIL set-rpath.sh (exit status: 1)

FAIL: add-rpath.sh
==================

patching ELF file 'scratch/add-rpath/main'
new rpath is '/home/joerg/git/patchelf/tests/scratch/add-rpath/libsA'
rpath is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 544
replacing section '.dynstr' with size 166
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x520/0x400520
first page is 0x400000
needed space is 1912
needed space is 1968
needed pages is 1
clearing first 4560 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 44) to offset 0x408 (size 44)
rewriting section '.gnu.hash' from offset 0x400 (size 28) to offset 0x438 (size 28)
rewriting section '.dynsym' from offset 0x420 (size 144) to offset 0x458 (size 144)
rewriting section '.dynstr' from offset 0x4b0 (size 111) to offset 0x4e8 (size 166)
rewriting section '.dynamic' from offset 0x3db0 (size 528) to offset 0x590 (size 544)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/add-rpath/main
patching ELF file 'scratch/add-rpath/main'
new rpath is '/home/joerg/git/patchelf/tests/scratch/add-rpath/libsA:/home/joerg/git/patchelf/tests/scratch/add-rpath/libsB'
rpath is too long, resizing...
replacing section '.dynstr' with size 276
this is an executable
using replaced section '.dynstr'
last replaced is 7
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
first reserved offset/addr is 0x590/0x3ff590
first page is 0x3ff000
needed space is 1536
needed space is 1592
needed pages is 1
clearing first 4616 bytes
rewriting section '.interp' from offset 0x350 (size 84) to offset 0x388 (size 84)
rewriting section '.note.gnu.property' from offset 0x3a8 (size 64) to offset 0x3e0 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3e8 (size 32) to offset 0x420 (size 32)
rewriting section '.hash' from offset 0x408 (size 44) to offset 0x440 (size 44)
rewriting section '.gnu.hash' from offset 0x438 (size 28) to offset 0x470 (size 28)
rewriting section '.dynsym' from offset 0x458 (size 144) to offset 0x490 (size 144)
rewriting section '.dynstr' from offset 0x4e8 (size 166) to offset 0x520 (size 276)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/add-rpath/main
./add-rpath.sh: line 21: 2780612 Segmentation fault      (core dumped) ./main
bad exit code!
FAIL add-rpath.sh (exit status: 1)

FAIL: no-rpath.sh
=================

patching ELF file 'scratch/no-rpath/no-rpath'
replacing section '.interp' with size 84
this is an executable
using replaced section '.interp'
last replaced is 1
looking at section '.interp'
first reserved offset/addr is 0x370/0x400370
first page is 0x400000
needed space is 880
clearing first 88 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x318 (size 84)
rewriting symbol table section 6
rewriting symbol table section 34
new rpath is '/foo:/bar:/xxxxxxxxxxxxxxx'
rpath is too long, resizing...
DT_NULL index is 26
replacing section '.dynamic' with size 528
replacing section '.dynstr' with size 99
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x4c0/0x4004c0
first page is 0x400000
needed space is 1776
needed space is 1832
needed pages is 1
clearing first 4464 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 36) to offset 0x408 (size 36)
rewriting section '.gnu.hash' from offset 0x3f8 (size 28) to offset 0x430 (size 28)
rewriting section '.dynsym' from offset 0x418 (size 96) to offset 0x450 (size 96)
rewriting section '.dynstr' from offset 0x478 (size 72) to offset 0x4b0 (size 99)
rewriting section '.dynamic' from offset 0x3dd0 (size 512) to offset 0x518 (size 528)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/no-rpath/no-rpath
./no-rpath.sh: line 21: 2780805 Segmentation fault      (core dumped) ./no-rpath
FAIL no-rpath.sh (exit status: 139)

FAIL: big-dynstr.sh
===================

patching ELF file 'scratch/big-dynstr/big-dynstr'
new rpath is '/oops:/home/joerg/git/patchelf/tests/scratch/big-dynstr/libsA:/home/joerg/git/patchelf/tests/scratch/big-dynstr/libsB'
rpath is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 544
replacing section '.dynstr' with size 229
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x520/0x400520
first page is 0x400000
needed space is 1976
needed space is 2032
needed pages is 1
clearing first 4560 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 44) to offset 0x408 (size 44)
rewriting section '.gnu.hash' from offset 0x400 (size 28) to offset 0x438 (size 28)
rewriting section '.dynsym' from offset 0x420 (size 144) to offset 0x458 (size 144)
rewriting section '.dynstr' from offset 0x4b0 (size 111) to offset 0x4e8 (size 229)
rewriting section '.dynamic' from offset 0x17db0 (size 528) to offset 0x5d0 (size 544)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/big-dynstr/big-dynstr
./big-dynstr.sh: line 22: 2780632 Segmentation fault      (core dumped) ./big-dynstr
bad exit code!
FAIL big-dynstr.sh (exit status: 1)

FAIL: set-rpath-library.sh
==========================

patching ELF file 'scratch/set-rpath-library/main-scoped'
new rpath is '/oops:/home/joerg/git/patchelf/tests/scratch/set-rpath-library/libsA:/home/joerg/git/patchelf/tests/scratch/set-rpath-library/libsB'
rpath is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 544
replacing section '.dynstr' with size 250
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x526/0x400526
first page is 0x400000
needed space is 2000
needed space is 2056
needed pages is 1
clearing first 4566 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 44) to offset 0x408 (size 44)
rewriting section '.gnu.hash' from offset 0x400 (size 28) to offset 0x438 (size 28)
rewriting section '.dynsym' from offset 0x420 (size 144) to offset 0x458 (size 144)
rewriting section '.dynstr' from offset 0x4b0 (size 118) to offset 0x4e8 (size 250)
rewriting section '.dynamic' from offset 0x3db0 (size 528) to offset 0x5e8 (size 544)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/set-rpath-library/main-scoped
./set-rpath-library.sh: line 25: 2780635 Segmentation fault      (core dumped) ./main-scoped
patching ELF file 'scratch/set-rpath-library/libsA/libfoo-scoped.so'
new rpath is '/oops:/home/joerg/git/patchelf/tests/scratch/set-rpath-library/libsB'
rpath is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 528
replacing section '.dynstr' with size 232
this is a dynamic library
last page is 0x5000
first page is 0x0
needed space is 1072
rewriting section '.hash' from offset 0x2e0 (size 56) to offset 0x5000 (size 56)
rewriting section '.gnu.hash' from offset 0x318 (size 40) to offset 0x5038 (size 40)
rewriting section '.dynsym' from offset 0x340 (size 216) to offset 0x5060 (size 216)
rewriting section '.dynstr' from offset 0x418 (size 163) to offset 0x5138 (size 232)
rewriting section '.dynamic' from offset 0x2db8 (size 512) to offset 0x5220 (size 528)
rewriting symbol table section 29
rewriting symbol table section 34
writing scratch/set-rpath-library/libsA/libfoo-scoped.so
./set-rpath-library.sh: line 37: 2780922 Segmentation fault      (core dumped) ./main-scoped
bad exit code!
FAIL set-rpath-library.sh (exit status: 1)

FAIL: output-flag.sh
====================

patching ELF file 'scratch/output-flag/main'
new rpath is '/oops:/home/joerg/git/patchelf/tests/scratch/output-flag/libsA:/home/joerg/git/patchelf/tests/scratch/output-flag/libsB'
rpath is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 544
replacing section '.dynstr' with size 231
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x520/0x400520
first page is 0x400000
needed space is 1976
needed space is 2032
needed pages is 1
clearing first 4560 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 44) to offset 0x408 (size 44)
rewriting section '.gnu.hash' from offset 0x400 (size 28) to offset 0x438 (size 28)
rewriting section '.dynsym' from offset 0x420 (size 144) to offset 0x458 (size 144)
rewriting section '.dynstr' from offset 0x4b0 (size 111) to offset 0x4e8 (size 231)
rewriting section '.dynamic' from offset 0x3db0 (size 528) to offset 0x5d0 (size 544)
rewriting symbol table section 6
rewriting symbol table section 34
writing scratch/output-flag/main2
patching ELF file 'scratch/output-flag/main2'
not modified, but alwaysWrite=true
writing scratch/output-flag/main3
./output-flag.sh: line 25: 2781044 Segmentation fault      (core dumped) ./main2
bad exit code!
FAIL output-flag.sh (exit status: 1)

FAIL: replace-add-needed.sh
===========================

patching ELF file './libbar.so'
new SONAME is 'libbar.so'
SONAME is too long, resizing...
DT_NULL index is 27
replacing section '.dynamic' with size 528
replacing section '.dynstr' with size 170
this is a dynamic library
last page is 0x5000
first page is 0x0
needed space is 960
rewriting section '.hash' from offset 0x270 (size 48) to offset 0x5000 (size 48)
rewriting section '.gnu.hash' from offset 0x2a0 (size 36) to offset 0x5030 (size 36)
rewriting section '.dynsym' from offset 0x2c8 (size 168) to offset 0x5058 (size 168)
rewriting section '.dynstr' from offset 0x370 (size 160) to offset 0x5100 (size 170)
rewriting section '.dynamic' from offset 0x2dc0 (size 512) to offset 0x51b0 (size 528)
rewriting symbol table section 28
rewriting symbol table section 33
writing ./libbar.so
patching ELF file './simple'
DT_NULL index is 26
replacing section '.dynamic' with size 528
replacing section '.dynstr' with size 83
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 21
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.gnu.version'
first reserved offset/addr is 0x4c0/0x4004c0
first page is 0x400000
needed space is 1760
needed space is 1816
needed pages is 1
clearing first 4464 bytes
rewriting section '.interp' from offset 0x318 (size 84) to offset 0x350 (size 84)
rewriting section '.note.gnu.property' from offset 0x370 (size 64) to offset 0x3a8 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3b0 (size 32) to offset 0x3e8 (size 32)
rewriting section '.hash' from offset 0x3d0 (size 36) to offset 0x408 (size 36)
rewriting section '.gnu.hash' from offset 0x3f8 (size 28) to offset 0x430 (size 28)
rewriting section '.dynsym' from offset 0x418 (size 96) to offset 0x450 (size 96)
rewriting section '.dynstr' from offset 0x478 (size 72) to offset 0x4b0 (size 83)
rewriting section '.dynamic' from offset 0x3dd0 (size 512) to offset 0x508 (size 528)
rewriting symbol table section 6
rewriting symbol table section 34
writing ./simple
patching ELF file './libfoo.so'
replacing DT_NEEDED entry 'libbar.so' with '/home/joerg/git/patchelf/tests/scratch/replace-add-needed/libbar.so'
resizing .dynstr ...
keeping DT_NEEDED entry 'libc.so.6'
found .gnu.version_r with 1 entries, strings in .dynstr
keeping .gnu.version_r entry 'libc.so.6'
replacing section '.dynstr' with size 224
this is a dynamic library
last page is 0x5000
first page is 0x0
needed space is 536
rewriting section '.hash' from offset 0x2e0 (size 56) to offset 0x5000 (size 56)
rewriting section '.gnu.hash' from offset 0x318 (size 40) to offset 0x5038 (size 40)
rewriting section '.dynsym' from offset 0x340 (size 216) to offset 0x5060 (size 216)
rewriting section '.dynstr' from offset 0x418 (size 156) to offset 0x5138 (size 224)
rewriting symbol table section 30
rewriting symbol table section 35
writing ./libfoo.so
patching ELF file './simple'
replacing DT_NEEDED entry 'libbar.so' with '/home/joerg/git/patchelf/tests/scratch/replace-add-needed/libbar.so'
resizing .dynstr ...
replacing DT_NEEDED entry 'libc.so.6' with '/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libc.so.6'
resizing .dynstr ...
found .gnu.version_r with 1 entries, strings in .dynstr
replacing .gnu.version_r entry 'libc.so.6' with '/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libc.so.6'
replacing section '.dynstr' with size 224
this is an executable
using replaced section '.dynstr'
last replaced is 7
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
first reserved offset/addr is 0x508/0x3ff508
first page is 0x3ff000
needed space is 1424
needed space is 1480
needed pages is 1
clearing first 4480 bytes
rewriting section '.interp' from offset 0x350 (size 84) to offset 0x388 (size 84)
rewriting section '.note.gnu.property' from offset 0x3a8 (size 64) to offset 0x3e0 (size 64)
rewriting section '.note.ABI-tag' from offset 0x3e8 (size 32) to offset 0x420 (size 32)
rewriting section '.hash' from offset 0x408 (size 36) to offset 0x440 (size 36)
rewriting section '.gnu.hash' from offset 0x430 (size 28) to offset 0x468 (size 28)
rewriting section '.dynsym' from offset 0x450 (size 96) to offset 0x488 (size 96)
rewriting section '.dynstr' from offset 0x4b0 (size 83) to offset 0x4e8 (size 224)
rewriting symbol table section 6
rewriting symbol table section 34
DT_NULL index is 27
replacing section '.dynamic' with size 544
replacing section '.dynstr' with size 293
this is an executable
using replaced section '.dynstr'
using replaced section '.dynamic'
last replaced is 8
looking at section '.interp'
replacing section '.interp' which is in the way
looking at section '.note.gnu.property'
replacing section '.note.gnu.property' which is in the way
looking at section '.note.ABI-tag'
replacing section '.note.ABI-tag' which is in the way
looking at section '.hash'
replacing section '.hash' which is in the way
looking at section '.gnu.hash'
replacing section '.gnu.hash' which is in the way
looking at section '.dynsym'
replacing section '.dynsym' which is in the way
looking at section '.dynstr'
looking at section '.dynamic'
first reserved offset/addr is 0x1508/0x3ff508
first page is 0x3fe000
needed space is 2096
clearing first 4480 bytes
rewriting section '.interp' from offset 0x388 (size 84) to offset 0x388 (size 84)
rewriting section '.note.gnu.property' from offset 0x3e0 (size 64) to offset 0x3e0 (size 64)
rewriting section '.note.ABI-tag' from offset 0x420 (size 32) to offset 0x420 (size 32)
rewriting section '.hash' from offset 0x440 (size 36) to offset 0x440 (size 36)
rewriting section '.gnu.hash' from offset 0x468 (size 28) to offset 0x468 (size 28)
rewriting section '.dynsym' from offset 0x488 (size 96) to offset 0x488 (size 96)
rewriting section '.dynstr' from offset 0x4e8 (size 224) to offset 0x4e8 (size 293)
rewriting section '.dynamic' from offset 0x1508 (size 528) to offset 0x610 (size 544)
rewriting symbol table section 6
rewriting symbol table section 34
writing ./simple
./replace-add-needed.sh: line 33: 2781422 Segmentation fault      (core dumped) ./simple
FAIL replace-add-needed.sh (exit status: 1)

Since nix is also just using normal binutils, glibc und gcc, I think those crashes are valid and might also affect ubuntu eventually or already under certain circumstances. If you have the bandwidth and motivation, I can give you access to a nixos machine: https://github.com/nix-community/infra#community-builder
There are also a few other options on how to run nix on other operating systems (including ubuntu): https://nix.dev/tutorials/install-nix

Bo98 · 2022-12-29T03:42:44Z

src/patchelf.cc

-    wri(phdr.p_filesz, wri(phdr.p_memsz, splitShift + shift));
-    wri(phdr.p_flags, PF_R | PF_W);
+    wri(phdr.p_filesz, wri(phdr.p_memsz, splitShift + extraBytes));
+    wri(phdr.p_flags, splitFlags);


This flags change made sense to me initially too but can be incorrect, since we can move a .dynamic section into here which needs write permissions, hence the hard RW - anything less than RW can actually often break and there should be no executable code here to need the X bit. I suppose ideally we'd harden this by splitting this more fine grained so we only give write permissions to sections that need it, but that's a more complex fix since we'd need to separate this into 3 pages. Worthwhile to probably do at some point though.

This change might be the cause of some CI failures.

OK, that makes sense. I am admittedly not an expert in the intricacies of the ELF format, so I wasn't aware of this. When I'm able to look at this again (maybe not until next week), I will back out the change to the flags.

I reverted the change to how the segment flags are handled. I'm not sure how to re-run the CI build though.

Bo98 · 2022-12-29T03:53:28Z

src/patchelf.cc

@@ -512,8 +516,8 @@ void ElfFile<ElfFileParamNames>::shiftFile(unsigned int extraPages, size_t start
    wri(phdr.p_offset, phdrs.at(splitIndex).p_offset - splitShift - shift);
    wri(phdr.p_paddr, phdrs.at(splitIndex).p_paddr - splitShift - shift);
    wri(phdr.p_vaddr, phdrs.at(splitIndex).p_vaddr - splitShift - shift);
-    wri(phdr.p_filesz, wri(phdr.p_memsz, splitShift + shift));
-    wri(phdr.p_flags, PF_R | PF_W);
+    wri(phdr.p_filesz, wri(phdr.p_memsz, splitShift + extraBytes));


Just to check: can you post what the sections look like for you after this change?

In particular, my head says the address of the moved section needs adjusting too, but I might not be reading this correctly so comparing the output before & after would be useful.

Yeah, I should have made that more clear in the beginning. Here is the output of readelf -l <file> for the three cases (again, from a Python 3.10 executable that I built from source)

Unmodified executable used as input to patchelf

Elf file type is EXEC (Executable file) Entry point 0x4006a0 There are 9 program headers, starting at offset 64 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040 0x00000000000001f8 0x00000000000001f8 R 0x8 INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238 0x000000000000001c 0x000000000000001c R 0x1 [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 0x0000000000000948 0x0000000000000948 R E 0x200000 LOAD 0x0000000000000d90 0x0000000000600d90 0x0000000000600d90 0x0000000000000294 0x0000000000000298 RW 0x200000 DYNAMIC 0x0000000000000da0 0x0000000000600da0 0x0000000000600da0 0x0000000000000240 0x0000000000000240 RW 0x8 NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254 0x0000000000000044 0x0000000000000044 R 0x4 GNU_EH_FRAME 0x0000000000000828 0x0000000000400828 0x0000000000400828 0x000000000000003c 0x000000000000003c R 0x4 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10 GNU_RELRO 0x0000000000000d90 0x0000000000600d90 0x0000000000600d90 0x0000000000000270 0x0000000000000270 R 0x1 Section to Segment mapping: Segment Sections... 00 01 .interp 02 .interp .note.ABI-tag .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 03 .init_array .fini_array .dynamic .got .got.plt .data .bss 04 .dynamic 05 .note.ABI-tag .note.gnu.build-id 06 .eh_frame_hdr 07 08 .init_array .fini_array .dynamic .got

Output of patchelf master branch

This is the version that segfaults upon execution.

Elf file type is EXEC (Executable file) Entry point 0x4006a0 There are 11 program headers, starting at offset 64 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align PHDR 0x0000000000000040 0x00000000003ff040 0x00000000003ff040 0x0000000000000268 0x0000000000000268 R 0x8 GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10 LOAD 0x0000000000000000 0x00000000003ff000 0x00000000003ff000 0x0000000000001594 0x0000000000001594 RW 0x1000 INTERP 0x00000000000005e0 0x00000000003ff5e0 0x00000000003ff5e0 0x000000000000001c 0x000000000000001c R 0x1 [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] NOTE 0x0000000000000600 0x00000000003ff600 0x00000000003ff600 0x0000000000000020 0x0000000000000020 R 0x4 NOTE 0x0000000000000620 0x00000000003ff620 0x00000000003ff620 0x0000000000000024 0x0000000000000024 R 0x4 LOAD 0x0000000000001594 0x0000000000400594 0x0000000000400594 0x00000000000003b4 0x00000000000003b4 R E 0x1000 GNU_EH_FRAME 0x0000000000001828 0x0000000000400828 0x0000000000400828 0x000000000000003c 0x000000000000003c R 0x4 LOAD 0x0000000000001d90 0x0000000000600d90 0x0000000000600d90 0x0000000000000294 0x0000000000000298 RW 0x1000 GNU_RELRO 0x0000000000001d90 0x0000000000600d90 0x0000000000600d90 0x0000000000000270 0x0000000000000270 R 0x1 DYNAMIC 0x0000000000001da0 0x0000000000600da0 0x0000000000600da0 0x0000000000000240 0x0000000000000240 RW 0x8 Section to Segment mapping: Segment Sections... 00 01 02 .dynstr .dynsym .gnu.hash .interp .note.ABI-tag .note.gnu.build-id 03 .interp 04 .note.ABI-tag 05 .note.gnu.build-id 06 .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 07 .eh_frame_hdr 08 .init_array .fini_array .dynamic .got .got.plt .data .bss 09 .init_array .fini_array .dynamic .got 10 .dynamic

Output of patchelf from this branch

Elf file type is EXEC (Executable file) Entry point 0x4006a0 There are 11 program headers, starting at offset 64 Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flags Align PHDR 0x0000000000000040 0x00000000003ff040 0x00000000003ff040 0x0000000000000268 0x0000000000000268 R 0x8 GNU_STACK 0x0000000000001000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 RW 0x10 LOAD 0x0000000000000000 0x00000000003ff000 0x00000000003ff000 0x0000000000001000 0x0000000000001000 RW 0x1000 INTERP 0x00000000000005e0 0x00000000003ff5e0 0x00000000003ff5e0 0x000000000000001c 0x000000000000001c R 0x1 [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] NOTE 0x0000000000000600 0x00000000003ff600 0x00000000003ff600 0x0000000000000020 0x0000000000000020 R 0x4 NOTE 0x0000000000000620 0x00000000003ff620 0x00000000003ff620 0x0000000000000024 0x0000000000000024 R 0x4 LOAD 0x0000000000001000 0x0000000000400000 0x0000000000400000 0x0000000000000948 0x0000000000000948 R E 0x1000 GNU_EH_FRAME 0x0000000000001828 0x0000000000400828 0x0000000000400828 0x000000000000003c 0x000000000000003c R 0x4 LOAD 0x0000000000001d90 0x0000000000600d90 0x0000000000600d90 0x0000000000000294 0x0000000000000298 RW 0x1000 GNU_RELRO 0x0000000000001d90 0x0000000000600d90 0x0000000000600d90 0x0000000000000270 0x0000000000000270 R 0x1 DYNAMIC 0x0000000000001da0 0x0000000000600da0 0x0000000000600da0 0x0000000000000240 0x0000000000000240 RW 0x8 Section to Segment mapping: Segment Sections... 00 01 02 .dynstr .dynsym .gnu.hash .interp .note.ABI-tag .note.gnu.build-id 03 .interp 04 .note.ABI-tag 05 .note.gnu.build-id 06 .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 07 .eh_frame_hdr 08 .init_array .fini_array .dynamic .got .got.plt .data .bss 09 .init_array .fini_array .dynamic .got 10 .dynamic

Bo98 · 2022-12-29T04:13:29Z

The Linux kernel's ELF loader (at least as of v4.18) does not accept this as a valid ELF executable

It's a weird one because I've never been able to reproduce the issue you mentioned - it seems like a certain kernel option is needed or something to make it actually reject the ELF, since it doesn't happen on my default Ubuntu install using an identical binary (sha256-matching) to what doesn't work for some others. I even deployed the latest patchelf to a few thousand users and the only feedback was from RHEL users. WSL2 too initially, but Microsoft already had a beta with the issue fixed which has now been rolled out for everyone.

Very odd, but at least you are able to reproduce it. I agree the general fix would be to create some empty space between the sections just to make sure they're on different pages since permissions etc work under that assumption - it seems like some of the WSL2 users who weren't affected are those who have it where the two sections are the same permission.

otherjason · 2022-12-29T04:17:56Z

The Linux kernel's ELF loader (at least as of v4.18) does not accept this as a valid ELF executable

It's a weird one because I've never been able to reproduce the issue you mentioned - it seems like a certain kernel option is needed or something to make it actually reject the ELF, since it doesn't happen on my default Ubuntu install using an identical binary (sha256-matching) to what doesn't work for some others. I even deployed the latest patchelf to a few thousand users and the only feedback was from RHEL users. WSL2 too initially, but Microsoft already had a beta with the issue fixed which has now been rolled out for everyone.

Very odd, but at least you are able to reproduce it. I agree the general fix would be to create some empty space between the sections just to make sure they're on different pages since I think permissions etc work under that assumption.

I've seen the same; I'm not sure what combination of kernel version/configuration triggers it, but I too have only observed it on RHEL (and its free derivatives).

… the original flags in both" This reverts commit f4f1848.

otherjason · 2022-12-29T04:52:14Z

@Mic92 We'll see if any of the CI failures clear up from reverting the flags change that @Bo98 pointed out above might be problematic. If there are still issues, then I will take you up on the offer to use the NixOS machine to try to get to the bottom of the issue.

rpurdie · 2023-01-10T14:12:27Z

Yocto project ran into this issue with patchelf 0.16.1 and 0.17.0 with many different binaries segfaulting. Applying the patch here seems to help.

Mic92 · 2023-01-10T17:28:31Z

bors merge

Mic92 · 2023-01-10T17:28:55Z

Sorry for the delay. I will make a patch release with this.

bors · 2023-01-10T17:32:36Z

Build succeeded:

Mic92 · 2023-01-10T18:22:56Z

I have created a new release for this.

ZhongRuoyu · 2023-01-10T18:59:13Z

I have created a new release for this.

Thanks for creating the release. Maybe the version file should be updated? It shows 0.18.0-alpha, so the assets on the 0.17.1 release page are named incorrectly.

Mic92 · 2023-01-10T19:46:49Z

Oh, shoot. I released 0.17.2 to fix this.

ZhongRuoyu · 2023-01-11T00:15:42Z

Thanks!

Jason added 2 commits December 2, 2022 09:58

shiftFile: when splitting a segment into two pieces, preserve the ori…

f4f1848

…ginal flags in both

This was referenced Dec 2, 2022

0.17.0: Segmentation fault after modifying RPATH #446

Open

Invalid header generated when modifying rpath with v1.6.1 #426

Open

mayeut mentioned this pull request Dec 10, 2022

patchelf 0.17.0 can lead to broken repairs pypa/manylinux#1421

Open

Bo98 reviewed Dec 29, 2022

View reviewed changes

Revert "shiftFile: when splitting a segment into two pieces, preserve…

3a6d771

… the original flags in both" This reverts commit f4f1848.

bors bot merged commit 583fd5a into NixOS:master Jan 10, 2023

jwhite007 mentioned this pull request Feb 14, 2023

Several bottles resulting in "segmentation fault" on Red Hat Enterprise Linux release 8.6 Homebrew/homebrew-core#116841

Closed

rcoup mentioned this pull request Apr 25, 2023

patchelf: update to v0.18.0 microsoft/vcpkg#31107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split segment size fix #447

Split segment size fix #447

otherjason commented Dec 2, 2022 •

edited

Loading

Mic92 commented Dec 14, 2022

Mic92 commented Dec 14, 2022

otherjason commented Dec 23, 2022

Mic92 commented Dec 24, 2022 •

edited

Loading

Bo98 Dec 29, 2022 •

edited

Loading

otherjason Dec 29, 2022

otherjason Dec 29, 2022

Bo98 Dec 29, 2022 •

edited

Loading

otherjason Dec 29, 2022

Bo98 commented Dec 29, 2022 •

edited

Loading

otherjason commented Dec 29, 2022

otherjason commented Dec 29, 2022

rpurdie commented Jan 10, 2023

Mic92 commented Jan 10, 2023

Mic92 commented Jan 10, 2023

bors bot commented Jan 10, 2023

Mic92 commented Jan 10, 2023

ZhongRuoyu commented Jan 10, 2023

Mic92 commented Jan 10, 2023

ZhongRuoyu commented Jan 11, 2023

Split segment size fix #447

Split segment size fix #447

Conversation

otherjason commented Dec 2, 2022 • edited Loading

Fix for #446

Fix for incorrect segment flags

Mic92 commented Dec 14, 2022

Mic92 commented Dec 14, 2022

otherjason commented Dec 23, 2022

Mic92 commented Dec 24, 2022 • edited Loading

Bo98 Dec 29, 2022 • edited Loading

Choose a reason for hiding this comment

otherjason Dec 29, 2022

Choose a reason for hiding this comment

otherjason Dec 29, 2022

Choose a reason for hiding this comment

Bo98 Dec 29, 2022 • edited Loading

Choose a reason for hiding this comment

otherjason Dec 29, 2022

Choose a reason for hiding this comment

Unmodified executable used as input to patchelf

Output of patchelf master branch

Output of patchelf from this branch

Bo98 commented Dec 29, 2022 • edited Loading

otherjason commented Dec 29, 2022

otherjason commented Dec 29, 2022

rpurdie commented Jan 10, 2023

Mic92 commented Jan 10, 2023

Mic92 commented Jan 10, 2023

bors bot commented Jan 10, 2023

Mic92 commented Jan 10, 2023

ZhongRuoyu commented Jan 10, 2023

Mic92 commented Jan 10, 2023

ZhongRuoyu commented Jan 11, 2023

otherjason commented Dec 2, 2022 •

edited

Loading

Mic92 commented Dec 24, 2022 •

edited

Loading

Bo98 Dec 29, 2022 •

edited

Loading

Bo98 Dec 29, 2022 •

edited

Loading

Unmodified executable used as input to `patchelf`

Output of `patchelf` master branch

Output of `patchelf` from this branch

Bo98 commented Dec 29, 2022 •

edited

Loading