Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for dylibs with Address Sanitizer #42711

Merged
merged 1 commit into from
Jul 17, 2017

Conversation

Firstyear
Copy link
Contributor

Many applications use address sanitizer to assert correct behaviour of their programs. When using Rust with C, it's much more important to assert correct programs with tools like asan/lsan due to the unsafe nature of the access across an ffi boundary. However, previously only rust bin types could use asan. This posed a challenge for existing C applications that link or dlopen .so when the C application is compiled with asan.

This PR enables asan to be linked to the dylib and cdylib crate type. We alter the test to check the proc-macro crate does not work with -Z sanitizer=address. Finally, we add a test that compiles a shared object in rust, then another rust program links it and demonstrates a crash through the call to the library.

This PR is nearly complete, but I do require advice on the change to fix the -lasan that currently exists in the dylib test. This is required because the link statement is not being added correctly to the rustc build when -Z sanitizer=address is added (and I'm not 100% sure why)

Thanks,

@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nikomatsakis (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@kennytm
Copy link
Member

kennytm commented Jun 17, 2017

Undefined symbol __asan_init_v5 in run-make/sanitizer-dylib-link.

[00:46:44] error: linking with `cc` failed: exit code: 1
[00:46:44]   |
[00:46:44]   = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu/program.0.o" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu/program" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "asan" "-l" "library" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-35ad9950c7e5074b.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/librand-20a50a22d4c2b1e9.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-fb44afc024bbc636.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-14b8f3202acdad6a.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-10b591f1a68dd370.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_unicode-f4f0ae88f5ad8ad4.rlib" "-Wl,--whole-archive" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu/rustc.IhCKfBkAHvHJ/librustc_asan-e61eb319c4489660.rlib" "-Wl,--no-whole-archive" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_system-06d737faf6076d7c.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-6ecacccb5bdc4911.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-bfaa82017ca17cb2.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-863b57a66ba6c3e1.rlib" "-Wl,-Bdynamic" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util"
[00:46:44]   = note: /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu/program.0.o: In function `asan.module_ctor':
[00:46:44]           program.cgu-0.rs:(.text.asan.module_ctor+0x2): undefined reference to `__asan_init_v5'
[00:46:44]           /usr/bin/ld generated: undefined reference to `__asan_init_v5'
[00:46:44]           collect2: error: ld returned 1 exit status
[00:46:44]           
[00:46:44] 
[00:46:44] error: aborting due to previous error(s)

Probably because LLVM 3.7 is too old (ABI changed in https://reviews.llvm.org/D11004?id=29191 ?)

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 17, 2017
@Firstyear
Copy link
Contributor Author

Possibly. I was worried the manual linking in the rustc process in the test could be part of the issue (note the -lasan).

@arielb1 arielb1 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 20, 2017
@Firstyear
Copy link
Contributor Author

I'm not sure why the "waiting on review" flag was removed. I explicitly asked for a review, because I need advice from a more experienced developer to help fix the issue from the CI regarding the fact that I have to force add -lasan at the moment (it should be getting added without that).

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 21, 2017
@Mark-Simulacrum
Copy link
Member

Sorry about that. I've readded it.

@cuviper
Copy link
Member

cuviper commented Jun 22, 2017

I think run-make/sanitizer-address/Makefile is a better model for you to follow, notable for how it checks for SANITIZER_SUPPORT. The travis-ci build using llvm3.7 does not set --enable-sanitizers, so these tests should get bypassed in that case.

You shouldn't need -lasan, pulling in gcc's libasan. If you built with --enable-sanitizers then librustc_asan.rlib will have all the necessary symbols matching LLVM's own ABI.

With that removed, I get a different fun error:

  = note: /usr/bin/ld: /[...]/librustc_asan-542cca8c7ea6387a.rlib(asan_preinit.cc.o): .preinit_array section is not allowed in DSO
          /usr/bin/ld: failed to set dynamic section sizes: Nonrepresentable section on output
          collect2: error: ld returned 1 exit status

@Firstyear
Copy link
Contributor Author

This latest commit resolves the test issues that were mentioned by @cuviper as well as resolving the ld issue that was noted. I incorrectly added the runtime of asan to dylibs when it was not required. Observing the test outputs now shows:

 LD_LIBRARY_PATH=/home/william/development/rust/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu /home/william/development/rust/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-dylib-link.stage2-x86_64-unknown-linux-gnu/program
=================================================================
==14626==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffd83a1c970 at pc 0x7f32a1ca64d0 bp 0x7ffd83a1c930 sp 0x7ffd83a1c928
READ of size 4 at 0x7ffd83a1c970 thread T0
    #0 0x7f32a1ca64cf in overflow /home/william/development/rust/src/test/run-make/sanitizer-dylib-link/library.rs:14
    #1 0x7f32a23187e8 in program::main::h06fb8338f7689bca /home/william/development/rust/src/test/run-make/sanitizer-dylib-link/program.rs:16
    #2 0x7f32a1ce034e in core::ops::function::FnOnce::call_once::h77fe44d6fc2de6f5 /home/william/development/rust/src/libcore/ops/function.rs:143

...

Additionally, looking at the elf symbols we can see the correct list of asan symbols in the dylib:

readelf -Ws liblibrary.so| grep -i asan
     2: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_load8
    10: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_load4
    20: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_unregister_globals
    37: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_register_globals
    43: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_stack_malloc_0
    59: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_store4
    61: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_store8
   126: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_init
   131: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_stack_malloc_1
   142: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_version_mismatch_check_v8
   169: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_option_detect_stack_use_after_return
    54: 000000000006f530    32 FUNC    LOCAL  DEFAULT   12 asan.module_ctor
    55: 000000000006f550    22 FUNC    LOCAL  DEFAULT   12 asan.module_dtor
 10963: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_load8
 11057: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_load4
 11144: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_unregister_globals
 11459: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_register_globals
 11544: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_stack_malloc_0
 11724: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_store4
 11728: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_report_store8
 12783: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_init
 12841: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_stack_malloc_1
 12993: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_version_mismatch_check_v8
 13442: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND __asan_option_detect_stack_use_after_return

Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks...pretty simple. I left some nits. I'm not sure I'm the best reviewer though, may seek out a 2nd opinion.

@@ -861,8 +861,11 @@ impl<'a> CrateLoader<'a> {
// This crate will be compiled with the required
// instrumentation pass
config::CrateTypeRlib => false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: use |, like

config::CrateTypeRlib |
config::CrateTypeDylib |
config::CrateTypeCdylib |
config::CrateTypeStaticLib =>
    false,
_ => {
    err(...)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we testing all of these modes?

@arielb1 arielb1 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 27, 2017
@nikomatsakis
Copy link
Contributor

r? @alexcrichton

@alexcrichton
Copy link
Member

Thanks for the PR @Firstyear! It looks like this also adds staticlib/cdylib support as well, right? Mind adding tests for those as well?

@Firstyear
Copy link
Contributor Author

Hey @nikomatsakis and @alexcrichton

Thanks for your reviews! I've fixed both nits raised in my branch. I'm running the tests now to be sure they work.

With the staticlib test, I can't think of a good way to do this "just" with rust. Would it be acceptable to use a small C program and link with the static lib to check this? Or do you have a better suggestion as to how I could write this test?

Thanks again,

@alexcrichton
Copy link
Member

Ah yeah having a small C shim is fine. You can poke around some other run-make tests perhaps? There should be some other examples of using staticlib there as well I believe.

@Firstyear
Copy link
Contributor Author

Thanks for that @alexcrichton. I have made the C shim using another test as an example to follow. I have also corrected the nits raised by @nikomatsakis . Looking forward to your review, thanks!

@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Jul 1, 2017

📌 Commit 96e8e34 has been approved by alexcrichton

@bors
Copy link
Contributor

bors commented Jul 1, 2017

⌛ Testing commit 96e8e34 with merge ccabdd95ea6594dcce40309e83322882fb69cd80...

@kennytm
Copy link
Member

kennytm commented Jul 12, 2017

@Mark-Simulacrum the initial usage in the previous successful build was 13G/30G. Why suddenly bloated by 7G? It seems #43026 failed with the same reason.

Edit: the last successful build (13G) runs on "Connie", the following ones run on "Sugilite", which probably explains the 7G increase. There's still no reason why "Sugilite" was chosen.

@bors
Copy link
Contributor

bors commented Jul 12, 2017

💔 Test failed - status-travis

@Mark-Simulacrum
Copy link
Member

@bors retry

@bors
Copy link
Contributor

bors commented Jul 12, 2017

⌛ Testing commit 9d339f4 with merge bbcedacb5a75d3388763996040ba9183d433b264...

@kennytm
Copy link
Member

kennytm commented Jul 12, 2017

@Mark-Simulacrum @alexcrichton looks like we need to change .travis.yml to language: shell due to travis-ci/packer-templates#454. travis-ci/docs-travis-ci-com#1267 (comment)

@bors
Copy link
Contributor

bors commented Jul 12, 2017

💔 Test failed - status-travis

@Mark-Simulacrum
Copy link
Member

@kennytm Would you like to file a PR? I will in a few hours when I get around to it if not..

@kennytm
Copy link
Member

kennytm commented Jul 12, 2017

@Mark-Simulacrum I'm on mobile right now, won't reach a real computer in the next few hours, so no, sorry.

@Firstyear
Copy link
Contributor Author

I'm assuming this means I just sit here and wait for the issue to be resolved by @kennytm ? Thanks again,

@kennytm
Copy link
Member

kennytm commented Jul 13, 2017

#43198 has been merged, this PR can be retried now.

@Mark-Simulacrum
Copy link
Member

@bors retry

@bors
Copy link
Contributor

bors commented Jul 13, 2017

⌛ Testing commit 9d339f4 with merge af67663687882a8908095d850d44dcc3318ae451...

@alexcrichton
Copy link
Member

@bors: retry

  • prioritizing beta backports

@bors
Copy link
Contributor

bors commented Jul 14, 2017

⌛ Testing commit 9d339f4 with merge 8fb712dedc39c431448817fd6e25640a88bb7c17...

@bors
Copy link
Contributor

bors commented Jul 14, 2017

💔 Test failed - status-travis

@kennytm
Copy link
Member

kennytm commented Jul 14, 2017

x86_64-gnu's run-make/sanitizer-invalid-cratetype failed, legit.

The command should end with grep -q -- '-Z sanitizer' (note two dashes)

[00:57:39] failures:
[00:57:39] 
[00:57:39] ---- [run-make] run-make/sanitizer-invalid-cratetype stdout ----
[00:57:39] 	
[00:57:39] error: make failed
[00:57:39] status: exit code: 2
[00:57:39] command: "make"
[00:57:39] stdout:
[00:57:39] ------------------------------------------
[00:57:39] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-invalid-cratetype.stage2-x86_64-unknown-linux-gnu:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-invalid-cratetype.stage2-x86_64-unknown-linux-gnu -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/sanitizer-invalid-cratetype.stage2-x86_64-unknown-linux-gnu  -Z sanitizer=address --crate-type proc-macro --target x86_64-unknown-linux-gnu hello.rs 2>&1 | grep -q '-Z sanitizer'
[00:57:39] Makefile:17: recipe for target 'all' failed
[00:57:39] 
[00:57:39] ------------------------------------------
[00:57:39] stderr:
[00:57:39] ------------------------------------------
[00:57:39] grep: invalid option -- ' '
[00:57:39] Usage: grep [OPTION]... PATTERN [FILE]...
[00:57:39] Try 'grep --help' for more information.
[00:57:39] make: *** [all] Error 2
[00:57:39] 
[00:57:39] ------------------------------------------

@Firstyear
Copy link
Contributor Author

It worked for me on my system .... sigh, okay, new update with the '--' coming in, which also passes for me on my system.

@Firstyear
Copy link
Contributor Author

error: RPC failed; curl 56 SSLRead() return error -36

fatal: The remote end hung up unexpectedly

fatal: early EOF

fatal: index-pack failed

�[31;1mThe command "eval git fetch origin +refs/pull/42711/merge: " failed 3 times.�[0m

travis_time:end:17d523f4:start=1500071550625829000,finish=1500071687585018000,duration=136959189000
�[0K

�[31;1mThe command "git fetch origin +refs/pull/42711/merge:" failed and exited with 128 during .�[0m

Yeah, that wasn't my fault ;) Anyway to make this run again?

@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Jul 17, 2017

📌 Commit 0af5c00 has been approved by alexcrichton

@bors
Copy link
Contributor

bors commented Jul 17, 2017

⌛ Testing commit 0af5c00 with merge 2652ce6...

bors added a commit that referenced this pull request Jul 17, 2017
Add support for dylibs with Address Sanitizer

Many applications use address sanitizer to assert correct behaviour of their programs. When using Rust with C, it's much more important to assert correct programs with tools like asan/lsan due to the unsafe nature of the access across an ffi boundary. However, previously only rust bin types could use asan. This posed a challenge for existing C applications that link or dlopen .so when the C application is compiled with asan.

This PR enables asan to be linked to the dylib and cdylib crate type. We alter the test to check the proc-macro crate does not work with -Z sanitizer=address. Finally, we add a test that compiles a shared object in rust, then another rust program links it and demonstrates a crash through the call to the library.

This PR is nearly complete, but I do require advice on the change to fix the -lasan that currently exists in the dylib test. This is required because the link statement is not being added correctly to the rustc build when -Z sanitizer=address is added (and I'm not 100% sure why)

Thanks,
@bors
Copy link
Contributor

bors commented Jul 17, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 2652ce6 to master...

@bors bors merged commit 0af5c00 into rust-lang:master Jul 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants