Testing infrastructure

[gz]

Ideally I'd like have an infrastructure for unit testing at least some parts of this crate. Potential candidates to use are qemu or directly @dschatzberg kvm bindings (https://github.com/dschatzberg/kvm).

[Ericson2314]

Definitely! I've been thinking about in the future custom test runners could make this really nice, but we can hack something together now.

[gz]

Indeed, do you happen to know if there is an RFC for custom test runners somewhere around?

[Ericson2314]

https://internals.rust-lang.org/t/test-and-external-test-harnesses/3145 this is all I know of.

[Ericson2314]

Well my rust-lang/rfcs#1133 might make test a real dep, but that only covers the runtime side of things.

[gz]

Thanks, I definitely look forward to rust-lang/rfcs#1133 this should simplify things quite a bit.

[dschatzberg]

It should be fairly straightforward to use the KVM bindings though that crate itself has been lightly tested. I think we can just use the native test harness as long as the test runner has access to /dev/kvm

[Ericson2314]

Let's get the ball rolling! QuiltOS@0a52a0d

[Ericson2314]

If anybody can figure out how to remove the mmap that would be cool. I'd like to avoid writing a crude linker :).

[dschatzberg]

In the past, I've mapped in the entire process using /proc/self/maps. That way you can write the tests in and use their address as normal. I'm traveling for the next couple days but happy to help when I'm back

[dschatzberg]

Still a bit buried for the next little while. I started on mapping the binary in via /proc/self/maps:
dschatzberg@71313a2

It's not working yet since the VCPU needs to be in long mode and therefore also needs a page table, but that should all work without needing to link it specially.

[gz]

I have some time issues as well on my side right now but this looks awesome, thanks! 👍

[gz]

I did quick&dirty initial virtual space layout by doing an identity map (which I assume I get away with for now :O?) and trying to set cr0/cr3/cr4 to what they should be. But to be honest, I'm not sure what you actually need to do here: Does it need to run some bootstrap code inside the VM to go in long mode or can it be set-up all externally? AFAIK it also needs a MSR write to enable the IA32-e paging...

[dschatzberg]

It can all be done externally. I worked on mine a bit and it works if I manually write in the inb instruction, but not if I use the naked function. I'm guessing I got my paging map incorrect but maybe another pair of eyes can help. Feel free to harvest from that code

dschatzberg@299d99c

[dschatzberg]

You'll see in my code how I set the control registers and the EFER MSR to enable paging, long mode, etc. Also the CS is set to long-mode and CR3 points to a (supposed) valid PML4 which was supposed to identity map the entire lower half of the address space.

[gz]

Got it :) that will help, thanks!

[Ericson2314]

I was working on this yesterday: https://github.com/QuiltOS/rust-x86 for some misc cleanups.

I think on my machine at least the program is being loaded into the upper half, which would be the problem. It's a chore, but I guess the solution is to map in each memmap as contiguous physical memory, and then page it in to its proper place. Haven't done that yet.

[Ericson2314]

https://github.com/japaric/utest @japaric might have just gotten this unstuck!

[gz]

Indeed, this looks pretty good!

[gz]

Btw. I think the reason we can't access the .text code pages seems to be because it's mapped at a very high host virtual address (rust binaries are weird, .text starts at ~0x560c8dd7d000). I did a few tests and it seems that every mapping with host virtual above 1<<40 is not readable by KVM (maybe because it restricts the guest physical address space to 40 bits?).

[gz]

Alright, some more tinkering it seems to execute the naked function now, when compiled with relocation-model=static:

RUSTFLAGS="-C relocation-model=static" RUST_BACKTRACE=1 cargo test --verbose --test kvm -- --nocapture

7acde70#diff-653c9c90e75858d15c1f62283a1270a1

[gz]

Alright, it seems the technique they use in utest also works without no_std, you simply override the default test crate in your Cargo.toml. This together with procedural macros should let us build something quite nice to automatically set-up the virtual CPU according to the test requirements etc. 👍

[Ericson2314]

Cool! Pretty busy ATM but I'll have to try this soon!

[gz]

Did a bit more hacking, this is slowly starting to look like proper tests:
https://github.com/gz/rust-x86/blob/master/tests/kvm/bin.rs
next steps are asserts, crashes and reporting such things I guess...

[gz]

Seems like a good way to handle panics would be with plugging in a custom libpanic implementation (rust-lang/rust#32837). Unfortunately this is not yet supported, rustc allows only libabort_panic and libabort_unwind.

[gz]

Ideally, there would be a way to override the panic_abort crate as a hack with xargo. I tried the following for now:

Using target.json:

{
  "arch": "x86_64",
  "cpu": "x86-64",
  "data-layout": "e-m:e-i64:64-f80:128-n8:16:32:64-S128",
  "dynamic-linking": true,
  "env": "gnu",
  "exe-allocation-crate": "alloc_system",
  "executables": true,
  "has-elf-tls": false,
  "has-rpath": false,
  "linker-flavor": "gcc",
  "linker-is-gnu": true,
  "llvm-target": "x86_64-unknown-linux-gnu",
  "max-atomic-width": 64,
  "os": "linux",
  "position-independent-executables": false,
  "pre-link-args": {
    "gcc": [
      "-Wl,--as-needed",
      "-Wl,-z,noexecstack",
      "-m64"
    ]
  },
  "target-endian": "little",
  "target-family": "unix",
  "target-pointer-width": "64",
  "vendor": "unknown",
  "panic-strategy": "abort"
}

And Xargo.toml:

[dependencies.std]
features = ["alloc_system"]
stage = 0

[dependencies]
panic_abort = { path = "./libpanic_kvm" }

Which gives:

RUSTFLAGS="-C relocation-model=dynamic-no-pic -C code-model=kernel" RUST_BACKTRACE=1 xargo test --target x86_64-test-linux --test kvm 
    Updating git repository `https://github.com/gz/rust-klogger.git`
    Updating registry `https://github.com/rust-lang/crates.io-index`
error: There are multiple `panic_abort` packages in your project, and the specification `panic_abort` is ambiguous.
Please re-run this command with `-p <spec>` where `<spec>` is one of the following:
  panic_abort:0.0.1
  panic_abort:0.0.0
error: `"cargo" "build" "--release" "--manifest-path" "/tmp/xargo.5kdfklPHXvMO/Cargo.toml" "--target" "x86_64-test-linux" "-p" "panic_abort"` failed with exit code: Some(101)

(and supplying -p panic_abort:0.0.1 on xargo doesn't seem to work)

[gz]

Now that rust-lang/rfcs#2318 has landed I plan to revisit this again.

[gz]

I cleaned this up a bit and since it is in good enough shape now to be easily extended and can run tests I am going to close this. Only took two years :)

https://github.com/gz/rust-x86/tree/master/x86test