Missed optimization when looping over bytes of a value

[theemathas]

I tried this code, which contains 3 functions which check if all the bits in a u64 are all ones:

#[no_mangle]
fn ne_bytes(input: u64) -> bool {
    let bytes = input.to_ne_bytes();
    bytes.iter().all(|x| *x == !0)
}

#[no_mangle]
fn black_box_ne_bytes(input: u64) -> bool {
    let bytes = input.to_ne_bytes();
    let bytes = std::hint::black_box(bytes);
    bytes.iter().all(|x| *x == !0)
}

#[no_mangle]
fn direct(input: u64) -> bool {
    input == !0
}

I expected to see this happen: ne_bytes() should be optimized to the same thing as direct(), while black_box_ne_bytes() should be optimized slightly worse

Instead, this happened: I got the following assembly, where ne_bytes() is somehow optimized worse than black_box_ne_bytes()

ne_bytes:
        mov     rax, rdi
        not     rax
        shl     rax, 8
        sete    cl
        shr     rdi, 56
        cmp     edi, 255
        setae   al
        and     al, cl
        ret

black_box_ne_bytes:
        mov     qword ptr [rsp - 8], rdi
        lea     rax, [rsp - 8]
        cmp     qword ptr [rsp - 8], -1
        sete    al
        ret

direct:
        cmp     rdi, -1
        sete    al
        ret

Godbolt

Meta

Reproducible on godbolt with stable rustc 1.82.0 (f6e511eec 2024-10-15) and nightly rustc 1.85.0-nightly (7db7489f9 2024-11-25)

[purplesyringa]

As far as I can see, something very similar is at least partially fixed on LLVM trunk: https://godbolt.org/z/MzqG7rf9d. There's also another similar issue: llvm/llvm-project#117853, but I'm not sure if it's relevant to this particular issue.

[clubby789]

Looks like all 3 functions optimize to the same thing on LLVM trunk opt

[madhav-madhusoodanan]

Hi
I was learning how LLVM integrates with rustc when I came across this issue.

Just wanted to know if this issue depends on LLVM releasing a new version that optimizes per-byte comparisons.

[DianQK]

Just wanted to know if this issue depends on LLVM releasing a new version that optimizes per-byte comparisons.

The master branch has included the new version (LLVM 20).

[madhav-madhusoodanan]

Got it, thank you!

[karolzwolak]

On the nightly, direct and ne_bytes produce identical code, and black_box_ne_bytes is not too far off.
current output from godbolt

black_box_ne_bytes:
        mov     qword ptr [rsp - 8], rdi
        lea     rax, [rsp - 8]
        cmp     qword ptr [rsp - 8], -1
        sete    al
        ret

direct:
        cmp     rdi, -1
        sete    al
        ret

I'd like to work on this issue, but I'm not sure how to go about writing good tests for this. I guess we could just test that direct and ne_bytes produce identical assembly, but what about black_box_ne_bytes?
Could someone guide me on how to implement in a sound way?

[clubby789]

@karolzwolak You should check the codegen tests (https://github.com/rust-lang/rust/blob/master/tests/codegen/) for example. You'll want to test that ne_bytes compiles to

define noundef zeroext i1 @ne_bytes(i64 noundef %input) unnamed_addr {
start:
  %_0 = icmp eq i64 %input, -1
  ret i1 %_0
}

Which will need a CHECK-LABEL, a CHECK for the icmp then a CHECK-NEXT for the ret

black_box_ne_bytes can be ignored - it has the same optimization, just with some extra stack moves

[karolzwolak]

Thanks, really appreciate the help.

[theemathas]

Closed by #137826