Skip to content

Commit

Permalink
Auto merge of rust-lang#109670 - lqd:init-mask, r=oli-obk
Browse files Browse the repository at this point in the history
Make init mask lazy for fully initialized/uninitialized const allocations

There are a few optimization opportunities in the `InitMask` and related const `Allocation`s (e.g. by taking advantage of the fact that it's a bitset that represents initialization, which is often entirely initialized or uninitialized in a single call, or gradually built up, etc).

There's a few overwrites to the same state, multiple writes in a row to the same indices, the RLE scheme for `memcpy` doesn't always compress, etc.

Here, we start with:
- avoiding materializing the bitset's blocks if the allocation is fully initialized/uninitialized
- dealloc blocks when fully overwriting, including when participating in `memcpy`s
- take care of the fixme about allocating blocks of 0s before overwriting them to the expected value
- expanding unit test coverage of the init mask

This should be most visible on benchmarks and crates where const allocations dominate the runtime (like `ctfe-stress-5` of course), but I was especially looking at the worst cases from rust-lang#93215.

This first change allows the majority of `set_range` calls to stay with a lazy init mask when bootstrapping rustc (not that the init mask is a big part of the process in cpu time or memory usage).

r? `@oli-obk`

I have another in-progress branch where I'll switch the singular initialized/uninitialized value to a watermark, recording the point after which everything is uninitialized. That will take care of cases where full initialization is monotonic and done in multiple steps (e.g. an array of a type without padding), which should then allow the vast majority of const allocations' init masks to stay lazy during bootstrapping (though interestingly I've seen such gradual initialization in both left-to-right and right-to-left directions, and I don't think a single watermark can handle both).
  • Loading branch information
bors committed Mar 29, 2023
2 parents cf32b9d + a857ba2 commit 8679208
Show file tree
Hide file tree
Showing 6 changed files with 447 additions and 103 deletions.
2 changes: 0 additions & 2 deletions compiler/rustc_middle/src/mir/interpret/allocation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
mod init_mask;
mod provenance_map;
#[cfg(test)]
mod tests;

use std::borrow::Cow;
use std::fmt;
Expand Down
Loading

0 comments on commit 8679208

Please sign in to comment.