From bc16b515c1e0a40572e5fa334e1168c37b1c9850 Mon Sep 17 00:00:00 2001 From: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com> Date: Mon, 28 Aug 2023 20:56:31 -0500 Subject: [PATCH 1/7] Add size != stride --- src/frequently-requested-changes.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index ff5c8c5..3caf1d6 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -178,3 +178,19 @@ statement, and only later see the `if` and realize it's a conditional return. Such a change would also have non-obvious evaluation order (evaluating the condition before the return expression). + +## Size != Stride + +Rust assumes that the size of an object is equivalent to the stride of an object - +this means that the size of `[T; N]` is `N * std::mem::size_of::`. Other languages +may have objects that take up less space in arrays due to the reuse of tail +padding, and allow interop with other languages which do this optimization. + +Rust makes several guarantees that make supporting these types difficult in the general case. +The combianation `std::array::from_ref` and array indexing is a stable guarantee that a pointer +(or reference) to an type is convertible to a pointer to a 1-array of that type. + +Such a change could also pose problems for existing unsafe code, which may assume that pointers +can be manually offset by the size of the type to access the next array element. Unsafe +code may also assume that overwriting trailing padding is allowed, which would conflict with +the repurposing of such padding for data storage. From c47db91faa330e3b84a64932beb09621933dbf9c Mon Sep 17 00:00:00 2001 From: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com> Date: Mon, 28 Aug 2023 22:42:20 -0500 Subject: [PATCH 2/7] Update src/frequently-requested-changes.md Co-authored-by: Mario Carneiro --- src/frequently-requested-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index 3caf1d6..d6e7c03 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -187,7 +187,7 @@ may have objects that take up less space in arrays due to the reuse of tail padding, and allow interop with other languages which do this optimization. Rust makes several guarantees that make supporting these types difficult in the general case. -The combianation `std::array::from_ref` and array indexing is a stable guarantee that a pointer +The combination of `std::array::from_ref` and array indexing is a stable guarantee that a pointer (or reference) to an type is convertible to a pointer to a 1-array of that type. Such a change could also pose problems for existing unsafe code, which may assume that pointers From 4dc29f0af7743f7a31944d6c0d1f14f966952f23 Mon Sep 17 00:00:00 2001 From: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com> Date: Mon, 28 Aug 2023 22:42:26 -0500 Subject: [PATCH 3/7] Update src/frequently-requested-changes.md Co-authored-by: Mario Carneiro --- src/frequently-requested-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index d6e7c03..a228c20 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -188,7 +188,7 @@ padding, and allow interop with other languages which do this optimization. Rust makes several guarantees that make supporting these types difficult in the general case. The combination of `std::array::from_ref` and array indexing is a stable guarantee that a pointer -(or reference) to an type is convertible to a pointer to a 1-array of that type. +(or reference) to a type is convertible to a pointer to a 1-array of that type. Such a change could also pose problems for existing unsafe code, which may assume that pointers can be manually offset by the size of the type to access the next array element. Unsafe From c72349fbce7f529bd148a695853789f207734072 Mon Sep 17 00:00:00 2001 From: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com> Date: Mon, 28 Aug 2023 22:55:30 -0500 Subject: [PATCH 4/7] Expand and clarify wording --- src/frequently-requested-changes.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index a228c20..c7f9d63 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -186,9 +186,19 @@ this means that the size of `[T; N]` is `N * std::mem::size_of::`. Other lang may have objects that take up less space in arrays due to the reuse of tail padding, and allow interop with other languages which do this optimization. -Rust makes several guarantees that make supporting these types difficult in the general case. +One downside of this assumption is that types with alignment greater than their size can +waste large amounts of space due to padding. An overaligned struct such as the following: +``` +#[repr(C, align(512))] +struct Overaligned(u8); +``` +will store only 1 byte of data, but will have 511 bytes of tail padding for a total size of +512 bytes. This tail padding will not be reusable, and adding `Overaligned` as a struct field +may exacerbate this waste as additional trailing padding be included after any other members. + +Rust makes several guarantees that make supporting size != stride difficult in the general case. The combination of `std::array::from_ref` and array indexing is a stable guarantee that a pointer -(or reference) to a type is convertible to a pointer to a 1-array of that type. +(or reference) to a type is convertible to a pointer to a 1-array of that type, and vice versa. Such a change could also pose problems for existing unsafe code, which may assume that pointers can be manually offset by the size of the type to access the next array element. Unsafe From d5632bedf1cb34dde3b5876fb817dce32d4b41e7 Mon Sep 17 00:00:00 2001 From: Niko Matsakis Date: Tue, 29 Aug 2023 14:06:20 -0400 Subject: [PATCH 5/7] add cross references --- src/frequently-requested-changes.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index c7f9d63..51f0e4a 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -204,3 +204,9 @@ Such a change could also pose problems for existing unsafe code, which may assum can be manually offset by the size of the type to access the next array element. Unsafe code may also assume that overwriting trailing padding is allowed, which would conflict with the repurposing of such padding for data storage. + +Cross-referencing to other discussions: + +* https://github.com/rust-lang/rfcs/issues/1397 +* https://github.com/rust-lang/rust/issues/17027 +* https://github.com/rust-lang/unsafe-code-guidelines/issues/176 From 8e8c230986e51408564779c934b79f08e1f98df4 Mon Sep 17 00:00:00 2001 From: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com> Date: Wed, 30 Aug 2023 01:37:02 -0500 Subject: [PATCH 6/7] Update src/frequently-requested-changes.md Co-authored-by: scottmcm --- src/frequently-requested-changes.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index 51f0e4a..5424c20 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -205,6 +205,8 @@ can be manually offset by the size of the type to access the next array element. code may also assume that overwriting trailing padding is allowed, which would conflict with the repurposing of such padding for data storage. +While changing the fundamental layout guarantees seems unlikely, it may be reasonable to add additional inspection APIs for code that wishes to opt into the possibility of copying smaller parts of an object -- an API to find out that copying only bytes `0..1` of `Overaligned` is sufficient might still be reasonable, or something `size_of_val`-like that could be variant-aware to say which bytes are sufficient for copying a particular instance. + Cross-referencing to other discussions: * https://github.com/rust-lang/rfcs/issues/1397 From aec954e2c21604a2a8bebcf35ffa4fe6e6cd9e63 Mon Sep 17 00:00:00 2001 From: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com> Date: Wed, 30 Aug 2023 01:39:11 -0500 Subject: [PATCH 7/7] Mention move-only fields + fix typo --- src/frequently-requested-changes.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/src/frequently-requested-changes.md b/src/frequently-requested-changes.md index 5424c20..3e5229b 100644 --- a/src/frequently-requested-changes.md +++ b/src/frequently-requested-changes.md @@ -182,9 +182,9 @@ condition before the return expression). ## Size != Stride Rust assumes that the size of an object is equivalent to the stride of an object - -this means that the size of `[T; N]` is `N * std::mem::size_of::`. Other languages -may have objects that take up less space in arrays due to the reuse of tail -padding, and allow interop with other languages which do this optimization. +this means that the size of `[T; N]` is `N * std::mem::size_of::`. Allowing +size to not equal stride may allow objects that take up less space in arrays due +to the reuse of tail padding, and allow interop with other languages with this behavior. One downside of this assumption is that types with alignment greater than their size can waste large amounts of space due to padding. An overaligned struct such as the following: @@ -205,7 +205,12 @@ can be manually offset by the size of the type to access the next array element. code may also assume that overwriting trailing padding is allowed, which would conflict with the repurposing of such padding for data storage. -While changing the fundamental layout guarantees seems unlikely, it may be reasonable to add additional inspection APIs for code that wishes to opt into the possibility of copying smaller parts of an object -- an API to find out that copying only bytes `0..1` of `Overaligned` is sufficient might still be reasonable, or something `size_of_val`-like that could be variant-aware to say which bytes are sufficient for copying a particular instance. +While changing the fundamental layout guarantees seems unlikely, it may be reasonable to add additional +inspection APIs for code that wishes to opt into the possibility of copying smaller parts of an object +-- an API to find out that copying only bytes `0..1` of `Overaligned` is sufficient might still be +reasonable, or something `size_of_val`-like that could be variant-aware to say which bytes are sufficient +for copying a particular instance. Similarly, move-only fields may allow users to mitigate the effects +of tail or internal padding, as they can be reused due to the lack of a possible reference or pointer. Cross-referencing to other discussions: