RFC: Generic member access for dyn Error trait objects

[yaahc]

Rendered

Also, I've mocked up the proposal in this repository - https://github.com/yaahc/nostd-error-poc/

[anp]

(didn't make it very far before my day filled up again, will return shortly)

[burdges]

One provide_context implementation supports multiple TypeIds, yes?

pub struct MyError<'a> {
    some_ref: &'a ..,
    /// All the rest are static
    something_any: ..,
    something_error: ..,
    something_read: ..,
}
impl<'a> Error for MyError<'a> {
    ...
    fn provide_context(&self, ty: TypeId) -> Option<&dyn Any> {
        if ty == TypeId::of::<dyn Any>() {
            &self.something_any as &dyn Any
        } else if ty == TypeId::of::<dyn Error>() {
            &self.something_error as &dyn Any
        } else if ty == TypeId::of::<dyn Read>() {
            &self.something_read as &dyn Any
        } else { None }
    }
}

You cannot make the default provide_context return Some(self as &dyn Any) whenever TypeId::of::<T>() == TypeId::of::<Self>() because doing so requires a Self: 'static bound.

[yaahc]

@burdges this is a general restriction on Any and downcast on Error types, I don't think it's currently possible to downcast types that aren't : 'static

The second design with the a Provider miiight work with this, not sure.

[anp]

I focused my feedback more on "technical writing" as requested, but in general I'm quite excited to see this proposal move forward! There are a number of cases at work where it would be nice to specialize error presentation in this way. At a high level I think this proposal would benefit from a "why not more general, i.e. all trait objects?" section. I also recommend putting more examples closer to the top of the "error reports" that you'd like to enhance -- they're quite motivating IMO.

[bstrie]

WRT "update RFC to be based on dyno design", can someone give a brief summary of the advantages of this new design over the old one?

[yaahc]

WRT "update RFC to be based on dyno design", can someone give a brief summary of the advantages of this new design over the old one?

I think this reply summarizes it best but the main difference is dyno uses a proxy type that itself only has a phantom data, and so it is 'static and gets a TypeID. That Tag type has a trait with an associated type that is the actual type being erased, where as the old design used the type being erased's TypeID directly, requiring that it is 'static. The new design allows for closures containing mutable references to be requested as shown in this earlier comment, which could be used on a type erased executor with this same generic member access API to request a callback that then modifies state in the executor on the other side of the trait object.

[yaahc]

This RFC feels very complex and I am a bit uncomfortable with adding so much API surface to std just for the Error trait.

I wonder if it is possible to offload some of the complexity of this API to crates.io, similar to how proc_macro only exposes a basic interface (just a token stream) while most of the functionality for building proc macros is located in the syn and quote crates which are not part of the standard library and can evolve independently of it.

@Amanieu fwiw I'm already working on extracting the type tagged downcasting part of this RFC into it's own independent RFC, which should hopefully make both RFCs much easier to review. I suppose I should update the todolist above to reflect that 😅 .

[CleanCut]

This looks fantastic! ❤️

What is the likely timeline for this to reach stable? Given that it hasn't hit nightly yet, I'm guessing this will be gated behind a Rust 2024 edition?

[yaahc]

This looks fantastic! ❤️

What is the likely timeline for this to reach stable? Given that it hasn't hit nightly yet, I'm guessing this will be gated behind a Rust 2024 edition?

There's no timeline but it won't be gated behind any edition. This RFC doesn't include any breaking changes. The current next step is to break out the type tagged downcasting logic into it's own RFC which is being done by @Plecra.

[yaahc]

Reminder for myself: Once this lands I would like to update the example in the docs added in rust-lang/rust#95356 to instead use generic member access to grab the ExitCode from the error.

[waynr]

This seems outdated. In master branch I don't see any backtrace function on the Error trait with or without that feature. Maybe the backtrace feature was already merged but left out the backtrace method on Error. This RFC makes it unnecessary anyway so I'll look into updating this section here.

[Amanieu]

There used to be an unstable backtrace function on Error, but it was removed in favor for the generic member access proposed in this RFC.

[lygstate]

maybe this proposal need updated to reflect that fact?

[CobaltCause]

A potential additional drawback is that requesting types dynamically like this can lead to surprising changes in runtime behavior when changing versions of (transitive) dependencies.

Given:

• A crate foo with versions 1.0.0 and 2.0.0, each of which provides a type Foo.
• A crate bar which depends on version 1.0.0 of foo and whose errors provide Foo.
• A crate baz which depends bar and on version 2.0.0 of foo and tries to request Foo from bar's errors.

Then a situation is created in which baz will fail to request Foo from bar's error because the requested Foo and provided Foo are actually distinct types because the transitive dependency foo's versions don't match up.

This could even happen in a semver-compatible release of bar and baz because technically the API is not broken: the type can still be requested, but it will simply begin returning None instead of Some(_). This would be even worse if foo, bar, and baz are all library crates depended on by an application crate and the semver-incompatible versions of foo are introduced by a semver-compatible change in bar or baz which would get pulled in by running cargo update on the application crate. The application crate has no recourse in this situation aside from waiting for bar and baz to agree on a semver-compatible version of foo again.

(I've been playing with an alternate design based on an associated type rather than generic member access which causes this situation to be a compile time error instead of a runtime behavior change, but that design requires traits with lifetime GATs to be dyn safe to be actually good, and associated type defaults for it to maybe be possible to retrofit that design onto the standard library's existing error trait.)

[yaahc]

I agree this is a footgun I'd prefer to avoid but as far as I can tell this is not a problem being uniquely introduced by generic member access. This is a problem with type erasure in general and the same situation could be encountered by having foo return Foo as a Box<dyn Any> in any of it's public API's. If baz tries to downcast using it's own path to foo::Foo it will run into the same type ID mismatch.

I avoid this in my crates currently by re-exporting types from crates that are part of my public API (https://docs.rs/color-eyre/latest/color_eyre/#reexports). Applying this to your example you could avoid this footgun if you downcast/request Foo via bar::foo::Foo which would correctly resolve to whatever version it is using. This of course gets a little silly when you want to be able to handle all the versions of Foo, because then presumably you have to try to request Foo once for each path through which you could possibly reach it, which is definitely a subpar user experience, so not necessarily a great solution.

That said, I was never able to figure out how to implement generic member access w/o type erasure and runtime checks. I'm curious to understand more about your potential alternative design, because I would definitely love to see a better solution but I don't see how it is possible given the constraints of the error trait and the goals of generic member access.

[CobaltCause]

I agree this is a footgun I'd prefer to avoid but as far as I can tell this is not a problem being uniquely introduced by generic member access.

True.

This is a problem with type erasure in general and the same situation could be encountered by having foo return Foo as a Box<dyn Any> in any of it's public API's.

However, it seems to me that this kind of thing is relatively rare currently, both in the standard library and ecosystem crates. The only standard library exception I can think of is std::io::Error, for which I found (part of?) the rationale for in rust-lang/rust#24793 (comment), and I haven't personally encountered any ecosystem crates that make use of this and tell users in documentation that they can/should downcast the inner error to some concrete type.

So, I think making type erasure a central part of the error handling API of the standard library would probably be the main place this kind of problem would come up. I'm not sure the existing occasional cases of downcasting in public APIs can be used to justify a standard library API built entirely on downcasting.

Applying this to your example you could avoid this footgun if you downcast/request Foo via bar::foo::Foo which would correctly resolve to whatever version it is using.

While true, I imagine another place this might become a problem is in a library whose purpose is to use generic member access to generate pretty error reports. This hypothetical library might hypothetically pull in the backtrace crate for handling backtraces. If any other crate in the dependency tree has errors that provide a backtrace from a different version of the backtrace crate, the hypothetical library will not be able to extract them. I don't think this can be solved with re-exports, because that would force the library to have a re-export of every crate that has errors that provide a backtrace crate backtrace.

Your next point about requesting the type through each path (including crate version) does provide a solution there, but also as you point out, would not be the most fun to maintain.

I'm curious to understand more about your potential alternative design, because I would definitely love to see a better solution but I don't see how it is possible given the constraints of the error trait and the goals of generic member access.

I can't promise that this particular idea is any better, but for what it's worth, it goes like this:

trait Error {
    /// Type that contains details about this error.
    ///
    /// For example, this could be a struct containing a String with help text and a Backtrace.
    type Details<'a> = ()
        where Self: 'a;

    /// Get the details about this error.
    fn details(&self) -> Self::Details<'_>;
}

As written, this requires some language features that aren't implemented yet:

1. Associated type defaults. Necessary to make it possible to backport this onto core::error::Error, but luckily not required for defining a new error trait in a new crate.
2. dyn-safety for traits with lifetime GATs. In my current implementation, I've simply dropped the lifetime and am just returning an owned value, which has some overhead but at least allows experimenting with the rest of the design.

Another part of this design is a map-like function to change the type of Details when composing errors that have different concrete types.

To approximate generic member access to make it possible to implement a hypothetical library for pretty-printing errors like described above, I imagine a crate could define some traits like trait GetBacktrace { fn get_backtrace(&self) -> Option<&Backtrace>; } (or one trait with many methods, I suppose) and then adding these traits as bounds on Error::Details on the pretty-printing functions. The author of a binary crate could then implement these traits for their Error::Details type and use the library's pretty-printing code.

[tmccombs]

Wouldn't that still have a similar problem? To adapt your example from before:

• A crate foo with versions 1.0.0 and 2.0.0, each of which provides a trait Foo.
• A crate bar which depends on version 1.0.0 of foo and whose errors have a Details type that implements Foo
• A crate baz which depends bar and on version 2.0.0 of foo and has uses a bound that requires Error::Details to implement Foo (from version 2.0.0 of foo)

baz still can't get the the Foo details from errors from bar.

[CobaltCause]

That's correct, but the difference is how it can't get the details. With the generic member access design, this would fail at run time: the generic member access functions would return None. With the associated type design, this would fail at compile time: a compilation error would be emitted about the trait bound being unsatisfied.

[tmccombs]

You can currently have a Box<dyn Error + 'static> (or reference, or some other lifetime). Associated types and dyn-safety for traits with lifetime GATs isn't enough. If you had a Details type that isn't equal to the default, your error won't be compatible with anything that types a dynamic error, without some other kind of mechanism. And there may be some benefit to a compile time check. But that also means you lose use cases where you might want to extract data dynamically without knowing if it exists at runtime.

[CobaltCause]

You can currently have a Box<dyn Error + 'static> (or reference, or some other lifetime). Associated types and dyn-safety for traits with lifetime GATs isn't enough. If you had a Details type that isn't equal to the default, your error won't be compatible with anything that types a dynamic error, without some other kind of mechanism.

I guess a really important question is how many notable occurrences of Box<dyn Error> (and similar) are there in public APIs? The only one I can think of off the top of my head is std::io::Error::{other, into_inner}. In such a case, I believe the map function I alluded to before should be sufficient. To extract the original Details, you'd have to downcast to the concrete type first, which I think is not really a regression because that's how std::io::Error is currently intended to work. For internal-only APIs, the type could simply be changed to specify the desired Details.

But that also means you lose use cases where you might want to extract data dynamically without knowing if it exists at runtime.

I suspect these cases could be covered for Error-processing functions by adding bounds on Details of traits that contain accessor functions that return Option<T> so that Error implementors have the, uh, option of returning None. Are there any situations where that would be insufficient?

[tmccombs]

I guess a really important question is how many notable occurrences of Box (and similar) are there in public APIs?

Well, I think returning a Result<(), Box<dyn Error + 'static> from main is a somewhat common pattern. And while you could maybe replace that with Result<(), Box<dyn Error<Details=SomeType> + 'static> , you would still need the same Details type for all errors, which gets messy if your errors come from different sources.

And you have libraries like anyhow that make heavy use of dyn core::error:Error, such as in root_cause() and chain() and from_boxed() in anyhow.

To extract the original Details, you'd have to downcast to the concrete type first, which I think is not really a regression because that's how std::io::Error is currently intended to work

But that means you have to know what the concrete type is. And Details would need to implement a trait that has support for downcasting. And how do you handle combining multiple detail types into the a single error. Do you try casting to a struct that has the exact combination of details you expect? Do you use a collection of Details objects, and then try downcasting each one of those to what you expect?