std::optional arguments and .none()

[oremanj]

I'm working with several of my coworkers on converting a bunch of pybind11 code to use nanobind this week. One of the things we've run into is the stricter nanobind requirements around None. This makes perfect sense to me when it comes to null pointers, which many C++ functions do not handle reasonably, even if they take an argument of pointer type.

We were rather more surprised that the None-guard also affects std::optional<T> and std::variant<std::monostate, Ts...>. In both of those cases, the type explicitly indicates that None (nullopt, monostate) is a valid option. If it were not, then a different type would've been chosen. There's no reason to accept std::optional<T> unless you plan to handle the possibility of receiving nullopt. That makes the additional need for .none() feel like a bit of a trap.

Is it a deliberate design choice that one must indicate that, yes, your T-or-None value can indeed accept None as a possibility? Or is it more of an emergent property of the current implementation that you'd be happy to see change, as long as null pointers still can't be passed without a .none()?

[wjakob]

It's an emergent property of the design. For example, nanobind currently tries very hard not to pull in any unnecessary STL headers. If you don't specify any argument annotations, then the only part that really knows about the types of the arguments is the closure called by the function dispatcher. And the detection of None is done one layer removed from that in the dispatch loop, for reasons of code generation efficiency (better to keep this on the "static" path than having to compile it thousands of times in every function binding).

I'm open to improvements here if they don't involve adding nontrivial cost/bloat to users of the library who don't use std::variant / std::optional. But I don't see how to do it, or I would have done it already.