make_iterator

[ajum]

Is it possible to add or reproduce this pybind11 functionality in nanobind?

[efifogel]

I've tried to port the implementation in pybind11 to nanobind, and immediately encountered the following problem.
The following statement appears in the pybind11 implementation:
class_<state>(handle(), "iterator", pybind11::module_local())
Is there an equivalent to this statement in nanobind?
I assume that the answer is negative.
So, without getting into too many details (related to lifetime of class and module wrappers), my assumption is that the best way to provide something close to make_iterator is to split it into two pieces:

1. Introduce a function that defines a wrapper for the iterator.
2. Add attributes to modules or classes for this iterator.

Can someone confirm my assumption?

BW, I think that the pybind11 implementation has a degeneracy; The user cannot really change ValueType (e.g., to be a reference type), because the result_type in iterator_access is hard coded.
Also, the name for the iterator attribute is hard coded. Splitting the functionality as described above enables the provision of a name and in turn obtaining a Python iterator explicitly using this name.

[efifogel]

Some parts are taken from pybind11. namespace py = nanobind; // template <typename Type, typename PyClass> bool add_attr(PyClass& cls, const char* name) { const py::handle info = py::type<Type>(); if (! info.is_valid() || ! py::type_check(info)) return false; cls.attr(name) = info; return true; } // template <typename Iterator, typename Sentinel> struct iterator_state { Iterator it; Sentinel end; bool first_or_done; }; // template <py::rv_policy Policy, typename Iterator, typename Sentinel, typename ValueType, typename... Extra, typename C> void add_iterator_impl(const char* name, C& c, Extra&&... extra) { using state = iterator_state<Iterator, Sentinel>; if (add_attr<state>(c, name)) return; py::class_<state>(c, name) .def("__iter__", [](state& s)->state& { return s; }) .def("__next__", [](state& s) -> ValueType { if (! s.first_or_done) ++s.it; else s.first_or_done = false; if (s.it == s.end) { s.first_or_done = true; throw py::stop_iteration(); } return *s.it; }, std::forward<Extra>(extra)..., Policy) ; } // Add (wrap) an iterator template <typename Iterator, typename Sentinel, typename ValueType = decltype(*std::declval<Iterator>()), py::rv_policy Policy = py::rv_policy::reference_internal, typename... Extra, typename C> void add_iterator(const char* name, C& c, Extra&&... extra) { add_iterator_impl<Policy, Iterator, Sentinel, ValueType, Extra...>(name, c, std::forward<Extra>(extra)...); } // Obtain a Python iterator template <typename Iterator, typename Sentinel> py::object make_iterator(Iterator begin, Sentinel end) { using state = iterator_state<Iterator, Sentinel>; return py::cast(state{begin, end, true}); }

…

____ _ ____ _ /_____/_) o /__________ __ // (____ ( ( ( (_/ (_/-(-'_(/ _/

On Sat, 1 Oct 2022 at 03:29, jswarup ***@***.***> wrote: Can someone share the implementation, even a partly working one ? Thanks.. — Reply to this email directly, view it on GitHub <#51 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVBNOFEZRNWH5RGZFMBTZTWA6AVLANCNFSM53QD7ZCQ> . You are receiving this because you commented.Message ID: ***@***.***>

[efifogel]

What do you mean? It's a function in nanobind defined in nb_error.h, which is included by nanobind.h.

…

____ _ ____ _ /_____/_) o /__________ __ // (____ ( ( ( (_/ (_/-(-'_(/ _/

On Sun, 2 Oct 2022 at 08:03, jswarup ***@***.***> wrote: Thanks Efi. I had tried this and got stuck in mapping this part.. throw py::stop_iteration(); to work. — Reply to this email directly, view it on GitHub <#51 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVBNOEVQIZGAQDDS4S6NOLWBEJQ3ANCNFSM53QD7ZCQ> . You are receiving this because you commented.Message ID: ***@***.***>

[jswarup]

Apologies for delay on geting back. I missed a call to register the iterator. It works fine. add_iterator<const double *, const double *>( name, mod); Thanks Jyoti

…

On Sun, Oct 2, 2022 at 3:24 PM Efi Fogel ***@***.***> wrote: What do you mean? It's a function in nanobind defined in nb_error.h, which is included by nanobind.h. ____ _ ____ _ /_____/_) o /__________ __ // (____ ( ( ( (_/ (_/-(-'_(/ _/ On Sun, 2 Oct 2022 at 08:03, jswarup ***@***.***> wrote: > Thanks Efi. > I had tried this and got stuck in mapping this part.. > > throw py::stop_iteration(); > > to work. > > — > Reply to this email directly, view it on GitHub > < #51 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/ABVBNOEVQIZGAQDDS4S6NOLWBEJQ3ANCNFSM53QD7ZCQ > > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#51 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADITIRAIFIEYLL2R4NHLNGDWBFLVFANCNFSM53QD7ZCQ> . You are receiving this because you commented.Message ID: ***@***.***>

[wjakob]

FYI: This functionality has now been officially added to nanobind. However, there are small differences compared to pybind11. See the change to README.md and the test suite example for details.

34d0be1