[BUG]: ndarray<> fails on read-only buffer object

[zingdle]

Problem description

when accessing numpy array with ndarray<>, nanobind fails when binding on a read-only object.

seems that

nanobind/src/nb_ndarray.cpp

Line 165 in a965f5b

if (PyObject_GetBuffer(o, view.get(), PyBUF_RECORDS)) {

PyBUF_RECORDS requires write permission. errors ValueError: buffer source array is read-only

Reproducible example code

#include <nanobind/nanobind.h>
#include <nanobind/ndarray.h>

namespace nb = nanobind;

NB_MODULE(myext, m) {
  m.def("foo", [](nb::object o) {
    auto a = nb::cast<nb::ndarray<>>(o);
    printf("%ld\n", a.ndim());
  });
}

>>> import myext
>>> import numpy as np
>>> a = np.broadcast_to(1, (2, 3))
>>> b = np.arange(16)
>>> myext.foo(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: nanobind::cast_error
>>> myext.foo(b)
1

[wjakob]

nanobind currently doesn't have the concept of a read-only tensor and so refusing this unsafe cast seems reasonable. I might consider adding a special nb::ndarray<nb::ro, ..> annotation (perhaps with alias nb::ndarray_ro<..> ) that is able to accept such arrays. Would that fix your problem?

I think that pybind11 simply did not perform checks on whether an array is read only (and shouldn't be written to) requiring the user to do the right thing, which is of course also a solution though arguably not the best one.

[eirrgang]

I might consider adding a special nb::ndarray<nb::ro, ..> annotation

At least one other user would definitely appreciate that! :-)

edit: I could probably offer a PR, if you like. If there is more discussion about such a feature, I could probably contribute after May 15.

[zingdle]

Yes, a special annotation fixed my problem and make semantics clear. Looking forward to this feature!

[wjakob]

@eirrgang @zingdle can you take a look at PR #217 and let me know if this addresses your needs?

[eirrgang]

@eirrgang @zingdle can you take a look at PR #217 and let me know if this addresses your needs?

Ah, okay. The C++ accessors on the ndarray object will respect the const-ness, and the dltensor member is private anyway. The dlpack interface necessarily loses the const-ness, but there's no way around that until dlpack grows "read-only" semantics.

I'm not seeing any modification to the o.__dlpack__ support. Shouldn't it be disabled or force a copy when re-sharing a buffer that was provided read-only, or from a const object? (How does numpy handle that?) Sorry, I'm less familiar with the dlpack use cases. If this is within the scope of the comment about dlpack ignoring ro, maybe there should be a stronger admonition or warning.

For numpy and buffer protocol users, though, it looks great to me! Thanks!

[wjakob]

The dlpack interface necessarily loses the const-ness, but there's no way around that until dlpack grows "read-only" semantics.

DLPack supports a read-only flag as of dmlc/dlpack#113. But the details of the Python interface aren't worked out, and in any case nobody is using this new versioned flavor of DLPack yet. I plan to update nanobind once there are some implementations of the interface to test against..

I'm not seeing any modification to the o.dlpack support. Shouldn't it be disabled or force a copy when re-sharing a buffer that was provided read-only, or from a const object? (How does numpy handle that?) Sorry, I'm less familiar with the dlpack use cases. If this is within the scope of the comment about dlpack ignoring ro, maybe there should be a stronger admonition or warning.

nanobind generally ignores constness in function parameters and return values. Ignoring the read-only flag in the DLPack interface seems to me in line with this overall behavior. PR #217 implements a limited form of const tracking for nd-arrays because we need this flag to be able to fully interface with NumPy (which is far more strict).