Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any Possible issue in creating std::span from buffer memory directly #35226

Open
ava6969 opened this issue Apr 19, 2023 · 1 comment
Open

Any Possible issue in creating std::span from buffer memory directly #35226

ava6969 opened this issue Apr 19, 2023 · 1 comment
Labels
Component: C++ Type: usage Issue is a user question

Comments

@ava6969
Copy link

ava6969 commented Apr 19, 2023

Describe the usage question you have. Please include as many useful details as possible.

I recently have been accessing const raw data from ArrayData holding doubles. by doing the following

auto ptr = reinterpret_cast<const double*>(m_array->data()->buffers[1]->data()) + m_array->offset();
std::span view(ptr, m_array->length);

i found when null values exist they are 0; which is fine for my application. but are they are any other thing i should be thinking of.

Component(s)

C++

@ava6969 ava6969 added the Type: usage Issue is a user question label Apr 19, 2023
@westonpace
Copy link
Member

Null values are not guaranteed to be 0. We use a separate bitmask validity vector to determine if an element is null or not. If an element is null then it's value is undefined and could be anything.

However, creating spans from buffers is fine.

Those spans will not keep the buffer alive though. So you need to avoid something like this:

std::span GetMyData() {
  std::shared_ptr<Array> m_array = GetSomeArray();
  auto ptr = reinterpret_cast<const double*>(m_array->data()->buffers[1]->data()) + m_array->offset();
  std::span view(ptr, m_array->length);
  return view;
}

Also, that reinterpret cast is only valid if m_array->data()->buffers[1]->data() is 8-byte aligned. This is true for any buffer allocated by Arrow-C++ and is supposed to be true for any buffer sent over IPC but it is not currently guaranteed for buffers received over Flight due to #32276.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: C++ Type: usage Issue is a user question
Projects
None yet
Development

No branches or pull requests

2 participants