Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the length field for Buffer #2524

Closed
HaoYang670 opened this issue Aug 19, 2022 · 2 comments · Fixed by #2557
Closed

Add the length field for Buffer #2524

HaoYang670 opened this issue Aug 19, 2022 · 2 comments · Fixed by #2557
Labels
arrow Changes to the arrow crate question Further information is requested

Comments

@HaoYang670
Copy link
Contributor

HaoYang670 commented Aug 19, 2022

Which part is this question about
So far, the Buffer doesn't have its own length, which means you can't slice on data with the length you want.:

pub struct Buffer {
    /// the internal byte buffer.
    data: Arc<Bytes>,

    /// The offset into the buffer.
    offset: usize,
}

This is an old question, and related to some bugs (such as #807), and some performance issues (such as #2510)
I reopen this question because we don't reach a consensus.

Personally, I hope Buffer could behave like &[u8] or Arc<Vector>, but we need more discussion.

Additionally, I hope we could reach a consensus, for example:

  • We should add the length field, because ...
  • we don't need to add this field, because ...

Describe your question

Additional context

@HaoYang670 HaoYang670 added the question Further information is requested label Aug 19, 2022
@tustvold
Copy link
Contributor

I would suggest lifting the length out of Bytes onto Buffer, this way we avoid redundant information

@HaoYang670
Copy link
Contributor Author

Thank you, @tustvold. I will try your suggestion and file a draft PR. (maybe need more help from you)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate question Further information is requested
Projects
None yet
3 participants