Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Datadog's zstd with Klauspost's zstd #2612

Closed
solracsf opened this issue Aug 26, 2022 · 1 comment
Closed

Replace Datadog's zstd with Klauspost's zstd #2612

solracsf opened this issue Aug 26, 2022 · 1 comment
Labels
kind/feature New feature or request

Comments

@solracsf
Copy link
Contributor

What would you like to be added:

Replace datadog's zstd with Klauspost's zstd

https://github.com/DataDog/zstd
https://github.com/klauspost/compress

Why is this needed:

Klauspost's package are pure Go and without use of "unsafe", highly maintained and optimized and also provide support for other compression methods. BadgerDB made the same move: dgraph-io/badger#1709

@solracsf solracsf added the kind/feature New feature or request label Aug 26, 2022
@davies
Copy link
Contributor

davies commented Aug 28, 2022

The original Zstd implementation is more battle tested and performant, we will NOT replace it with a pure Go version in near future, because we compress/decompress the block in one shot, the CGO is not the problem here.

In the case that when CGO is not available, we may switch to the pure Go version using build tags.

@juicedata juicedata locked and limited conversation to collaborators Aug 28, 2022
@davies davies converted this issue into discussion #2618 Aug 28, 2022

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
kind/feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants