Add zstd pre-and-postprocessor.

[dak-x]

Signed-off-by: Daksh Chauhan dak-x@outlook.com

Pull request

Add zstd pre-and-postprocessors

Description

Related

• Related Issues: closes Add zstd pre- and postprocessors #1100

Checklist

• The RFC, if required, has been submitted and approved
• Any user-facing impact of the changes is reflected in docs.tremor.rs
• The code is tested
• Use of unsafe code is reasoned about in a comment
• Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
• The performance impact of the change is measured (see below)

Performance

[dak-x]

The code required seems too straightforward. I feel I am missing some stuff. Also I should probably add some tests, but I do not know what tests.

[coveralls]

Coverage increased (+0.01%) to 84.019% when pulling b8665a2 on dak-x:main into 83970b5 on tremor-rs:main.

[Licenser]

Heya :) awesome! thank you! only a few small things, one I commented inline, then there should be some tests for the processors and the lookup. You can find examples on how to dot hat in the preprocessors.rs there the other compression algorithms are tested too.

The https://github.com/tremor-rs/tremor-www-docs/blob/main/docs/artefacts/postprocessors.md and https://github.com/tremor-rs/tremor-www-docs/blob/main/docs/artefacts/preprocessors.md will need updating to reflect the new processor.

Otherwise 👍 great job :D

[mfelsche]

That was quick! 😎 🚀

[mfelsche]

We might also need to add libzstd as dependencies to our rpm, deb packages and to the docker image.

[dak-x]

@mfelsche Where should I update this dependency for the build.

[mfelsche]

I think zstd brings zstd-sys which is bringing its own libzstd, so it is actually fine, no need to add anything :) sorry for the confusion.

[Licenser]

Woooh thank you!! 🚀