Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't fill unencodable ALP floats with first encodable value #2560

Closed
wants to merge 6 commits into from
Closed

Don't fill unencodable ALP floats with first encodable value #2560

wants to merge 6 commits into from

Conversation

robert3005
Copy link
Member

@robert3005 robert3005 commented Feb 28, 2025 •
edited
Loading

No description provided.

@robert3005 robert3005 mentioned this pull request Mar 1, 2025
Open
@robert3005 robert3005 added the benchmark Run benchmarks on this branch label Mar 1, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Mar 1, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 1, 2025

Benchmarks: random_access

Table of Results
name PR 229c44c base 3876b18 ratio (PR/base) unit
random-access/vortex-tokio-local-disk 2496131 2.6077e+06 0.957214 ns
random-access/parquet-tokio-local-disk 277290814 2.731e+08 1.01535 ns

@codspeed-hq CodSpeed HQ
Copy link

codspeed-hq bot commented Mar 1, 2025 •
edited
Loading

CodSpeed Performance Report

Merging #2560 will degrade performances by 49.78%

Comparing rk/removefill (d113ae4) with develop (156ccae)

Summary

⚡ 23 improvements
❌ 21 regressions
✅ 731 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
❌ compress_alp[f32, (1000, 0.0, 0.25)] 80.1 µs 110.6 µs -27.61%
❌ compress_alp[f32, (1000, 0.0, 0.95)] 79.9 µs 110.3 µs -27.55%
❌ compress_alp[f32, (1000, 0.0, 1.0)] 72.4 µs 102.5 µs -29.39%
❌ compress_alp[f32, (10000, 0.0, 0.25)] 156.3 µs 303.6 µs -48.51%
❌ compress_alp[f32, (10000, 0.0, 0.95)] 155.9 µs 303.5 µs -48.63%
❌ compress_alp[f32, (10000, 0.0, 1.0)] 148.2 µs 295.1 µs -49.78%
⚡ compress_alp[f32, (10000, 0.1, 0.95)] 346 µs 307.4 µs +12.55%
❌ compress_alp[f64, (1000, 0.0, 0.25)] 201 µs 278.9 µs -27.91%
❌ compress_alp[f64, (1000, 0.0, 0.95)] 201.1 µs 278.9 µs -27.88%
❌ compress_alp[f64, (1000, 0.0, 1.0)] 193.4 µs 270.9 µs -28.59%
❌ compress_alp[f64, (1000, 0.01, 0.25)] 242.8 µs 294.8 µs -17.66%
❌ compress_alp[f64, (1000, 0.01, 0.95)] 243.5 µs 294.8 µs -17.4%
❌ compress_alp[f64, (1000, 0.01, 1.0)] 234.2 µs 286.8 µs -18.33%
❌ compress_alp[f64, (1000, 0.1, 0.25)] 246.8 µs 294.9 µs -16.31%
❌ compress_alp[f64, (1000, 0.1, 0.95)] 251.3 µs 294.8 µs -14.75%
❌ compress_alp[f64, (1000, 0.1, 1.0)] 235.5 µs 286.9 µs -17.92%
❌ compress_alp[f64, (10000, 0.0, 0.25)] 317.1 µs 517.5 µs -38.72%
❌ compress_alp[f64, (10000, 0.0, 0.95)] 317.1 µs 517.3 µs -38.71%
❌ compress_alp[f64, (10000, 0.0, 1.0)] 308.2 µs 508.7 µs -39.41%
❌ compress_alp[f64, (10000, 0.01, 0.25)] 477.9 µs 533.2 µs -10.36%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 1, 2025

Benchmarks: TPC-H on S3

Table of Results
name PR 229c44c base 3876b18 ratio (PR/base) unit
tpch_q01/parquet 316463639 3.0077e+08 1.05218 ns
tpch_q02/parquet 727433830 7.50286e+08 0.969542 ns
tpch_q03/parquet 464001509 4.73968e+08 0.978971 ns
tpch_q04/parquet 298867889 2.69489e+08 1.10902 ns
tpch_q05/parquet 633626967 6.39539e+08 0.990756 ns
tpch_q06/parquet 189300764 2.22209e+08 0.851903 ns
tpch_q07/parquet 726146427 7.44838e+08 0.974905 ns
tpch_q08/parquet 898758180 8.92551e+08 1.00695 ns
tpch_q09/parquet 995526664 7.51724e+08 1.32433 ns
tpch_q10/parquet 582960966 6.04673e+08 0.964093 ns
tpch_q11/parquet 317165411 3.07629e+08 1.031 ns
tpch_q12/parquet 306362466 3.28007e+08 0.934011 ns
tpch_q13/parquet 455485950 4.35188e+08 1.04664 ns
tpch_q14/parquet 289616486 2.82498e+08 1.0252 ns
tpch_q15/parquet 524455521 5.05225e+08 1.03806 ns
tpch_q16/parquet 287947841 3.03119e+08 0.949951 ns
tpch_q17/parquet 440449342 5.25025e+08 0.838912 ns
tpch_q18/parquet 632301105 6.9253e+08 0.91303 ns
tpch_q19/parquet 324571861 3.38607e+08 0.95855 ns
tpch_q20/parquet 536402834 5.90716e+08 0.908056 ns
tpch_q21/parquet 733894642 7.86658e+08 0.932927 ns
tpch_q22/parquet 338076013 3.31895e+08 1.01862 ns
tpch_q01/vortex-file-compressed 447358035 5.44557e+08 0.821509 ns
tpch_q02/vortex-file-compressed 470346497 4.85636e+08 0.968516 ns
tpch_q03/vortex-file-compressed 518220827 6.12824e+08 0.845627 ns
tpch_q04/vortex-file-compressed 461002584 4.16593e+08 1.1066 ns
tpch_q05/vortex-file-compressed 604825043 6.53491e+08 0.925529 ns
tpch_q06/vortex-file-compressed 510737321 4.34116e+08 1.1765 ns
tpch_q07/vortex-file-compressed 809338293 7.64274e+08 1.05896 ns
tpch_q08/vortex-file-compressed 821271721 8.41536e+08 0.97592 ns
tpch_q09/vortex-file-compressed 631138761 7.6849e+08 0.821271 ns
tpch_q10/vortex-file-compressed 638480436 6.31854e+08 1.01049 ns
tpch_q11/vortex-file-compressed 170283561 1.75257e+08 0.971619 ns
tpch_q12/vortex-file-compressed 556813036 5.68894e+08 0.978765 ns
tpch_q13/vortex-file-compressed 173224312 2.80536e+08 0.617476 ns
tpch_q14/vortex-file-compressed 394574834 5.53813e+08 0.712469 ns
tpch_q15/vortex-file-compressed 916185217 8.99121e+08 1.01898 ns
tpch_q16/vortex-file-compressed 208243234 1.92601e+08 1.08122 ns
tpch_q17/vortex-file-compressed 498025478 4.50706e+08 1.10499 ns
tpch_q18/vortex-file-compressed 467853420 4.60755e+08 1.01541 ns
tpch_q19/vortex-file-compressed 659042356 6.46301e+08 1.01971 ns
tpch_q20/vortex-file-compressed 630517192 6.12982e+08 1.02861 ns
tpch_q21/vortex-file-compressed 1046638053 1.11783e+09 0.936309 ns
tpch_q22/vortex-file-compressed 182780958 1.80907e+08 1.01036 ns

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 1, 2025

Benchmarks: TPC-H on NVME

Table of Results
name PR 229c44c base 3876b18 ratio (PR/base) unit
tpch_q01/arrow 77967515 8.2076e+07 0.949943 ns
tpch_q02/arrow 41844587 4.04799e+07 1.03371 ns
tpch_q03/arrow 33679071 3.35842e+07 1.00283 ns
tpch_q04/arrow 31114277 3.14169e+07 0.990366 ns
tpch_q05/arrow 56147597 5.66573e+07 0.991004 ns
tpch_q06/arrow 8084093 8.47529e+06 0.953843 ns
tpch_q07/arrow 99267656 9.87179e+07 1.00557 ns
tpch_q08/arrow 57356886 5.64108e+07 1.01677 ns
tpch_q09/arrow 84208163 8.811e+07 0.955717 ns
tpch_q10/arrow 54579300 5.64352e+07 0.967114 ns
tpch_q11/arrow 24882702 2.48061e+07 1.00309 ns
tpch_q12/arrow 30580952 3.14941e+07 0.971007 ns
tpch_q13/arrow 25320576 2.55714e+07 0.990192 ns
tpch_q14/arrow 11262546 1.21953e+07 0.923517 ns
tpch_q15/arrow 24832587 2.55401e+07 0.972298 ns
tpch_q16/arrow 21298608 2.1391e+07 0.995683 ns
tpch_q17/arrow 79461264 8.18726e+07 0.970548 ns
tpch_q18/arrow 160501110 1.63659e+08 0.980706 ns
tpch_q19/arrow 25406231 2.69504e+07 0.942704 ns
tpch_q20/arrow 37153701 3.89532e+07 0.953802 ns
tpch_q21/arrow 156186867 1.59941e+08 0.976526 ns
tpch_q22/arrow 16673711 1.72224e+07 0.968143 ns
tpch_q01/parquet 151706303 1.52532e+08 0.994588 ns
tpch_q02/parquet 94371855 9.95795e+07 0.947704 ns
tpch_q03/parquet 105248181 1.03389e+08 1.01799 ns
tpch_q04/parquet 60363559 6.11801e+07 0.986654 ns
tpch_q05/parquet 114824523 1.13047e+08 1.01572 ns
tpch_q06/parquet 37001657 2.94135e+07 1.25798 ns
tpch_q07/parquet 145250149 1.48485e+08 0.978213 ns
tpch_q08/parquet 146180985 1.53358e+08 0.953202 ns
tpch_q09/parquet 193743876 1.99785e+08 0.96976 ns
tpch_q10/parquet 147185628 1.52592e+08 0.96457 ns
tpch_q11/parquet 46870021 4.76495e+07 0.983641 ns
tpch_q12/parquet 99271752 8.46947e+07 1.17211 ns
tpch_q13/parquet 184345070 1.86418e+08 0.988882 ns
tpch_q14/parquet 67418104 4.89665e+07 1.37682 ns
tpch_q15/parquet 88073402 8.68283e+07 1.01434 ns
tpch_q16/parquet 47321566 4.74699e+07 0.996876 ns
tpch_q17/parquet 151294395 1.51938e+08 0.995762 ns
tpch_q18/parquet 231669319 2.50219e+08 0.925866 ns
tpch_q19/parquet 83864990 8.74299e+07 0.959226 ns
tpch_q20/parquet 93577862 1.00421e+08 0.931856 ns
tpch_q21/parquet 211128619 2.16168e+08 0.976687 ns
tpch_q22/parquet 46601362 5.03133e+07 0.926224 ns
tpch_q01/vortex-file-compressed 61587414 6.13439e+07 1.00397 ns
tpch_q02/vortex-file-compressed 46446701 4.73942e+07 0.980009 ns
tpch_q03/vortex-file-compressed 32952339 3.3489e+07 0.983974 ns
tpch_q04/vortex-file-compressed 21262727 2.08985e+07 1.01743 ns
tpch_q05/vortex-file-compressed 54004298 5.53272e+07 0.97609 ns
tpch_q06/vortex-file-compressed 10836238 1.11617e+07 0.970839 ns
tpch_q07/vortex-file-compressed 86286272 9.02019e+07 0.956591 ns
tpch_q08/vortex-file-compressed 61283687 6.29762e+07 0.973125 ns
tpch_q09/vortex-file-compressed 86203857 9.24235e+07 0.932705 ns
tpch_q10/vortex-file-compressed 57343280 6.01737e+07 0.952962 ns
tpch_q11/vortex-file-compressed 24408706 2.53432e+07 0.963127 ns
tpch_q12/vortex-file-compressed 29733615 3.07092e+07 0.968232 ns
tpch_q13/vortex-file-compressed 30542903 3.25691e+07 0.937787 ns
tpch_q14/vortex-file-compressed 14010242 1.48133e+07 0.945788 ns
tpch_q15/vortex-file-compressed 27410057 2.93939e+07 0.932509 ns
tpch_q16/vortex-file-compressed 25868405 2.54459e+07 1.01661 ns
tpch_q17/vortex-file-compressed 76977646 7.84698e+07 0.980985 ns
tpch_q18/vortex-file-compressed 141627215 1.43338e+08 0.988061 ns
tpch_q19/vortex-file-compressed 30956211 3.28929e+07 0.94112 ns
tpch_q20/vortex-file-compressed 38882285 4.13326e+07 0.940717 ns
tpch_q21/vortex-file-compressed 126830382 1.28753e+08 0.985066 ns
tpch_q22/vortex-file-compressed 28671575 2.86123e+07 1.00207 ns

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 1, 2025

Benchmarks: Clickbench on NVME

Table of Results
name PR 229c44c base 3876b18 ratio (PR/base) unit
clickbench_q00/parquet 2296841 2.17594e+06 1.05556 ns
clickbench_q01/parquet 65962743 6.06963e+07 1.08677 ns
clickbench_q02/parquet 125052757 1.18489e+08 1.05539 ns
clickbench_q03/parquet 93218362 8.48505e+07 1.09862 ns
clickbench_q04/parquet 752370905 6.55058e+08 1.14856 ns
clickbench_q05/parquet 822219753 7.42261e+08 1.10772 ns
clickbench_q06/parquet 2820990 2.23779e+06 1.26061 ns
clickbench_q07/parquet 68410431 6.43791e+07 1.06262 ns
clickbench_q08/parquet 823897223 7.48457e+08 1.10079 ns
clickbench_q09/parquet 1143563144 1.01234e+09 1.12962 ns
clickbench_q10/parquet 289945822 2.6816e+08 1.08124 ns
clickbench_q11/parquet 338326190 3.13543e+08 1.07904 ns
clickbench_q12/parquet 838426395 7.4418e+08 1.12664 ns
clickbench_q13/parquet 1078847750 9.9313e+08 1.08631 ns
clickbench_q14/parquet 812569523 7.33632e+08 1.1076 ns
clickbench_q15/parquet 848770210 7.2553e+08 1.16986 ns
clickbench_q16/parquet 1767698695 1.59837e+09 1.10594 ns
clickbench_q17/parquet 1660527806 1.44934e+09 1.14571 ns
clickbench_q18/parquet 3317465666 2.97662e+09 1.11451 ns
clickbench_q19/parquet 79745793 7.10605e+07 1.12222 ns
clickbench_q20/parquet 1239826700 1.10275e+09 1.12431 ns
clickbench_q21/parquet 1395129795 1.30389e+09 1.06997 ns
clickbench_q22/parquet 2064251241 1.88044e+09 1.09775 ns
clickbench_q23/parquet 8737922453 7.71748e+09 1.13222 ns
clickbench_q24/parquet 519743786 4.50509e+08 1.15368 ns
clickbench_q25/parquet 436575082 3.95443e+08 1.10401 ns
clickbench_q26/parquet 559070067 5.03471e+08 1.11043 ns
clickbench_q27/parquet 1734587878 1.62504e+09 1.06741 ns
clickbench_q28/parquet 11897804103 1.15453e+10 1.03053 ns
clickbench_q29/parquet 475558060 4.42768e+08 1.07406 ns
clickbench_q30/parquet 764606514 6.98433e+08 1.09475 ns
clickbench_q31/parquet 824566807 7.42762e+08 1.11014 ns
clickbench_q32/parquet 3095555425 2.81295e+09 1.10046 ns
clickbench_q33/parquet 3218260937 2.89118e+09 1.11313 ns
clickbench_q34/parquet 3227183868 2.78547e+09 1.15858 ns
clickbench_q35/parquet 1003613724 8.58097e+08 1.16958 ns
clickbench_q36/parquet 203255433 1.81025e+08 1.12281 ns
clickbench_q37/parquet 97595712 8.78403e+07 1.11106 ns
clickbench_q38/parquet 121837327 1.10762e+08 1.09999 ns
clickbench_q39/parquet 351538490 3.27297e+08 1.07407 ns
clickbench_q40/parquet 61290114 5.72972e+07 1.06969 ns
clickbench_q41/parquet 59056223 5.32687e+07 1.10865 ns
clickbench_q42/parquet 76501885 7.26249e+07 1.05338 ns
clickbench_q00/vortex-file-compressed 5484627 4.44533e+06 1.2338 ns
clickbench_q01/vortex-file-compressed 25851091 2.14774e+07 1.20364 ns
clickbench_q02/vortex-file-compressed 46215398 4.19403e+07 1.10193 ns
clickbench_q03/vortex-file-compressed 70403022 6.32664e+07 1.1128 ns
clickbench_q04/vortex-file-compressed 671841789 5.83543e+08 1.15132 ns
clickbench_q05/vortex-file-compressed 654993441 6.00829e+08 1.09015 ns
clickbench_q06/vortex-file-compressed 5063646 4.6464e+06 1.0898 ns
clickbench_q07/vortex-file-compressed 27143572 2.62505e+07 1.03402 ns
clickbench_q08/vortex-file-compressed 749609957 6.73662e+08 1.11274 ns
clickbench_q09/vortex-file-compressed 915847553 8.06944e+08 1.13496 ns
clickbench_q10/vortex-file-compressed 123551619 1.14639e+08 1.07774 ns
clickbench_q11/vortex-file-compressed 139981766 1.33247e+08 1.05054 ns
clickbench_q12/vortex-file-compressed 538640421 4.7534e+08 1.13317 ns
clickbench_q13/vortex-file-compressed 782644061 7.02374e+08 1.11428 ns
clickbench_q14/vortex-file-compressed 475405629 4.43421e+08 1.07213 ns
clickbench_q15/vortex-file-compressed 753137366 6.81376e+08 1.10532 ns
clickbench_q16/vortex-file-compressed 1456902914 1.37742e+09 1.0577 ns
clickbench_q17/vortex-file-compressed 1490609237 1.28108e+09 1.16356 ns
clickbench_q18/vortex-file-compressed 3031999759 2.82549e+09 1.07309 ns
clickbench_q19/vortex-file-compressed 37799053 3.57051e+07 1.05865 ns
clickbench_q20/vortex-file-compressed 799063571 7.41384e+08 1.0778 ns
clickbench_q21/vortex-file-compressed 898984853 8.10899e+08 1.10863 ns
clickbench_q22/vortex-file-compressed 1254552718 1.12972e+09 1.1105 ns
clickbench_q23/vortex-file-compressed 2150426065 1.98311e+09 1.08437 ns
clickbench_q24/vortex-file-compressed 190181688 1.83519e+08 1.03631 ns
clickbench_q25/vortex-file-compressed 205791712 1.89913e+08 1.08361 ns
clickbench_q26/vortex-file-compressed 243864979 2.28963e+08 1.06508 ns
clickbench_q27/vortex-file-compressed 1272424248 1.23216e+09 1.03268 ns
clickbench_q28/vortex-file-compressed 10515770111 1.03537e+10 1.01566 ns
clickbench_q29/vortex-file-compressed 733784673 7.0231e+08 1.04482 ns
clickbench_q30/vortex-file-compressed 416210670 3.97978e+08 1.04581 ns
clickbench_q31/vortex-file-compressed 430958025 4.0201e+08 1.07201 ns
clickbench_q32/vortex-file-compressed 2871571743 2.75505e+09 1.04229 ns
clickbench_q33/vortex-file-compressed 2652768028 2.37637e+09 1.11631 ns
clickbench_q34/vortex-file-compressed 2679734696 2.38659e+09 1.12283 ns
clickbench_q35/vortex-file-compressed 1031365889 9.2875e+08 1.11049 ns
clickbench_q36/vortex-file-compressed 109184597 1.00751e+08 1.08371 ns
clickbench_q37/vortex-file-compressed 59844933 5.91698e+07 1.01141 ns
clickbench_q38/vortex-file-compressed 44929085 4.09277e+07 1.09777 ns
clickbench_q39/vortex-file-compressed 178637557 1.64619e+08 1.08516 ns
clickbench_q40/vortex-file-compressed 35325657 3.24854e+07 1.08743 ns
clickbench_q41/vortex-file-compressed 33397956 3.17104e+07 1.05322 ns
clickbench_q42/vortex-file-compressed 55155726 5.22334e+07 1.05595 ns

@robert3005 robert3005 marked this pull request as draft March 1, 2025 00:14
@robert3005
Copy link
Member Author

this requires deeper changes

@robert3005 robert3005 closed this Mar 1, 2025
@robert3005 robert3005 reopened this Mar 1, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 1, 2025

Benchmarks: compress

Table of Results
name PR 229c44c base 3876b18 ratio (PR/base) unit
compress time/taxi throughput 0.213094 0.210739 1.01117 bytes/ns
parquet_rs-zstd compress time/taxi throughput 0.280821 0.271174 1.03557 bytes/ns
decompress time/taxi throughput 1.77033 1.66692 1.06203 bytes/ns
parquet_rs-zstd decompress time/taxi throughput 1.64071 1.61779 1.01416 bytes/ns
compress time/AirlineSentiment throughput 0.0026848 0.00278896 0.962652 bytes/ns
parquet_rs-zstd compress time/AirlineSentiment throughput 0.0537083 0.0516339 1.04017 bytes/ns
decompress time/AirlineSentiment throughput 0.0288399 0.022758 1.26724 bytes/ns
parquet_rs-zstd decompress time/AirlineSentiment throughput 0.0980545 0.0977259 1.00336 bytes/ns
compress time/Arade throughput 0.153921 0.136911 1.12425 bytes/ns
parquet_rs-zstd compress time/Arade throughput 0.394867 0.38295 1.03112 bytes/ns
decompress time/Arade throughput 1.81225 1.75335 1.03359 bytes/ns
parquet_rs-zstd decompress time/Arade throughput 1.90459 1.87922 1.0135 bytes/ns
compress time/Bimbo throughput 0.367795 0.343265 1.07146 bytes/ns
parquet_rs-zstd compress time/Bimbo throughput 0.336615 0.326223 1.03186 bytes/ns
decompress time/Bimbo throughput 2.19772 2.21899 0.990415 bytes/ns
parquet_rs-zstd decompress time/Bimbo throughput 2.80727 2.76713 1.01451 bytes/ns
compress time/CMSprovider throughput 0.0532993 0.0515006 1.03493 bytes/ns
parquet_rs-zstd compress time/CMSprovider throughput 0.349281 0.340794 1.02491 bytes/ns
decompress time/CMSprovider throughput 3.9987 3.73354 1.07102 bytes/ns
parquet_rs-zstd decompress time/CMSprovider throughput 1.81993 1.75559 1.03665 bytes/ns
compress time/Euro2016 throughput 0.157686 0.148406 1.06253 bytes/ns
parquet_rs-zstd compress time/Euro2016 throughput 0.30088 0.293002 1.02689 bytes/ns
decompress time/Euro2016 throughput 2.71234 2.37557 1.14176 bytes/ns
parquet_rs-zstd decompress time/Euro2016 throughput 0.988136 0.968413 1.02037 bytes/ns
compress time/Food throughput 0.191076 0.1775 1.07648 bytes/ns
parquet_rs-zstd compress time/Food throughput 0.317791 0.313219 1.0146 bytes/ns
decompress time/Food throughput 5.181 5.03614 1.02877 bytes/ns
parquet_rs-zstd decompress time/Food throughput 1.57566 1.57888 0.997964 bytes/ns
compress time/HashTags throughput 0.190643 0.187772 1.01529 bytes/ns
parquet_rs-zstd compress time/HashTags throughput 0.8079 0.788099 1.02513 bytes/ns
decompress time/HashTags throughput 6.24916 5.6508 1.10589 bytes/ns
parquet_rs-zstd decompress time/HashTags throughput 2.73176 2.58158 1.05817 bytes/ns
compress time/TPC-H l_comment chunked throughput 0.214721 0.207778 1.03342 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput 0.283188 0.276574 1.02391 bytes/ns
decompress time/TPC-H l_comment chunked throughput 3.12265 3.00136 1.04041 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput 1.0795 1.0818 0.997874 bytes/ns
compress time/TPC-H l_comment canonical throughput 0.0293542 0.0283844 1.03417 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput 0.282387 0.277614 1.01719 bytes/ns
decompress time/TPC-H l_comment canonical throughput 3.17063 3.03441 1.04489 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput 1.07217 1.05108 1.02006 bytes/ns
compress time/wide table cols=10 chunks=1 rows=1000 throughput 0.127835 0.126439 1.01104 bytes/ns
parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 throughput 0.186557 0.188203 0.991257 bytes/ns
decompress time/wide table cols=10 chunks=1 rows=1000 throughput 0.674759 0.790965 0.853083 bytes/ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 throughput 0.494127 0.505761 0.976997 bytes/ns
compress time/wide table cols=100 chunks=1 rows=1000 throughput 0.128978 0.125431 1.02827 bytes/ns
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 throughput 0.195225 0.174463 1.11901 bytes/ns
decompress time/wide table cols=100 chunks=1 rows=1000 throughput 1.06095 1.07204 0.989654 bytes/ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 throughput 0.488751 0.489995 0.997461 bytes/ns
compress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.120501 0.11615 1.03745 bytes/ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.165911 0.164316 1.0097 bytes/ns
decompress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.830848 0.746379 1.11317 bytes/ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.43908 0.406062 1.08131 bytes/ns
compress time/wide table cols=10 chunks=50 rows=1000 throughput 0.0680649 0.0666914 1.02059 bytes/ns
parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 throughput 0.128803 0.123983 1.03888 bytes/ns
decompress time/wide table cols=10 chunks=50 rows=1000 throughput 0.665805 0.648358 1.02691 bytes/ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 throughput 0.475412 0.517687 0.918339 bytes/ns
compress time/wide table cols=100 chunks=50 rows=1000 throughput 0.0683907 0.0579308 1.18056 bytes/ns
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 throughput 0.126011 0.108279 1.16376 bytes/ns
decompress time/wide table cols=100 chunks=50 rows=1000 throughput 1.09792 1.12689 0.974292 bytes/ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 throughput 0.48278 0.496603 0.972166 bytes/ns
compress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.0553512 0.0534215 1.03612 bytes/ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.0984654 0.0886248 1.11104 bytes/ns
decompress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.84088 0.719841 1.16815 bytes/ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.43078 0.386015 1.11597 bytes/ns
vortex:raw size/taxi 0.119118 0.118637 1.00406
vortex size/taxi 5.89345e+07 5.86962e+07 1.00406
vortex:parquet-zstd size/taxi 1.05313 1.04887 1.00406
vortex:raw size/AirlineSentiment 1.35456 1.35456 1
vortex size/AirlineSentiment 4424 4424 1
vortex:parquet-zstd size/AirlineSentiment 4.57497 4.57497 1
vortex:raw size/Arade 0.26608 0.255857 1.03996
vortex size/Arade 3.15752e+08 3.03621e+08 1.03996
vortex:parquet-zstd size/Arade 1.03392 0.994196 1.03996
vortex:raw size/Bimbo 0.11585 0.11691 0.990931
vortex size/Bimbo 8.28226e+08 8.35806e+08 0.990931
vortex:parquet-zstd size/Bimbo 2.13377 2.1533 0.990931
vortex:raw size/CMSprovider 0.187916 0.185111 1.01515
vortex size/CMSprovider 1.18107e+09 1.1628e+09 1.01572
vortex:parquet-zstd size/CMSprovider 1.53481 1.51106 1.01572
vortex:raw size/Euro2016 0.474912 0.471341 1.00758
vortex size/Euro2016 2.16096e+08 2.14471e+08 1.00758
vortex:parquet-zstd size/Euro2016 1.81761 1.80394 1.00758
vortex:raw size/Food 0.182981 0.177321 1.03192
vortex size/Food 6.16359e+07 5.97294e+07 1.03192
vortex:parquet-zstd size/Food 1.70124 1.64862 1.03192
vortex:raw size/HashTags 0.151464 0.142603 1.06214
vortex size/HashTags 2.90183e+08 2.73207e+08 1.06214
vortex:parquet-zstd size/HashTags 2.16525 2.03788 1.0625
vortex:raw size/TPC-H l_comment chunked 0.417792 0.416826 1.00232
vortex size/TPC-H l_comment chunked 1.04113e+08 1.03872e+08 1.00232
vortex:parquet-zstd size/TPC-H l_comment chunked 1.8285 1.82425 1.00233
vortex:raw size/TPC-H l_comment canonical 0.425484 0.425546 0.999853
vortex size/TPC-H l_comment canonical 1.06027e+08 1.06042e+08 0.999853
vortex:parquet-zstd size/TPC-H l_comment canonical 1.8623 1.86258 0.999849
vortex:raw size/wide table cols=10 chunks=1 rows=1000 0.626224 0.626224 1
vortex size/wide table cols=10 chunks=1 rows=1000 100256 100256 1
vortex:parquet-zstd size/wide table cols=10 chunks=1 rows=1000 1.07244 1.07244 1
vortex:raw size/wide table cols=100 chunks=1 rows=1000 0.623267 0.623267 1
vortex size/wide table cols=100 chunks=1 rows=1000 997736 997736 1
vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000 1.06732 1.06732 1
vortex:raw size/wide table cols=1000 chunks=1 rows=1000 0.622971 0.622971 1
vortex size/wide table cols=1000 chunks=1 rows=1000 9.97254e+06 9.97254e+06 1
vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000 1.06681 1.06681 1
vortex:raw size/wide table cols=10 chunks=50 rows=1000 0.6098 0.6098 1
vortex size/wide table cols=10 chunks=50 rows=1000 100256 100256 1
vortex:parquet-zstd size/wide table cols=10 chunks=50 rows=1000 1.07244 1.07244 1
vortex:raw size/wide table cols=100 chunks=50 rows=1000 0.608224 0.608224 1
vortex size/wide table cols=100 chunks=50 rows=1000 997736 997736 1
vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000 1.06732 1.06732 1
vortex:raw size/wide table cols=1000 chunks=50 rows=1000 0.608066 0.608066 1
vortex size/wide table cols=1000 chunks=50 rows=1000 9.97254e+06 9.97254e+06 1
vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000 1.06681 1.06681 1

@gatesn
Copy link
Contributor

gatesn commented Mar 1, 2025

Why?

@robert3005
Copy link
Member Author

Mostly trying to compare things to original alp implementation which doesn’t do this

@robert3005 robert3005 closed this Mar 1, 2025
@robert3005 robert3005 deleted the rk/removefill branch March 1, 2025 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@robert3005 @gatesn