Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: Use buffered writing mode in parquet (#203)
Closes #137 #### Benchmarks Performed by the following command in `parquet` dir: ```sh go test \ -test.run=BenchmarkWrite \ -test.bench=BenchmarkWrite \ -test.count 10 -test.benchmem -test.benchtime 10000x ``` <details><summary>Before this update</summary> ``` goos: darwin goarch: arm64 pkg: github.com/cloudquery/filetypes/v3/parquet BenchmarkWrite-10 10000 4628263 ns/op 5796480 B/op 44245 allocs/op BenchmarkWrite-10 10000 4480788 ns/op 5796474 B/op 44245 allocs/op BenchmarkWrite-10 10000 4591783 ns/op 5796471 B/op 44245 allocs/op BenchmarkWrite-10 10000 4610580 ns/op 5796477 B/op 44245 allocs/op BenchmarkWrite-10 10000 4524806 ns/op 5796473 B/op 44245 allocs/op BenchmarkWrite-10 10000 4557667 ns/op 5796466 B/op 44245 allocs/op BenchmarkWrite-10 10000 4869530 ns/op 5796476 B/op 44245 allocs/op BenchmarkWrite-10 10000 4894571 ns/op 5796474 B/op 44245 allocs/op BenchmarkWrite-10 10000 4700499 ns/op 5796468 B/op 44245 allocs/op BenchmarkWrite-10 10000 4793868 ns/op 5796473 B/op 44245 allocs/op PASS ok github.com/cloudquery/filetypes/v3/parquet 539.889s ``` </details> <details><summary>After this update</summary> ``` goos: darwin goarch: arm64 pkg: github.com/cloudquery/filetypes/v3/parquet BenchmarkWrite-10 10000 923740 ns/op 1146573 B/op 15695 allocs/op BenchmarkWrite-10 10000 970047 ns/op 1146193 B/op 15695 allocs/op BenchmarkWrite-10 10000 920979 ns/op 1146542 B/op 15695 allocs/op BenchmarkWrite-10 10000 923738 ns/op 1146486 B/op 15695 allocs/op BenchmarkWrite-10 10000 918581 ns/op 1146055 B/op 15694 allocs/op BenchmarkWrite-10 10000 906547 ns/op 1146690 B/op 15695 allocs/op BenchmarkWrite-10 10000 912946 ns/op 1146381 B/op 15695 allocs/op BenchmarkWrite-10 10000 921024 ns/op 1146378 B/op 15695 allocs/op BenchmarkWrite-10 10000 905637 ns/op 1146371 B/op 15695 allocs/op BenchmarkWrite-10 10000 919410 ns/op 1146494 B/op 15695 allocs/op PASS ok github.com/cloudquery/filetypes/v3/parquet 158.831s ``` </details>
- Loading branch information