Skip to content

Commit

Permalink
Compress the lossless part of MGARD data with Zstd. It was uncompress…
Browse files Browse the repository at this point in the history 
…ed until now. (ornladios#4339)

$ ctest -R MGARD.*/.*BP5.Serial
$ find testing/adios2/engine/bp/operations/bp5 -name "*bp" -exec du -sh {} \;
48K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.00001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.01.bp
36K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.00001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.0001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.00001.bp
36K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.0001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.0001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.01.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.00001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.01.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.0001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.01.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.001.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.01.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.0001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.0001.bp
32K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.01.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.00001.bp
40K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.01.bp
44K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.0001.bp

Before this commit:

$ find testing/adios2/engine/bp/operations/bp5 -name "*bp" -exec du -sh {} \;
352K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.00001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.01.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.00001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.0001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.00001.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.0001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.0001.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.01.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.00001.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.01.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.001.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.0001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1D_0.01.bp
352K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.001.bp
344K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.01.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.0001.bp
100K    testing/adios2/engine/bp/operations/bp5/BPWRMGARDNull_0.00001.bp
60K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD2D_0.0001.bp
96K     testing/adios2/engine/bp/operations/bp5/BPWRMGARD3D_0.01.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD2DSel_0.00001.bp
176K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD1DSel_0.01.bp
352K    testing/adios2/engine/bp/operations/bp5/BPWRMGARD3DSel_0.0001.bp

(cherry picked from commit d1d871b)
  • Loading branch information
@pnorbert @vicentebolea
pnorbert authored and vicentebolea committed Oct 23, 2024
1 parent 0db55f0 commit ebaa549
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion source/adios2/operator/compress/CompressMGARD.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -151,10 +151,13 @@ size_t CompressMGARD::Operate(const char *dataIn, const Dims &blockStart, const
return 0;
}

mgard_x::Config config;
config.lossless = mgard_x::lossless_type::Huffman_Zstd;

PutParameter(bufferOut, bufferOutOffset, true);
void *compressedData = bufferOut + bufferOutOffset;
mgard_x::compress(mgardDim, mgardType, mgardCount, tolerance, s, errorBoundType, dataIn,
compressedData, sizeOut, true);
compressedData, sizeOut, config, true);
bufferOutOffset += sizeOut;

return bufferOutOffset;
Expand Down

0 comments on commit ebaa549

Please sign in to comment.