what is the purpose of ggml_half2 dm in quantization structure?

[PenutChen]

I notice that there is ggml_half2 dm in many k-quant structures, e.g. block_q2_K and block_q4_K , but I'm not sure of its purpose. I didn't see it being used in quantization or dequantization. What is the purpose of this value? Is it for memory alignment or something?

[ggerganov]

It's used in GPU code that supports half2 ops. For example in CUDA:

https://github.com/ggerganov/ggml/blob/a3c0188a4b5d3dec052ff87c9f773baa53631d70/src/ggml-cuda/fattn-common.cuh#L120

Since this is a union, the dm member is basically an alias for the d and dmin factors

[PenutChen]

Thanks for explaining, really cool tips!