Skip mm_mul kernel functions additions if on Intel

[nchudleigh]

Tempfix for issue detailed here #1292

[jhen0409]

It's also related to ggerganov/llama.cpp#2965, there may be more related issues in llama.cpp.

[bobqianic]

It looks good, but this is making encoding really slow.

[nchudleigh]

Otherwise, doesnt seem to work with intel at all. I wonder if its possible to write patches for the kernels.

[nchudleigh]

@ggerganov @bobqianic I dont feel confident in this merge, feels like the fix should be more elegant / actually fix the issue. What do you think?

[bobqianic]

I dont feel confident in this merge, feels like the fix should be more elegant / actually fix the issue.

Agree.

the fix should be more elegant / actually fix the issue

[nchudleigh]

Going to close given the above conversation.

[ggerganov]

I'm not sure what is the best approach here.

@nchudleigh Does using Metal provide significant speed-up compared to running on the CPU for Mac Intel?
I am considering disabling Metal all together for Mac Intel if it does not offer a significant benefit

[nchudleigh]

@ggerganov I missed this entirely!

Hard to say whether it does or not... because the functions are missing I am not sure if the speed is improved or not. Probably depends on the GPU that is available.

[bobqianic]

I've reopened this PR after noticing that @ggerganov addressed a similar issue in #3524. Instead of introducing a new kernel, he chose to disable the buggy kernels on older devices as a fix. We have two choices moving forward:

1. Wait for the next ggml sync.
2. Merge this PR into master directly.

        if ([ctx->device supportsFamily:MTLGPUFamilyApple7]) {
            GGML_METAL_ADD_KERNEL(mul_mm_f32_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_f16_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q4_0_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q8_0_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q4_1_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q2_K_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q3_K_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q4_K_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q5_K_f32);
            GGML_METAL_ADD_KERNEL(mul_mm_q6_K_f32);
        }

[z11h]

@ggerganov any idea when the next GGML sync will be? thank you :)

[ggerganov]

@ggerganov any idea when the next GGML sync will be? thank you :)

Hopefully today or tomorrow

[ggerganov]

Yup, let's try to fix this as part of #1422