[GPU] Match Tile And Fuse skinny matmul bail-out to Vector Distribute

[nirvedhmeshram]

This PR matches the failure criteria to what we see in the SetContractConfig for vector distribute for bailing out on skinny matmuls.

The dispatch in #19855 goes to 0.068 ms vs the default path which gets 1.64 ms as this skinny matmul with multiple dims cannot be currently supported by vector reduction and warp reduction but tile and fuse can support it using padding.

This needs the flag --iree-codegen-llvmgpu-test-tile-and-fuse-matmul=true

Also the GPUMatmulShapeType was becoming too large as this PR adds batch sizes to it and was giving the following error.

 error: static_assert failed due to requirement 'sizeof(mlir::iree_compiler::GPUMatmulShapeType) <= 256' "You are trying to use a default number of inlined elements for SmallVector<T> but sizeof(T) is really big! Please use an explicit number of inlined elements with SmallVector<T, N> to make sure you really want that much inline storage."

This PR fixes this issue both by explictly mentioning vector sizes in the struct members.

[kuhar]

I think we'd need a bit more evidence to change this heuristics. Can you run this over matves-like shapes with, say, 1x to 8x columns in the 'vector' operand and report the numbers? We can add these to iree-kernel-benchmark.

When I benchmarked this in the past, the vector reduction pipeline did a very good job on matvecs with 4 columns.

[nirvedhmeshram]

@kuhar would something like this work?
for i=1,2,4,8
NARROW = [
(i, 128, 128),
(i, 256, 256),
(i, 512, 512),
(i, 1024, 1024),
(i, 2048, 2048),
(i, 4096, 4096),
(i, 8192, 8192),
(128, i, 128),
(256, i, 256),
(512, i, 512),
(1024, i, 1024),
(2048, i, 2048),
(4096, i, 4096),
(8192, i, 8192),
]

[MaheshRavishankar]

Can we make the path that doesn't map to intrinsic directly because of shapes go down the tile and fuse by default?

[kuhar]

Can we make the path that doesn't map to intrinsic directly because of shapes go down the tile and fuse by default?

I think we need this case split:

1. Multiple of the intrinsic of supported type --> Default (vector distribution or tile and fuse)
2. Matvec-like --> warp reduction
3. Everything else --> tile and fuse

[nirvedhmeshram]

Can we make the path that doesn't map to intrinsic directly because of shapes go down the tile and fuse by default?

I think we need this case split:

1. Multiple of the intrinsic of supported type --> Default (vector distribution or tile and fuse)
2. Matvec-like --> warp reduction
3. Everything else --> tile and fuse

point 2. and 3. are in scope for this PR. I have re-invented this PR to make it so that we correctly fail on only those cases that vector reduction would actually support.

[nirvedhmeshram]

@kuhar did not end up doing the shape sweep as I am not changing the heuristic.

[jerryyin]

LGTM

[kuhar]

Seems fine

[nirvedhmeshram]

W7900 runner seems down but since other runners are green, I believe this is okay to merge so landing it.