Enable OpaquePointers on LLVM 15

[vchuravy]

No description provided.

[vchuravy]

@nanosoldier runbenchmarks(!"scalar", vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

[maleadt]

Opaque pointers are a per-context setting, right? It would be nice if the GPU stack can keep working with opaque pointers disabled, as we have a couple of (closed-source) back-ends that haven't been adapted yet.

[Seelengrab]

If this becomes a flag in GPUCompiler, it would be great to have some API for setting that as well.

[maleadt]

LLVM.jl has a wrapper for the LLVM C API already: https://github.com/maleadt/LLVM.jl/blob/e44c430c4a48225114f217c36b166b85a3fcbe5a/lib/15/libLLVM_h.jl#L602-L604. Now that GPUCompiler create its own contexts for use with Julia, theoretically this should just work, but we should verify.

[vtjnash]

@nanosoldier runbenchmarks("union", vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

[vchuravy]

@pxl-th @maleadt do you think the GPU ecosystem is at a point where turning opaque-pointer should be fine?

@vtjnash and I are inclined on giving this a go.

[pxl-th]

Last time I tried opaque pointers it needed some adjustments on GPUCompiler side:
JuliaGPU/GPUCompiler.jl#511.

[vtjnash]

I changed this PR so it only turns them on for the JIT and not globally.

[maleadt]

The main problem isn't GPUCompiler, it's the out-of-tree back-ends (SPIR-V translator for oneAPI, the AIR "back-end" for Metal, and NVVM for CUDA). Updating these is going to be a pain regardless, so it shouldn't hold up the migration here. I just was hoping that the next LTS would still support typed pointers, so that we have something to point users to while we're updating back-ends.

[maleadt]

I've been testing this, and it works great. I wonder if we could have the NFC version of this in 1.10, i.e., calling setOpaquePointers instead of using the global option. That way, we can start migrating GPUCompiler even if Julia is still using typed pointers (currently, using the global option "taints" the context, making it impossible to change the setting afterwards).
EDIT: see #51840

[maleadt]

For the record, with #51840 it doesn't matter whether Julia uses opaque pointers or not (LLVM's default is still typed, and Julia doesn't use the clopt anymore allowing GPUCompiler to override per-context). But I've also started work on updating GPUCompiler to opaque pointers in JuliaGPU/GPUCompiler.jl#528. Regardless, it should be fine to flip the switch here.

Let's see what else breaks though:

@nanosoldier runtests()

[nanosoldier]

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

[vchuravy]

Will fix #52363 on for 1.11-dev

[gbaraldi]

Opaque pointers introduce a bunch of regressions that are only fixed on a patched llvm 16

[vtjnash]

Did it get worse? The results above seemed to look fine
@nanosoldier runbenchmarks(!"scalar", vs=":master")

[gbaraldi]

Unions are on average 50% slower

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

[vchuravy]

Superseded by #51720