Plan to upstream SYCL to LLVM project.

[bader]

Milestones

• Compile device part of "hello world"-like SYCL application.
• Convert device part of the "hello world"-like SYCL application to SPIR-V format.
• Run "hello world"-like SYCL application via OpenCL on GPU.
• Integrate SYCL compilation flow into clang driver.
• Compile/run multi-source application
• Compile/run multi-source application linking with an external SYCL library

Components to upstream

Device code outlining

Current approach: we added two new attributes to the compiler: sycl_kernel and sycl_device. SYCL runtime is supposed to use these attributes to mark functions in the single source which are supposed to be compiled for the device. Compiler is supposed to understand if there are other functions/constants/variables needed to compiler.

Idea(s) to try: clang has infrastructure to mark the code compiler is supposed to offload for CUDA. We can utilize this infrastructure. E.g. AdaptiveCpp/AdaptiveCpp#34 (comment) contains some ideas how already supported functionality can be re-used.

Patch to LLVM: https://reviews.llvm.org/D60455

Integration header

Current approach: SYCL device compiler emits "integration header", which provides "implementation defined" part of the host/device ABI (e.g. order of parameters captured by SYCL kernel, SYCL kernel function names, etc.)

Idea(s) to try: integration header is required only if SYCL device compiler is used custom host compiler. To enable compilation flow with clang compiler only, we could emit host-side object file instead of integration header.

Driver support

Current approach: we added new toolchain to support SYCL compilation flow. This toolchain handles host/device compilers invocation, translation to SPIR-V, etc.

Idea(s) to try: current approach can be improved by implementing suggestions from Andrew to make implementation more friendly for adding custom back-ends support.

Device code diagnostics

Current approach: we implemented new diagnostic messages in SemaSYCL.cpp for most of the requirements in SYCL specification.

Idea(s) to try: re-use existing delayed diagnostics engine similar to OpenMP/CUDA. Re-using existing infrastructure for device code outlining should help here too.

SPIR-V format support

Current approach: SYCL compilation flow introduced a new LLVM transformation pass (CxxReflower) that adjust SPIR-V type and SPIR-V built-in function names declared in the SYCL headers to match the format described here: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/master/docs/SPIRVRepresentationInLLVM.rst

Idea(s) to try: declare SPIR-V built-ins in the SYCL headers, so that transformation is not required and use device specific lib to transform SPIR-V built-ins otherwise. User can link built-ins implementation with -mlink-builtin-bitcode option. We can use SPIR-V built-ins in SYCL headers and define them in device specific library. For instance, Xilinx tool-chain can implement SPIR-V built-ins as SPIR-df built-ins in device specific library, although it's might be better to use something else to avoid name collision between user defined functions and OpenCL built-ins (e.g. __xilinx_sqrt).
This approach is used for lowering AMDGPU/NVidia specific built-ins. Here is recent patch adding tool-chain support for linking with device library.
https://reviews.llvm.org/D59321 - AMDGPU: Teach toolchain to link ROCm device libs. Take a look at the driver test: test/Driver/rocm-device-libs.cl.

SYCL library

Current approach: sub-project in LLVM project. Currently is tested only with Intel OpenCL drivers.

Idea(s) to try: No obvious changes, but can be adjusted based on external users feedback to make it vendor neutral.

[bader]

Also created a wiki page with "upstream status" for SYCL patches: https://github.com/intel/llvm/wiki/SYCL-patches-upstream-status.

[wangliwei-intel]

Hi, @bader
Codeplaysoftware is driving upstreaming their SYCL back-end for tensor in Eigen:
https://bitbucket.org/codeplaysoftware/eigen/src/default/
As far as you know, Is anyone trying to build it with Intel/llvm/sycl?
I'm not sure this is worth a new issue, I can't find a proper issue relevant either, so I just ask here.
Thanks.

[bader]

@wangliwei-intel, I'm not aware of any progress in this direction.

[wangliwei-intel]

Hi, @bader
Then I'll give a try.
Thanks.

[airlied]

ping, is there still a plan to upstream SYCL to llvm?

[bader]

Yes, we still plan to upstream SYCL llvm. Unfortunately, the progress is slower than we would like to see, so any help is very welcome.

I'm glad to know that there is an interest in having SYCL support in upstream llvm.

[bader]

A little bit of statistics before 2023 year starts.
Here is the chart with the code difference between sycl (DPC++) branch and main (llvm.org):

The chart starts 109794 added and 182 deleted lines of code on Jan 29 of 2019 (669 modified files) and ends with 767401 added and 10563 deleted lines of code on Dec 29 of 2022 (5470 modified files).

I also build a chart with git diff --diff-filter=M, which I think should demonstrate the track of changes in community files (i.e. excluding new files added to the sycl branch).

The chart starts 1416 added and 182 deleted lines of code on Jan 29 of 2019 (82 modified files) and ends with 40954 added and 6533 deleted lines of code on Dec 29 of 2022 (817 modified files).

The trend is clear: the difference is growing between DPC++ source code base and LLVM community code base.

[asudarsa]

hi @bader

Based on some of my 'very' preliminary analysis, it seems like we have considerable difference in the test cases available in DPC++ source code and LLVM community code. I can take a closer look if that is something of use.

Thanks

[keryell]

A little bit of statistics before 2023 year starts

Quite interesting. Could you combine the 2 graphics in one and put a log scale on vertical axis? Or send the scripts so I can run them myself and experiment with the graphics. :-)

[keryell]

I'm glad to know that there is an interest in having SYCL support in upstream llvm.

Of course there is! :-)
But considering that Khronos has not finished to produce a Conformance Test Suite for SYCL 2020 yet, there is no official full SYCL 2020 implementation yet and it makes sense to direct the work force mostly on polishing the current SYCL 2020 and preparing SYCL Next rather than up-streaming everything today as is, since it might change a little bit with a better big picture coming from the feedbacks learned with the complete SYCL 2020 implementations.

[bader]

A little bit of statistics before 2023 year starts

Quite interesting. Could you combine the 2 graphics in one and put a log scale on vertical axis? Or send the scripts so I can run them myself and experiment with the graphics. :-)

Here is the bash command I used to get the data:

for D in {0..47}
do
  x=`git log --first-parent --pretty=%H origin/sycl -1 --before="$D months"`
  echo $D , `git diff --diff-filter=M --shortstat -l 0 origin/main...$x`, `git log --pretty='format:%h %ad' -1 $x`

I put the results to the Excel and build these charts there.

To get the data for the first chart, just remove --diff-filter=M flag.

NOTE: there is one outlier in the data. Usually we merge llvm.org patches to the sycl branch with git merge command, but one time we accidentally used git rebase. Somewhere around this time these commands return incorrect values. I used interpolation to make lines continuous.

[bader]

I'm glad to know that there is an interest in having SYCL support in upstream llvm.

Of course there is! :-)

@keryell, you are welcome to join sync meetings if interested (details).

But considering that Khronos has not finished to produce a Conformance Test Suite for SYCL 2020 yet, there is no official full SYCL 2020 implementation yet and it makes sense to direct the work force mostly on polishing the current SYCL 2020 and preparing SYCL Next rather than up-streaming everything today as is, since it might change a little bit with a better big picture coming from the feedbacks learned with the complete SYCL 2020 implementations.

That's basically what's happening right now. In addition to enabling SYCL 2020 features, we are also discussing other improvements which can make the upstream process easier.

[keryell]

@keryell, you are welcome to join sync meetings if interested (details).

@bader Oh thanks. I missed the fact that this meeting started again.

[keryell]

Here is the bash command I used to get the data:

@bader thanks, I will try.

[nullr0ute]

Yes, we still plan to upstream SYCL llvm. Unfortunately, the progress is slower than we would like to see, so any help is very welcome.

I'm glad to know that there is an interest in having SYCL support in upstream llvm.

What's the latest on this initiative, it's hard to know what, if any of the major points above, have been achieved or any links to bugs for the components or requests so hard to know where to assist/look.