Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add GPU support for
spark.sql.catalyst.expressions.Slice
(#12183)
Closes #11607 Depends on * NVIDIA/spark-rapids-jni#2921 This PR adds support for `org.apache.spark.sql.catalyst.expressions.Slice`. Ran a simple perf test for 5 times ``` scala // use big data gen import org.apache.spark.sql.tests.datagen._ val dataTable = DBGen().addTable("data", "x array<int>", 10000000) dataTable.toDF(spark).write.mode("OVERWRITE").parquet("PERF") // spark-rapids val df = spark.read.parquet("PERF") spark.time(df.selectExpr("sum(size(slice(x, 2, 3)))").show()) ``` Results: | | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | |----------|-------|-------|-------|-------|-------| | GPU | 325 ms | 329 ms | 334 ms | 328 ms | 341 ms | | CPU | 1307 ms | 1224 ms | 1353 ms | 1185 ms | 1175 ms | The results showed that the `GpuSlice` (TITAN RTX) demonstrated approximately 4 times speedup compared to the `CpuSlice` (8 CPU cores). --------- Signed-off-by: Yan Feng <fengyan_@mail.ustc.edu.cn>
- Loading branch information