Code

Shuffle gpu serde#28

Code

Open

wjxiz1992 wants to merge 25 commits intonvliyuan:0625nvliyuan/yuali-spark-rapids:0625from firestarman:shuffle-gpu-serdefirestarman/spark-rapids:shuffle-gpu-serdeCopy head branch name to clipboard

+2,497-348

Commits

Commits on May 14, 2024

• Support serializing packed tables directly for shuffle write

  Show description for 3a984f2
  firestarman

  committed
  3a984f2

  Copy full SHA for 3a984f2

Commits on May 16, 2024

• Disble GPU serde for the AQE tests

  Show description for baadb4b
  firestarman

  committed
  baadb4b

  Copy full SHA for baadb4b

• Disable by default

  Show description for 11e933d
  firestarman

  committed
  11e933d

  Copy full SHA for 11e933d

Commits on May 17, 2024

• Fix a build error

  Show description for 6e8bb5c
  firestarman

  committed
  6e8bb5c

  Copy full SHA for 6e8bb5c

Commits on May 20, 2024

• Address comments

  Show description for d6082ae
  firestarman

  committed
  d6082ae

  Copy full SHA for d6082ae

• Merge branch 'branch-24.06' of github.com:NVIDIA/spark-rapids into shuffle-gpu-serde

  Show description for 0419224
  firestarman

  committed
  0419224

  Copy full SHA for 0419224

Commits on May 27, 2024

• Support buffering small tables for Shuffle read

  Show description for 99820e1
  firestarman

  committed
  99820e1

  Copy full SHA for 99820e1

Commits on May 28, 2024

• Merge remote-tracking branch 'NVDA/branch-24.08' into shuffle-gpu-serde

  firestarman

  committed
  9727161

  Copy full SHA for 9727161

Commits on May 29, 2024

• Moving split batches to host by a single copying

  Show description for 1bb4cfc
  firestarman

  committed
  1bb4cfc

  Copy full SHA for 1bb4cfc

Commits on Jun 25, 2024

• Add GpuBucketingUtils shim to Spark 4.0.0 (NVIDIA#11092)

  Show description for b3b5b5e
  razajafri

  authored
  b3b5b5e

  Copy full SHA for b3b5b5e

• Improve the diagnostics for 'conv' fallback explain (NVIDIA#11076)

  Show description for 6455396
  

  jihoonson

  and

  gerashegalov

  authored
  6455396

  Copy full SHA for 6455396

Commits on Jun 26, 2024

• Disable ANSI mode for window function tests [databricks] (NVIDIA#11073)

  Show description for 34e6bc8
  mythrocks

  authored
  34e6bc8

  Copy full SHA for 34e6bc8

Commits on Jun 27, 2024

• Fix some test issues in Spark UT and keep RapidsTestSettings update-to-date (NVIDIA#10997)

  Show description for 3cb54c4
  thirtiseven

  authored
  3cb54c4

  Copy full SHA for 3cb54c4

• exclude a case based on JDK version (NVIDIA#11083)

  Show description for 9dafc54
  thirtiseven

  authored
  9dafc54

  Copy full SHA for 9dafc54

Commits on Jun 28, 2024

• Replaced spark3xx-common references to spark-shared [databricks] (NVIDIA#11066)

  Show description for 3b6c5cd
  razajafri

  authored
  3b6c5cd

  Copy full SHA for 3b6c5cd

• Fixed some cast_tests (NVIDIA#11049)

  Show description for 7dc52bc
  razajafri

  authored
  7dc52bc

  Copy full SHA for 7dc52bc

• Fixed array_tests for Spark 4.0.0 [databricks] (NVIDIA#11048)

  Show description for dd62000
  razajafri

  authored
  dd62000

  Copy full SHA for dd62000

Commits on Jun 29, 2024

• Add a heuristic to skip second or third agg pass (NVIDIA#10950)

  Show description for f954026
  binmahone

  authored
  f954026

  Copy full SHA for f954026

• Support regex patterns with brackets when rewriting to PrefixRange pattern in rlike. (NVIDIA#11088)

  Show description for 2498204
  thirtiseven

  authored
  2498204

  Copy full SHA for 2498204

Commits on Jul 1, 2024

• Fix match error in RapidsShuffleIterator.scala [scala2.13] (NVIDIA#11115)

  Show description for f56fe2c
  xieshuaihu

  authored
  f56fe2c

  Copy full SHA for f56fe2c

• Spark 4: Handle ANSI mode in sort_test.py (NVIDIA#11099)

  Show description for 850365c
  mythrocks

  authored
  850365c

  Copy full SHA for 850365c

Commits on Jul 2, 2024

• Introduce LORE framework. (NVIDIA#11084)

  Show description for 9bb295a
  liurenjie1024

  authored
  9bb295a

  Copy full SHA for 9bb295a

• Merge branch 'branch-24.08' into shuffle-gpu-serde

  firestarman

  committed
  b52038e

  Copy full SHA for b52038e

Commits on Jul 4, 2024

• d1

  Show description for b4ea48f
  firestarman

  committed
  b4ea48f

  Copy full SHA for b4ea48f

Commits on Jul 15, 2024

• retry when copying data to host for merged buffers

  Show description for 8c4b318
  firestarman

  committed
  8c4b318

  Copy full SHA for 8c4b318