[#1358] fix(spark): pre-check bytebuffer whether is direct before uncompress

[zuston]

What changes were proposed in this pull request?

precheck bytebuffer whether is direct before uncompress, and rebuild the temp bytebuffer.

Why are the changes needed?

For #1358 . But this PR is not the fundamental solution

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

[zuston]

PTAL @leixm

[leixm]

Can you explain why the original code fails to make the ByteBuffer consistent?

[zuston]

Can you explain why the original code fails to make the ByteBuffer consistent?

I haven't found the root cause, but I think the bug is related with the nioBuffer() .

Anyway, this PR is to use workaround to fix this. But we still need to dig this more.

[codecov-commenter]

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (453e6d6) 53.27% compared to head (1f8d253) 54.11%.
Report is 7 commits behind head on master.

Files	Patch %	Lines
...e/spark/shuffle/reader/RssShuffleDataIterator.java	11.11%	4 Missing and 4 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #1360      +/-   ##
============================================
+ Coverage     53.27%   54.11%   +0.84%     
- Complexity     2712     2715       +3     
============================================
  Files           418      398      -20     
  Lines         23913    21568    -2345     
  Branches       2043     2045       +2     
============================================
- Hits          12739    11672    -1067     
+ Misses        10388     9180    -1208     
+ Partials        786      716      -70     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jerqi]

Maybe index BytBbuffer and data ByteBuffer have different types.

[zuston]

Maybe index BytBbuffer and data ByteBuffer have different types.

I think no. The possible point has been attached in the issue. nioBuffer() may be the root cause. .

And I hope this PR could be merged into the master to prevent the ZSTD error ASSP. And also this should be merged into the 0.8.1 version.

[jerqi]

cc @LuciferYang Could you give us some advice?

[zuston]

PTAL again @jerqi @LuciferYang

[LuciferYang]

+1 from myside

[zuston]

Thanks for your review @jerqi @LuciferYang , Merged.