[SPARK-33518][ML][FOLLOWUP] MatrixFactorizationModel use GEMV #31279
Conversation
|
Kubernetes integration test starting |
|
Test build #134324 has finished for PR 31279 at commit
|
|
Kubernetes integration test status failure |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
|
ping @srowen |
| val n = dstIds.length | ||
| if (scores == null || scores.length < n) { | ||
| scores = Array.ofDim[Double](n) | ||
| idxOrd = new GuavaOrdering[Int] { |
There was a problem hiding this comment.
I suppose this can be defined outside the flatMap, but it won't matter much.
There was a problem hiding this comment.
This looks like the one in ml.ALS, however, in ml.ALS, vector are represented as array[Float], here use array[Double].
There was a problem hiding this comment.
OK, I guess same goes for the other.. idxOrd doesn't change right? No big deal.
|
merged to master, thanks @srowen |
### What changes were proposed in this pull request? 1, update related doc; 2, MatrixFactorizationModel use GEMV; ### Why are the changes needed? see performance gain in apache#30468 ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? existing testsuites Closes apache#31279 from zhengruifeng/als_follow_up. Authored-by: Ruifeng Zheng <ruifengz@foxmail.com> Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
What changes were proposed in this pull request?
1, update related doc;
2, MatrixFactorizationModel use GEMV;
Why are the changes needed?
see performance gain in #30468
Does this PR introduce any user-facing change?
NO
How was this patch tested?
existing testsuites