[SPARK-2843][MLLIB] add a section about regularization parameter in ALS #2064
Conversation
|
I think this particular technique is called "ALS-WR", at least, that's how I know of it from this paper: http://www.hpl.hp.com/personal/Robert_Schreiber/papers/2008%20AAIM%20Netflix/netflix_aaim08(submitted).pdf You could add a mention of this. |
|
@srowen Thanks for providing the reference! I added it to the doc. |
|
QA tests have started for PR 2064 at commit
|
|
QA tests have finished for PR 2064 at commit
|
| Since v1.1, we scale the regularization parameter `lambda` in solving each least squares problem by | ||
| the number of ratings the user generated in updating user factors, | ||
| or the number of ratings the product received in updating product factors. | ||
| This approach is named "ALS-WR" and introduced in the paper |
There was a problem hiding this comment.
This technique was used by many groups in the Netflix Prize (see section 5 of this paper for some more references: http://www.cs.toronto.edu/~rsalakhu/papers/weighted_tc.pdf). Perhaps this sentence should be changed to "This approach is discussed in the following paper..."
|
I merged this into master and branch-1.1. Thanks @srowen and @atalwalkar for reviewing! |
atalwalkar srowen Author: Xiangrui Meng <meng@databricks.com> Closes #2064 from mengxr/als-doc and squashes the following commits: b2e20ab [Xiangrui Meng] introduced -> discussed 98abdd7 [Xiangrui Meng] add reference 339bd08 [Xiangrui Meng] add a section about regularization parameter in ALS (cherry picked from commit e0f9462) Signed-off-by: Xiangrui Meng <meng@databricks.com>
atalwalkar srowen Author: Xiangrui Meng <meng@databricks.com> Closes apache#2064 from mengxr/als-doc and squashes the following commits: b2e20ab [Xiangrui Meng] introduced -> discussed 98abdd7 [Xiangrui Meng] add reference 339bd08 [Xiangrui Meng] add a section about regularization parameter in ALS
@atalwalkar @srowen