Skip to content

Commit

Permalink
[SPARK-22905][ML][FOLLOWUP] Fix GaussianMixtureModel save
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?
make sure model data is stored in order.  WeichenXu123

## How was this patch tested?
existing tests

Author: Zheng RuiFeng <ruifengz@foxmail.com>

Closes #20113 from zhengruifeng/gmm_save.
  • Loading branch information
zhengruifeng authored and bersprockets committed Aug 13, 2018
1 parent 81f57fe commit bab8e68
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ object GaussianMixtureModel extends Loader[GaussianMixtureModel] {
val dataArray = Array.tabulate(weights.length) { i =>
Data(weights(i), gaussians(i).mu, gaussians(i).sigma)
}
spark.createDataFrame(dataArray).repartition(1).write.parquet(Loader.dataPath(path))
spark.createDataFrame(sc.makeRDD(dataArray, 1)).write.parquet(Loader.dataPath(path))
}

def load(sc: SparkContext, path: String): GaussianMixtureModel = {
Expand Down

0 comments on commit bab8e68

Please sign in to comment.