From 206bd63725cb5d466413ed47ddba247363a766cd Mon Sep 17 00:00:00 2001 From: Nicholas Nezis Date: Wed, 6 Jan 2021 01:41:48 -0500 Subject: [PATCH 1/2] Updated to Kryo 5.0.3 --- dev/deps/spark-deps-hadoop-2.7-hive-2.3 | 2 +- dev/deps/spark-deps-hadoop-3.2-hive-2.3 | 2 +- docs/tuning.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/dev/deps/spark-deps-hadoop-2.7-hive-2.3 b/dev/deps/spark-deps-hadoop-2.7-hive-2.3 index 9c516203dd3fa..4bfe3edb73329 100644 --- a/dev/deps/spark-deps-hadoop-2.7-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-2.7-hive-2.3 @@ -155,7 +155,7 @@ jsp-api/2.1//jsp-api-2.1.jar jsr305/3.0.0//jsr305-3.0.0.jar jta/1.1//jta-1.1.jar jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar -kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar +kryo-shaded/5.0.3//kryo-shaded-5.0.3.jar kubernetes-client/4.12.0//kubernetes-client-4.12.0.jar kubernetes-model-admissionregistration/4.12.0//kubernetes-model-admissionregistration-4.12.0.jar kubernetes-model-apiextensions/4.12.0//kubernetes-model-apiextensions-4.12.0.jar diff --git a/dev/deps/spark-deps-hadoop-3.2-hive-2.3 b/dev/deps/spark-deps-hadoop-3.2-hive-2.3 index 1d80fadb5762a..b63ebde115b3e 100644 --- a/dev/deps/spark-deps-hadoop-3.2-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-3.2-hive-2.3 @@ -168,7 +168,7 @@ kerby-config/1.0.1//kerby-config-1.0.1.jar kerby-pkix/1.0.1//kerby-pkix-1.0.1.jar kerby-util/1.0.1//kerby-util-1.0.1.jar kerby-xdr/1.0.1//kerby-xdr-1.0.1.jar -kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar +kryo-shaded/5.0.3//kryo-shaded-5.0.3.jar kubernetes-client/4.12.0//kubernetes-client-4.12.0.jar kubernetes-model-admissionregistration/4.12.0//kubernetes-model-admissionregistration-4.12.0.jar kubernetes-model-apiextensions/4.12.0//kubernetes-model-apiextensions-4.12.0.jar diff --git a/docs/tuning.md b/docs/tuning.md index 18d4a6205f4ff..632497a64da32 100644 --- a/docs/tuning.md +++ b/docs/tuning.md @@ -50,7 +50,7 @@ in your operations) and performance. It provides two serialization libraries: Java serialization is flexible but often quite slow, and leads to large serialized formats for many classes. * [Kryo serialization](https://github.com/EsotericSoftware/kryo): Spark can also use - the Kryo library (version 4) to serialize objects more quickly. Kryo is significantly + the Kryo library (version 5) to serialize objects more quickly. Kryo is significantly faster and more compact than Java serialization (often as much as 10x), but does not support all `Serializable` types and requires you to *register* the classes you'll use in the program in advance for best performance. From 0d962736afee53e934b98ca25059d0a19ada0198 Mon Sep 17 00:00:00 2001 From: Nicholas Nezis Date: Wed, 6 Jan 2021 19:40:52 +0000 Subject: [PATCH 2/2] Removing the Kryo version from the tuning document --- docs/tuning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tuning.md b/docs/tuning.md index 632497a64da32..e91bb30499f22 100644 --- a/docs/tuning.md +++ b/docs/tuning.md @@ -50,7 +50,7 @@ in your operations) and performance. It provides two serialization libraries: Java serialization is flexible but often quite slow, and leads to large serialized formats for many classes. * [Kryo serialization](https://github.com/EsotericSoftware/kryo): Spark can also use - the Kryo library (version 5) to serialize objects more quickly. Kryo is significantly + the Kryo library to serialize objects more quickly. Kryo is significantly faster and more compact than Java serialization (often as much as 10x), but does not support all `Serializable` types and requires you to *register* the classes you'll use in the program in advance for best performance.