This repository has been archived by the owner on Jan 9, 2020. It is now read-only.
Adding clarification pre-alpha #181
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mccheah
reviewed
Mar 8, 2017
docs/running-on-kubernetes.md
Outdated
| @@ -10,7 +10,7 @@ currently limited and not well-tested. This should not be used in production env | |||
|
|
|||
| * You must have a running Kubernetes cluster with access configured to it using [kubectl](https://kubernetes.io/docs/user-guide/prereqs/). If you do not already have a working Kubernetes cluster, you may setup a test cluster on your local machine using [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/). | |||
| * You must have appropriate permissions to create and list [pods](https://kubernetes.io/docs/user-guide/pods/), [nodes](https://kubernetes.io/docs/admin/node/) and [services](https://kubernetes.io/docs/user-guide/services/) in your cluster. You can verify that you can list these resources by running `kubectl get nodes`, `kubectl get pods` and `kubectl get svc` which should give you a list of nodes, pods and services (if any) respectively. | |||
| * You must have an extracted spark distribution with Kubernetes support, or build one from [source](https://github.com/apache-spark-on-k8s/spark). | |||
| * You must [build Spark with Kubernetes support](https://github.com/apache-spark-on-k8s/spark/blob/k8s-support-alternate-incremental/resource-managers/kubernetes/README.md#building-spark-with-kubernetes-support) from source. | |||
There was a problem hiding this comment.
This is a temporary step until we've placed a tarball somewhere for download?
There was a problem hiding this comment.
Yes, for people who may hit the link today, because of the GCP announcement.
|
+1, merging immediately because docs shouldn't affect the stability of the build. |
ash211
pushed a commit
that referenced
this pull request
Mar 16, 2017
(cherry picked from commit be109ab)
foxish
added a commit
that referenced
this pull request
Jul 24, 2017
(cherry picked from commit be109ab)
ifilonenko
pushed a commit
to ifilonenko/spark
that referenced
this pull request
Feb 26, 2019
Don't log the full compiled code unless debug mode is on, to prevent ooming and crashing the spark driver in certain scenarios
puneetloya
pushed a commit
to puneetloya/spark
that referenced
this pull request
Mar 11, 2019
(cherry picked from commit be109ab)
ifilonenko
pushed a commit
to ifilonenko/spark
that referenced
this pull request
Oct 11, 2019
…comparison assertions ## What changes were proposed in this pull request? This PR removes a few hardware-dependent assertions which can cause a failure in `aarch64`. **x86_64** ``` rootdonotdel-openlab-allinone-l00242678:/home/ubuntu# uname -a Linux donotdel-openlab-allinone-l00242678 4.4.0-154-generic apache-spark-on-k8s#181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux scala> import java.lang.Float.floatToRawIntBits import java.lang.Float.floatToRawIntBits scala> floatToRawIntBits(0.0f/0.0f) res0: Int = -4194304 scala> floatToRawIntBits(Float.NaN) res1: Int = 2143289344 ``` **aarch64** ``` [rootarm-huangtianhua spark]# uname -a Linux arm-huangtianhua 4.14.0-49.el7a.aarch64 #1 SMP Tue Apr 10 17:22:26 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux scala> import java.lang.Float.floatToRawIntBits import java.lang.Float.floatToRawIntBits scala> floatToRawIntBits(0.0f/0.0f) res1: Int = 2143289344 scala> floatToRawIntBits(Float.NaN) res2: Int = 2143289344 ``` ## How was this patch tested? Pass the Jenkins (This removes the test coverage). Closes apache#25186 from huangtianhua/special-test-case-for-aarch64. Authored-by: huangtianhua <huangtianhua@huawei.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Just making the process clearer
cc @mccheah