Skip to content
This repository has been archived by the owner on Jan 9, 2020. It is now read-only.

Commit

Permalink
Improved section on running in the cloud
Browse files Browse the repository at this point in the history
  • Loading branch information
foxish committed Mar 8, 2017
1 parent 5c7e787 commit 0abf312
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
4 changes: 3 additions & 1 deletion docs/running-on-kubernetes-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ A Kubernetes cluster may be brought up on different cloud providers or on premis
## Running on Google Container Engine (GKE)

* Create a GKE [container cluster](https://cloud.google.com/container-engine/docs/clusters/operations).
* Find the name of the master associated with this project.
* Obtain kubectl and [configure](https://cloud.google.com/container-engine/docs/clusters/operations#configuring_kubectl) it appropriately.
* Find the identity of the master associated with this project.

> kubectl cluster-info
Kubernetes master is running at https://x.y.z.w:443

* Run spark-submit with the master option set to `k8s://https://x.y.z.w:443`. The instructions for running spark-submit are provided in the [running on kubernetes](running-on-kubernetes.md) tutorial.
* Check that your driver pod, and subsequently your executor pods are launched using `kubectl get pods`.
* Read the stdout and stderr of the driver pod using `kubectl get logs`.
Expand Down
2 changes: 1 addition & 1 deletion docs/running-on-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ or `local:`. A scheme of `file:` corresponds to the keyStore being located on th
the driver container as a [secret volume](https://kubernetes.io/docs/user-guide/secrets/). When the URI has the scheme
`local:`, the file is assumed to already be on the container's disk at the appropriate path.

### Submission of local files through Ingress/External controller
### Submission of Local Files through Ingress/External controller

Kubernetes pods run with their own IP address space. If Spark is run in cluster mode, the driver pod may not be
accessible to the submitter. However, the submitter needs to send local dependencies from its local disk to the driver
Expand Down

0 comments on commit 0abf312

Please sign in to comment.