From 0abf3125395f718ec14aa329535244257c851135 Mon Sep 17 00:00:00 2001 From: foxish Date: Wed, 8 Mar 2017 03:58:58 -0800 Subject: [PATCH] Improved section on running in the cloud --- docs/running-on-kubernetes-cloud.md | 4 +++- docs/running-on-kubernetes.md | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/running-on-kubernetes-cloud.md b/docs/running-on-kubernetes-cloud.md index 53cf0d9390500..a4f94c3bc6357 100644 --- a/docs/running-on-kubernetes-cloud.md +++ b/docs/running-on-kubernetes-cloud.md @@ -10,10 +10,12 @@ A Kubernetes cluster may be brought up on different cloud providers or on premis ## Running on Google Container Engine (GKE) * Create a GKE [container cluster](https://cloud.google.com/container-engine/docs/clusters/operations). -* Find the name of the master associated with this project. +* Obtain kubectl and [configure](https://cloud.google.com/container-engine/docs/clusters/operations#configuring_kubectl) it appropriately. +* Find the identity of the master associated with this project. > kubectl cluster-info Kubernetes master is running at https://x.y.z.w:443 + * Run spark-submit with the master option set to `k8s://https://x.y.z.w:443`. The instructions for running spark-submit are provided in the [running on kubernetes](running-on-kubernetes.md) tutorial. * Check that your driver pod, and subsequently your executor pods are launched using `kubectl get pods`. * Read the stdout and stderr of the driver pod using `kubectl get logs`. diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md index aaf1232a0179f..5e35255db681c 100644 --- a/docs/running-on-kubernetes.md +++ b/docs/running-on-kubernetes.md @@ -137,7 +137,7 @@ or `local:`. A scheme of `file:` corresponds to the keyStore being located on th the driver container as a [secret volume](https://kubernetes.io/docs/user-guide/secrets/). When the URI has the scheme `local:`, the file is assumed to already be on the container's disk at the appropriate path. -### Submission of local files through Ingress/External controller +### Submission of Local Files through Ingress/External controller Kubernetes pods run with their own IP address space. If Spark is run in cluster mode, the driver pod may not be accessible to the submitter. However, the submitter needs to send local dependencies from its local disk to the driver