From cde19354ce68c2e0dd50c01e870789f821848b7d Mon Sep 17 00:00:00 2001 From: foxish Date: Fri, 10 Feb 2017 12:43:19 -0800 Subject: [PATCH] Addressed comments --- docs/running-on-kubernetes.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md index be1755cfdb8d9..19f406039e261 100644 --- a/docs/running-on-kubernetes.md +++ b/docs/running-on-kubernetes.md @@ -153,9 +153,14 @@ the driver container as a [secret volume](https://kubernetes.io/docs/user-guide/ ### Kubernetes Clusters and the authenticated proxy endpoint Spark-submit also supports submission through the -local kubectl proxy](https://kubernetes.io/docs/user-guide/connecting-to-applications-proxy/). One can use the +[local kubectl proxy](https://kubernetes.io/docs/user-guide/connecting-to-applications-proxy/). One can use the authenticating proxy to communicate with the api server directly without passing credentials to spark-submit. -For example, if our local proxy were listening on port 8001, we would have our submission looking like the following: + +The local proxy can be started by running: + + kubectl proxy + +If our local proxy were listening on port 8001, we would have our submission looking like the following: bin/spark-submit \ --deploy-mode cluster \ @@ -168,9 +173,10 @@ For example, if our local proxy were listening on port 8001, we would have our s --conf spark.kubernetes.executor.docker.image=registry-host:5000/spark-executor:latest \ examples/jars/spark_examples_2.11-2.2.0.jar -This mechanism can also be useful when we have authentication schemes that the client library does not support -completely. - +Communication between Spark and Kubernetes clusters is performed using the fabric8 kubernetes-client library. +The above mechanism using `kubectl proxy` can be used when we have authentication providers that the fabric8 +kubernetes-client library does not support. Authentication using X509 Client Certs and oauth tokens +is currently supported. ### Spark Properties