This repository has been archived by the owner on Jan 9, 2020. It is now read-only.
Run executor -> executor comms via IPs rather than FQDNs #108
Labels
Comments
|
I'm not sure what we stand to gain by switching over to IPs. Definitely seems like something we should do in beta. WDYT? |
|
We gain resiliency against flaky DNS services, though arguable if we're seeing flaky DNS we should diagnose that part of k8s separately. |
|
Maybe we can close this issue as well? |
|
Fixed in #215 |
ifilonenko
pushed a commit
to ifilonenko/spark
that referenced
this issue
Feb 25, 2019
…-kubernetes Put kubernetes profiles in the published artifacts.
ifilonenko
pushed a commit
to bloomberg/apache-spark-on-k8s
that referenced
this issue
Oct 21, 2019
…n SQLQueryTestSuite
### What changes were proposed in this pull request?
This PR fix Fix the associated location already exists in `SQLQueryTestSuite`:
```
build/sbt "~sql/test-only *SQLQueryTestSuite -- -z postgreSQL/join.sql"
...
[info] - postgreSQL/join.sql *** FAILED *** (35 seconds, 420 milliseconds)
[info] postgreSQL/join.sql
[info] Expected "[]", but got "[org.apache.spark.sql.AnalysisException
[info] Can not create the managed table('`default`.`tt3`'). The associated location('file:/root/spark/sql/core/spark-warehouse/org.apache.spark.sql.SQLQueryTestSuite/tt3') already exists.;]" Result did not match for query apache-spark-on-k8s#108
```
### Why are the changes needed?
Fix bug.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
N/A
Closes apache#26181 from wangyum/TestError.
Authored-by: Yuming Wang <yumwang@ebay.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
See thread at #102 (comment) -- it's probably more reliable in k8s to run pod to pod communication via IP instead of via hostname. Need to make sure we don't lose much descriptiveness in Spark UIs and elsewhere by going to IPs from Hostnames though (hostnames are more familiar and memorable for debugging).
The text was updated successfully, but these errors were encountered: