-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-19236][Core] Added createOrReplaceGlobalTempView method #16598
Conversation
Can one of the admins verify this patch? |
* preserved database `_global_temp`, and we must use the qualified name to refer a global temp | ||
* view, e.g. `SELECT * FROM _global_temp.view1`. | ||
* | ||
* @group basic |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should add since version 2.2.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually 2.1.1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also port it to Python? https://github.com/apache/spark/blob/master/python/pyspark/sql/dataframe.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But it seems it's not added in 2.1.1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Global temp view support was added in 2.1. See the PR #14897
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The createOrReplaceGlobalTempView method is not in Scala API
@rxin said it should be added since 2.1.1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JAVA API?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, in the Scala API
I edited the comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This API is added by you in this PR. This is for Scala API. Could you also add the corresponding one to Python API?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I'm not expert in Python
Hi @arman1371, is it still active? |
@HyukjinKwon Yes |
Merging to master. I will submit a separate PR for resolving the comment. Thanks! |
@gatorsmile this didn't run any tests!!! |
@gatorsmile, merging this commit ended up breaking tests because this commit fails Scalastyle checks and the blank line actually had trailing whitespace. I'm pushing a hotfix now to address this, but next time please run tests, even for trivial changes. |
Sorry, I forgot it. |
…method ### What changes were proposed in this pull request? This PR is to backport two PRs for adding the `createOrReplaceGlobalTempView` method #18147 #16598 --- Added the createOrReplaceGlobalTempView method for dataset API ### How was this patch tested? N/A Author: Xiao Li <gatorsmile@gmail.com> Closes #18167 from gatorsmile/Backport18147.
What changes were proposed in this pull request?
Added the createOrReplaceGlobalTempView method for dataset