-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-14460] [SQL] properly handling of column name contains space #12252
Conversation
Test build #55312 has finished for PR 12252 at commit
|
Test build #55309 has finished for PR 12252 at commit
|
Test build #55330 has finished for PR 12252 at commit
|
Test build #55328 has finished for PR 12252 at commit
|
@andrewor14 Could you please review my codes? |
Test build #55851 has finished for PR 12252 at commit
|
Test build #55854 has finished for PR 12252 at commit
|
* Compute the schema string for this RDD. | ||
*/ | ||
def schemaString(df: DataFrame, url: String): String = { | ||
def schemaString(dialect: JdbcDialect, df: DataFrame, url: String): String = { | ||
val sb = new StringBuilder() | ||
val dialect = JdbcDialects.get(url) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the new dialect you're passing in different from this one in some way?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The purpose to pass in dialect is to get proper quote for columns based on its data source. Any suggestion?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You pass in a parameter named dialect to the schemaString method, but there's also the dialect that comes from JdbcDialects.get(url) --- that's the duplicate I was trying to point out
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing out. I've modified the codes. Please check it out.
Test build #55889 has finished for PR 12252 at commit
|
Test build #55891 has finished for PR 12252 at commit
|
Test build #56384 has finished for PR 12252 at commit
|
seems it's fixed in #15662 ? |
ping @bomeng for ^. |
What changes were proposed in this pull request?
Although it is not recommended, table can be created with column name containing space. For example,
When we query the table via JDBC, we also need to quote the column name in the same way if the column name contains space.
I am leveraging the quote method in the dialect, it will add the proper quote to the column name based on its data source.
How was this patch tested?
I've updated the test case to test this scenario.