-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-6967] [SQL] fix date type convertion in jdbcrdd #5590
Conversation
Test build #30581 has finished for PR 5590 at commit
|
Could we add a unit test in |
Test build #30727 has finished for PR 5590 at commit
|
Merging in master and branch-1.3. |
This pr convert java.sql.Date type into Int for JDBCRDD. Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes #5590 from adrian-wang/datebug and squashes the following commits: f897b81 [Daoyuan Wang] add a test case 3c9184c [Daoyuan Wang] fix date type convertion in jdbcrdd (cherry picked from commit 04525c0) Signed-off-by: Reynold Xin <rxin@databricks.com> Conflicts: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JDBCRDD.scala
This pr convert java.sql.Date type into Int for JDBCRDD. Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes apache#5590 from adrian-wang/datebug and squashes the following commits: f897b81 [Daoyuan Wang] add a test case 3c9184c [Daoyuan Wang] fix date type convertion in jdbcrdd (cherry picked from commit 04525c0) Signed-off-by: Reynold Xin <rxin@databricks.com> Conflicts: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JDBCRDD.scala
hi @adrian-wang ! I am using spark and needed to include this fix, since I am reading from a table that has Date types. I rebuilt spark with this change & it works well. I just ran into a new problem though: when the Date field has null values, spark throws a java.lang.NullPointerException Thank you for your help! |
@nadavoosh Thanks for your feedback! I will take a look at this soon. |
@adrian-wang this worked for me:
|
This pr convert java.sql.Date type into Int for JDBCRDD. Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes apache#5590 from adrian-wang/datebug and squashes the following commits: f897b81 [Daoyuan Wang] add a test case 3c9184c [Daoyuan Wang] fix date type convertion in jdbcrdd
Thank nadavoosh point this out in apache#5590 Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes apache#5877 from adrian-wang/jdbcrdd and squashes the following commits: cc11900 [Daoyuan Wang] avoid NPE in jdbcrdd
Thank nadavoosh point this out in apache#5590 Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes apache#5877 from adrian-wang/jdbcrdd and squashes the following commits: cc11900 [Daoyuan Wang] avoid NPE in jdbcrdd
This pr convert java.sql.Date type into Int for JDBCRDD. Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes apache#5590 from adrian-wang/datebug and squashes the following commits: f897b81 [Daoyuan Wang] add a test case 3c9184c [Daoyuan Wang] fix date type convertion in jdbcrdd
Thank nadavoosh point this out in apache#5590 Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes apache#5877 from adrian-wang/jdbcrdd and squashes the following commits: cc11900 [Daoyuan Wang] avoid NPE in jdbcrdd
This pr convert java.sql.Date type into Int for JDBCRDD.