[SPARK-20668][SQL] Modify ScalaUDF to handle nullability. #17911
Conversation
|
Test build #76632 has started for PR 17911 at commit |
|
cc @gatorsmile |
|
retest this please |
|
This pr looks good to me though, do we have any plan to update the interface of |
|
Test build #76647 has finished for PR 17911 at commit
|
|
Jenkins, retest this please. |
|
Test build #76677 has finished for PR 17911 at commit
|
| udf | ||
| } | ||
|
|
||
| protected[sql] def withNullability(nullable: Boolean): UserDefinedFunction = { |
There was a problem hiding this comment.
This can be a public function?
Also add a function description?
| @@ -47,12 +47,37 @@ case class UserDefinedFunction protected[sql] ( | |||
| dataType: DataType, | |||
| inputTypes: Option[Seq[DataType]]) { | |||
|
|
|||
| private var _nullable: Boolean = true | |||
|
|
|||
| def nullable: Boolean = _nullable | |||
There was a problem hiding this comment.
Since this is public, could you add a function description with @since?
|
LGTM pending test |
|
Test build #76717 has finished for PR 17911 at commit
|
|
Thanks! Merging to master. |
## What changes were proposed in this pull request? When registering Scala UDF, we can know if the udf will return nullable value or not. `ScalaUDF` and related classes should handle the nullability. ## How was this patch tested? Existing tests. Author: Takuya UESHIN <ueshin@databricks.com> Closes apache#17911 from ueshin/issues/SPARK-20668.
What changes were proposed in this pull request?
When registering Scala UDF, we can know if the udf will return nullable value or not.
ScalaUDFand related classes should handle the nullability.How was this patch tested?
Existing tests.