Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-14637][SQL] object expressions cleanup #12399

Closed
wants to merge 3 commits into from

Conversation

cloud-fan
Copy link
Contributor

What changes were proposed in this pull request?

Simplify and clean up some object expressions:

  1. simplify the logic to handle propagateNull
  2. add propagateNull parameter to Invoke
  3. simplify the unbox logic in Invoke
  4. other minor cleanup

TODO: simplify MapObjects

How was this patch tested?

existing tests.

@cloud-fan
Copy link
Contributor Author

cc @marmbrus @yhuai @liancheng

@SparkQA
Copy link

SparkQA commented Apr 14, 2016

Test build #55825 has finished for PR 12399 at commit 43644ef.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 19, 2016

Test build #56164 has finished for PR 12399 at commit db476f5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

}

val value = unboxer(s"${obj.value}.$functionName($argString)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to do unboxing anymore here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do, but I simplified it, see the callFunc above.

@marmbrus
Copy link
Contributor

Sorry this went stale. Do we still want to try and get this in?

@SparkQA
Copy link

SparkQA commented Apr 30, 2016

Test build #57408 has finished for PR 12399 at commit adc3158.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor Author

retest this please.

@SparkQA
Copy link

SparkQA commented Apr 30, 2016

Test build #57410 has finished for PR 12399 at commit adc3158.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Apr 30, 2016

Test build #57427 has finished for PR 12399 at commit adc3158.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@marmbrus
Copy link
Contributor

marmbrus commented May 2, 2016

Thanks, merging to master and 2.0

asfgit pushed a commit that referenced this pull request May 2, 2016
## What changes were proposed in this pull request?

Simplify and clean up some object expressions:

1. simplify the logic to handle `propagateNull`
2. add `propagateNull` parameter to `Invoke`
3. simplify the unbox logic in `Invoke`
4. other minor cleanup

TODO: simplify `MapObjects`

## How was this patch tested?

existing tests.

Author: Wenchen Fan <wenchen@databricks.com>

Closes #12399 from cloud-fan/object.

(cherry picked from commit 0513c3a)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in 0513c3a May 2, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants