Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-8668][VL] Support complex type in ColumnarPartialProject #8669

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

[GLUTEN-8668][VL] Support complex type in ColumnarPartialProject #8669

wants to merge 7 commits into from

Conversation

WangGuangxin
Copy link
Contributor

What changes were proposed in this pull request?

Make ColumnarPartialProject supports complex type

(Fixes: #8668)

How was this patch tested?

add ut and mannually test

@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Feb 6, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

#8668

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

@WangGuangxin
Copy link
Contributor Author

@jinchengchenghh @PHILO-HE @zhztheplayer Can you please help review this?

@zhouyuan
Copy link
Contributor

zhouyuan commented Feb 6, 2025

@WangGuangxin looks like the failure is on GHA upload/restore action, I have made a quick patch on this:
#8672

@WangGuangxin WangGuangxin force-pushed the partial_project_supports_complex_type branch from 5143851 to 66cef26 Compare February 6, 2025 05:11
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

@WangGuangxin WangGuangxin force-pushed the partial_project_supports_complex_type branch from 66cef26 to c5a99f5 Compare February 6, 2025 07:27
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

2 similar comments
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

Run Gluten Clickhouse CI on x86

Yohahaha
Yohahaha reviewed Feb 7, 2025
View reviewed changes
gluten-ut/spark35/src/test/scala/org/apache/spark/sql/hive/execution/GlutenHiveUDFSuite.scala Outdated
}

test("udf with map") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add or move complex type tests in UDFPartialProjectSuite?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Yohahaha done

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 8, 2025

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 8, 2025

Run Gluten Clickhouse CI on x86

PHILO-HE
PHILO-HE reviewed Feb 10, 2025
View reviewed changes
Copy link
Contributor

@PHILO-HE PHILO-HE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WangGuangxin, the pr basically looks good. One comment related to the test. Thanks!

gluten-ut/spark35/src/test/scala/org/apache/spark/sql/hive/execution/GlutenHiveUDFSuite.scala Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

@WangGuangxin WangGuangxin force-pushed the partial_project_supports_complex_type branch from 59365a3 to 24690b4 Compare February 12, 2025 14:42
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

@WangGuangxin WangGuangxin force-pushed the partial_project_supports_complex_type branch from 5593f4f to be79926 Compare February 13, 2025 13:53
@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI on x86

@WangGuangxin
Copy link
Contributor Author

@PHILO-HE @Yohahaha @jinchengchenghh comments addressed. please review again

@PHILO-HE PHILO-HE changed the title [GLUTEN-8668][VL] ColumnarPartialProject supports complex type [GLUTEN-8668][VL] Support complex type in ColumnarPartialProject Feb 14, 2025
PHILO-HE
PHILO-HE approved these changes Feb 14, 2025
View reviewed changes
Copy link
Contributor

@PHILO-HE PHILO-HE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Just one comment.

backends-velox/src/test/java/org/apache/gluten/test/udf/CustomerUDF.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.gluten.execution;
package org.apache.gluten.test.udf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove "test" if there is no reason to keep it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yohahaha Yohahaha Yohahaha left review comments

@PHILO-HE PHILO-HE PHILO-HE approved these changes

Assignees
No one assigned
Labels
CORE works for Gluten Core VELOX
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[VL] ColumnarPartialProject supports complex type
4 participants
@WangGuangxin @zhouyuan @PHILO-HE @Yohahaha