Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Substrait consumer doesn't respect final output column names #10817

Closed
Tracked by #5173
Blizzara opened this issue Jun 7, 2024 · 1 comment · Fixed by #10829
Closed
Tracked by #5173

Substrait consumer doesn't respect final output column names #10817

Blizzara opened this issue Jun 7, 2024 · 1 comment · Fixed by #10829
Labels
bug Something isn't working

Comments

@Blizzara
Copy link
Contributor

Blizzara commented Jun 7, 2024

Describe the bug

Substrait doesn't support the concept of ALIASes, as inside the plan there are no column names, just indices (https://substrait.io/expressions/field_references/#field-references). Ensuring the final plan has the right column names is done through the "names" field on the RootRel.

DataFusion supports already producing that "names" field (

names: plan.schema().field_names(),
), though the produced list lacks the inner names of struct fields (the "names" should be a flattened list of a depth-first search through all the fields). However the consumer side completely disregards the given names (
Ok(from_substrait_rel(ctx, root.input.as_ref().unwrap(), &function_extension).await?)
).

To Reproduce

#[tokio::test]
async fn select_with_alias() -> Result<()> {
    roundtrip("SELECT a AS aliased_a FROM data").await
}

fails with

assertion `left == right` failed
  left: "Projection: data.a AS aliased_a\n  TableScan: data projection=[a]"
 right: "TableScan: data projection=[a]"

Expected behavior

Should succeed producing the correct column names

Additional context

I'll write up a PR to fix

@alamb
Copy link
Contributor

alamb commented Jun 9, 2024

I added to the substrait support epic: #5173

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants