AVG UDAF doesn't work with DELIMITED format #4294
Labels
Comments
big-andy-coates
added a commit
to big-andy-coates/ksql
that referenced
this issue
Jan 13, 2020
See confluentinc#4294 `AVG` doesn't work with `DELIMITED` format and the error message isn't great. Example statements that cause the error: ```sql -- Given: CREATE STREAM INPUT (VALUE integer) WITH (kafka_topic='test_topic', value_format='DELIMITED'); -- When: CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY; ``` Old error message: ``` ksql> CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY; CREATE TABLE OUTPUT AS SELECT avg(value)Value format does not support value schema. format: DELIMITED schema: Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR, KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL, unwrapped=false} reason: The 'DELIMITED' format does not support type 'STRUCT' Caused by: The 'DELIMITED' format does not support type 'STRUCT' ``` This PR improves the error message a bit: New error message: ``` One of the functions used in the statement has an intermediate type that the value format can not handle. Please remove the function or change the format. Consider up-voting confluentinc#3950, which will resolve this limitation Caused by: Value format does not support value schema. format: DELIMITED schema: Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR, KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL, unwrapped=false} reason: The 'DELIMITED' format does not support type 'STRUCT' Caused by: The 'DELIMITED' format does not support type 'STRUCT' ```
2 tasks
|
#4295 improves the error message. |
|
Closing issue as a duplicate of #3950 |
big-andy-coates
added a commit
that referenced
this issue
Jan 13, 2020
See #4294 `AVG` doesn't work with `DELIMITED` format and the error message isn't great. Example statements that cause the error: ```sql -- Given: CREATE STREAM INPUT (VALUE integer) WITH (kafka_topic='test_topic', value_format='DELIMITED'); -- When: CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY; ``` Old error message: ``` ksql> CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY; CREATE TABLE OUTPUT AS SELECT avg(value)Value format does not support value schema. format: DELIMITED schema: Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR, KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL, unwrapped=false} reason: The 'DELIMITED' format does not support type 'STRUCT' Caused by: The 'DELIMITED' format does not support type 'STRUCT' ``` This PR improves the error message a bit: New error message: ``` One of the functions used in the statement has an intermediate type that the value format can not handle. Please remove the function or change the format. Consider up-voting #3950, which will resolve this limitation Caused by: Value format does not support value schema. format: DELIMITED schema: Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR, KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL, unwrapped=false} reason: The 'DELIMITED' format does not support type 'STRUCT' Caused by: The 'DELIMITED' format does not support type 'STRUCT' ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The
DELIMITEDformat does not supportSTRUCTs. But theAVGuses aSTRUCTin its intermediate format. HenceAVGdoesn't work withDELIMITED.Error message is:
Which is likely to cause users confusion.
We should:
DELIMITEDas an internal format. There is already a issue that covers this: Design format for internal topics/stores for extensibility #3950The text was updated successfully, but these errors were encountered: