Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: report clearer error message when AVG used with DELIMITED #4295

Merged
merged 1 commit into from
Jan 13, 2020

Conversation

big-andy-coates
Copy link
Contributor

Description

See #4294

AVG doesn't work with DELIMITED format and the error message isn't great.

Example statements that cause the error:

-- Given:
CREATE STREAM INPUT (VALUE integer) WITH (kafka_topic='test_topic', value_format='DELIMITED');

-- When:
CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY;

Old error message:

ksql> CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY;
CREATE TABLE OUTPUT AS SELECT avg(value)Value format does not support value schema.
format: DELIMITED
schema: Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR, KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL, unwrapped=false}
reason: The 'DELIMITED' format does not support type 'STRUCT'
Caused by: The 'DELIMITED' format does not support type 'STRUCT'

This PR improves the error message a bit:

New error message:

One of the functions used in the statement has an intermediate type that the value format can not handle. Please remove the function or change the format.
Consider up-voting https://github.com/confluentinc/ksql/issues/3950, which will resolve this limitation
Caused by: Value format does not support value schema.
format: DELIMITED
schema:
	Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR,
	KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL,
	unwrapped=false}
reason: The 'DELIMITED' format does not support type 'STRUCT'
Caused by: The 'DELIMITED' format does not support type 'STRUCT'

Testing done

manual, unit and QTT.

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

See confluentinc#4294

`AVG` doesn't work with `DELIMITED` format and the error message isn't great.

Example statements that cause the error:

```sql
-- Given:
CREATE STREAM INPUT (VALUE integer) WITH (kafka_topic='test_topic', value_format='DELIMITED');

-- When:
CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY;
```

Old error message:

```
ksql> CREATE TABLE OUTPUT AS SELECT avg(value) AS avg FROM INPUT group by ROWKEY;
CREATE TABLE OUTPUT AS SELECT avg(value)Value format does not support value schema.
format: DELIMITED
schema: Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR, KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL, unwrapped=false}
reason: The 'DELIMITED' format does not support type 'STRUCT'
Caused by: The 'DELIMITED' format does not support type 'STRUCT'
```

This PR improves the error message a bit:

New error message:

```
One of the functions used in the statement has an intermediate type that the value format can not handle. Please remove the function or change the format.
Consider up-voting confluentinc#3950, which will resolve this limitation
Caused by: Value format does not support value schema.
format: DELIMITED
schema:
	Persistence{schema=STRUCT<KSQL_INTERNAL_COL_0 INT, KSQL_INTERNAL_COL_1 VARCHAR,
	KSQL_AGG_VARIABLE_0 STRUCT<SUM INT, COUNT BIGINT>> NOT NULL,
	unwrapped=false}
reason: The 'DELIMITED' format does not support type 'STRUCT'
Caused by: The 'DELIMITED' format does not support type 'STRUCT'
```
Copy link
Member

@spena spena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@big-andy-coates big-andy-coates merged commit 307bf4d into confluentinc:master Jan 13, 2020
@big-andy-coates big-andy-coates deleted the avg_delimited branch January 13, 2020 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants