Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error message exposes internal column name #3650

Closed
vpapavas opened this issue Oct 22, 2019 · 2 comments
Closed

Error message exposes internal column name #3650

vpapavas opened this issue Oct 22, 2019 · 2 comments
Assignees
Milestone

Comments

@vpapavas
Copy link
Member

vpapavas commented Oct 22, 2019

Describe the bug
Error message displayed is:
Invalid key field, not found in schema: KSQL_INTERNAL_COL_1

To Reproduce
Steps to reproduce the behavior, include:

  1. The version of KSQL.
    Master
  2. Sample source data.

Assume stream with schema
CREATE STREAM metrics_stream (metrics_name STRING, metrics_value DOUBLE, dimensions STRUCT<cluster STRING, topic_name STRING, env STRING>, ts BIGINT) WITH (KAFKA_TOPIC = 'metrics', VALUE_FORMAT = 'JSON', timestamp='ts');

  1. Any SQL statements you ran
    create table test as select dimensions->cluster as cluster , collect_set(dimensions->cluster) as collected from metrics_stream_2 group by dimensions->cluster;

Expected behavior
The query should not fail, but even if it were supposed to fail, the error message does not help identifying what is wrong. Moreover, it should not expose the internal column name KSQL_INTERNAL_COL_1.

Actual behaviour
A clear and concise description of what actually happens, including:

  1. CLI output
  2. Error messages
    Invalid key field, not found in schema: KSQL_INTERNAL_COL_1
  3. KSQL logs

Additional context
This query does not fail:
create table test as select metrics_name as cluster , collect_set(metrics_name) from metrics_stream_2 group by metrics_name;
The problem seems to be in the STRUCT column.

@big-andy-coates
Copy link
Contributor

big-andy-coates commented Oct 25, 2019

This shouldn't be an issue once ##3537 and then #3669 is done, as 'KEY fields' won't be a thing any more.

@derekjn derekjn added this to the 0.11.0 milestone Jun 12, 2020
@big-andy-coates big-andy-coates self-assigned this Jun 23, 2020
@big-andy-coates
Copy link
Contributor

No longer reproducible in master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants