KeyQueryMetadata not available for state store Aggregate-Aggregate-Materialize and key Struct{X=X}

[rmoff]

ksqlDB v0.11

This error happens if you run a pull query against a table too quickly after creation.

If you paste the following into the CLI in one go and hit enter, you see the error

➜ docker exec -it ksqldb ksql http://ksqldb:8088

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

                  ===========================================
                  =       _              _ ____  ____       =
                  =      | | _____  __ _| |  _ \| __ )      =
                  =      | |/ / __|/ _` | | | | |  _ \      =
                  =      |   <\__ \ (_| | | |_| | |_) |     =
                  =      |_|\_\___/\__, |_|____/|____/      =
                  =                   |_|                   =
                  =  Event Streaming Database purpose-built =
                  =        for stream processing apps       =
                  ===========================================

Copyright 2017-2020 Confluent Inc.

CLI v0.11.0, Server v0.11.0 located at http://ksqldb:8088

Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!

ksql>
ksql> CREATE STREAM pageviews (msg VARCHAR)
>  WITH (KAFKA_TOPIC ='pageviews',
>        VALUE_FORMAT='JSON');
>
>SET 'auto.offset.reset' = 'earliest';
>
>CREATE TABLE MSG_COUNT AS
>    SELECT 'X' AS X,
>        COUNT(*) AS MSG_CT
>    FROM PAGEVIEWS
>    GROUP BY 'X'
>    EMIT CHANGES;
>
>SELECT * FROM MSG_COUNT WHERE X='X';
>

 Message
----------------
 Stream created
----------------
Successfully changed local property 'auto.offset.reset' to 'earliest'. Use the UNSET command to revert your change.

 Message
----------------------------------------
 Created query with ID CTAS_MSG_COUNT_0
----------------------------------------
KeyQueryMetadata not available for state store Aggregate-Aggregate-Materialize and key Struct{X=X}

If you then run the SELECT again after a few moments, it works

ksql> SELECT * FROM MSG_COUNT WHERE X='X';
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+
|X                                                                            |MSG_CT                                                                       |
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+
|X                                                                            |42                                                                           |
Query terminated
ksql>

Can we put a better error around this?

[rmoff]

One option to work around this for automated processes is to force it to wait with a push query and a LIMIT:

CLI v0.11.0, Server v0.11.0 located at http://ksqldb:8088

Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!

ksql>
ksql> CREATE STREAM pageviews (msg VARCHAR)
>  WITH (KAFKA_TOPIC ='pageviews',
>        VALUE_FORMAT='JSON');
>
>SET 'auto.offset.reset' = 'earliest';
>
>CREATE TABLE MSG_COUNT AS
>    SELECT 'X' AS X,
>        COUNT(*) AS MSG_CT
>    FROM PAGEVIEWS
>    GROUP BY 'X'
>    EMIT CHANGES;
>SELECT * FROM MSG_COUNT WHERE X='X' EMIT CHANGES LIMIT 1;
>SELECT * FROM MSG_COUNT WHERE X='X';
>
>
>

 Message
----------------
 Stream created
----------------
Successfully changed local property 'auto.offset.reset' to 'earliest'. Use the UNSET command to revert your change.

 Message
----------------------------------------
 Created query with ID CTAS_MSG_COUNT_0
----------------------------------------
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+
|X                                                                            |MSG_CT                                                                       |
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+
|X                                                                            |42                                                                           |
Limit Reached
Query terminated
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+
|X                                                                            |MSG_CT                                                                       |
+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------+
|X                                                                            |42                                                                           |
Query terminated
ksql>