diff --git a/docs-md/concepts/index.md b/docs-md/concepts/index.md index f8dac584bd51..1b6c1a86be02 100644 --- a/docs-md/concepts/index.md +++ b/docs-md/concepts/index.md @@ -6,13 +6,45 @@ description: Learn about ksqlDB under the hood. keywords: ksqldb, architecture, collection, query, schema, window, view --- +ksqlDB is built on these conceptual pillars: + + +- [Collections: streams and tables](#collections-streams-and-tables) +- [Events: facts with timestamps](#events-facts-with-timestamps) +- [Queries: ask questions about materialized views](#queries-ask-questions-about-materialized-views) +- [Materialized Views: enable efficient queries](#materialized-views-enable-efficient-queries) +- [Functions: enhance queries with built-in and custom logic](#functions-enhance-queries-with-built-in-and-custom-logic) +- [Connectors: get data in and send data out](#connectors-get-data-in-and-send-data-out) +- [Schemas: define the structure of your event data](#schemas-define-the-structure-of-your-event-data) + + +More in the **Concepts** section: + - [Architecture](ksql-architecture.md) -- [Collections](./collections/overview.md) - [Kafka Streams and kslDB](ksql-and-kafka-streams.md) -- [Materialized Views](materialized-views.md) - [Processing Guarantees](ksqldb-processing-guarantees.md) -- [Push and Pull Queries](push-and-pull-queries.md) -- [Schemas](ksqldb-schemas.md) - [Time and Windows](time-and-windows-in-ksql-queries.md) +Collections: streams and tables +------------------------------- + +Events: facts with timestamps +----------------------------- + +Queries: ask questions about materialized views +----------------------------------------------- + +Materialized Views: enable efficient queries +-------------------------------------------- + +Functions: enhance queries with built-in and custom logic +--------------------------------------------------------- + +Connectors: get data in and send data out +----------------------------------------- + +Schemas: define the structure of your event data +------------------------------------------------ + + Page last revised on: {{ git_revision_date }} diff --git a/docs-md/concepts/push-and-pull-queries.md b/docs-md/concepts/push-and-pull-queries.md deleted file mode 100644 index 8770f4d7c317..000000000000 --- a/docs-md/concepts/push-and-pull-queries.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -layout: page -title: Push and Pull Queries -tagline: Query materialized views -description: Learn how to query materialized views by using the SELECT statement -keywords: ksqldb, query, select ---- - -Push and Pull Queries -===================== - -TODO: expand on this - -API Reference -------------- - -- [SELECT (Push Query)](../developer-guide/ksqldb-reference/select-push-query.md) -- [SELECT (Pull Query)](../developer-guide/ksqldb-reference/select-pull-query.md) - -Pull Query ----------- - -Pulls the current value from the materialized table and terminates. The result -of this statement isn't persisted in a Kafka topic and is printed out only in -the console. - -Pull queries enable you to fetch the current state of a materialized view. -Because materialized views are incrementally updated as new events arrive, -pull queries run with predictably low latency. They're a great match for -request/response flows. For asynchronous application flows, see -[Push Query](#push-query). - -Execute a pull query by sending an HTTP request to the ksqlDB REST API, and -the API responds with a single response. - -Push Query ----------- - -Push a continuous stream of updates to the ksqlDB stream or table. The result of -this statement isn't persisted in a Kafka topic and is printed out only in -the console. To stop the continuous query in the CLI press Ctrl+C. -Note that the WINDOW clause can only be used if the `from_item` is a -stream. - -Push queries enable you to query a materialized view with a subscription to -the results. Push queries emit refinements to materialized views, which enable -reacting to new information in real-time. They’re a good fit for asynchronous -application flows. For request/response flows, see -[Pull Query](#pull-query). - -Execute a push query by sending an HTTP request to the ksqlDB REST API, and -the API sends back a chunked response of indefinite length. - - - -Page last revised on: {{ git_revision_date }} diff --git a/docs-md/concepts/queries/index.md b/docs-md/concepts/queries/index.md index 6188603d9f49..8b0fbf8c89e0 100644 --- a/docs-md/concepts/queries/index.md +++ b/docs-md/concepts/queries/index.md @@ -1,10 +1,35 @@ --- layout: page title: Queries -tagline: Query event streams -description: Learn how to query event streams by using the SELECT statement. -keywords: ksqldb, query, select +tagline: Query materialized views of event streams +description: Learn how to query materialized views of event streams by using the SELECT statement. +keywords: ksqldb, query, select, pull, push, materialized view --- -- [Pull Queries](pull.md) -- [Push Queries](push.md) \ No newline at end of file +ksqlDB has a rich set of constructs for both storing events in collections and +deriving new ones through stream processing. Sometimes, you need to process +your events by aggregating them together into a materialized view. In this +case, you need a way for your applications or microservices to leverage this +view. That's where queries come in. + +Push and pull queries +--------------------- + +Queries enable you to ask questions about materialized views. ksqlDB supports +two different kinds of client-issued queries: push and pull. + +- [Push Queries](push.md) enable you to subscribe to a result as it changes in + real-time. +- [Pull Queries](pull.md) enable you to look up information at a point in time. + +ksqlDB supports both kinds of queries by using SQL over its REST API. Combining +them enables you to build powerful real-time applications. + +API Reference +------------- + +- [SELECT (Push Query)](../../developer-guide/ksqldb-reference/select-push-query.md) +- [SELECT (Pull Query)](../../developer-guide/ksqldb-reference/select-pull-query.md) + + +Page last revised on: {{ git_revision_date }} \ No newline at end of file diff --git a/docs-md/concepts/queries/pull.md b/docs-md/concepts/queries/pull.md index 590912180ace..41d6ba4a130b 100644 --- a/docs-md/concepts/queries/pull.md +++ b/docs-md/concepts/queries/pull.md @@ -3,5 +3,29 @@ layout: page title: Pull Queries tagline: Query instantaneous state description: Learn how to use point-in-time queries by using the SELECT statement. -keywords: ksqldb, query, select ---- \ No newline at end of file +keywords: ksqldb, pull, query, select +--- + +Pull queries are a form of query issued by a client that retrieve a result as +of "now". As a dual to the push query example, a pull query for a credit score +would be asking for the current score of a particular user. Because it is a +pull query, it returns immediately with a finite result and closes its +connection. This is ideal for rendering a user interface once at page load +time. It's generally a good fit for any sort of synchronous control flow. + +Pull queries are expressed using a strict subset of ANSI SQL. They can only be +used to query tables that have been processed into materialized views. +Currently, pull queries only support looking up events by key. They're executed +by running over the ksqlDB REST API. The result of a pull query isn't persisted +anywhere. + +Pull queries enable you to fetch the current state of a materialized view. +Because materialized views are incrementally updated as new events arrive, +pull queries run with predictably low latency. They're a great match for +request/response flows. For asynchronous application flows, see +[Push Query](push.md). + +Execute a pull query by sending an HTTP request to the ksqlDB REST API, and +the API responds with a single response. + +Page last revised on: {{ git_revision_date }} \ No newline at end of file diff --git a/docs-md/concepts/queries/push.md b/docs-md/concepts/queries/push.md index a96c676b6507..9741245b14f5 100644 --- a/docs-md/concepts/queries/push.md +++ b/docs-md/concepts/queries/push.md @@ -1,7 +1,31 @@ --- layout: page title: Push Queries -tagline: Query instantaneous state +tagline: Query state continuously description: Learn about continuously updated queries by using the SELECT statement. -keywords: ksqldb, query, select ---- \ No newline at end of file +keywords: ksqldb, push, query, select +--- + +Push queries are a form of query issued by a client that subscribe to a result +as it changes in real-time. A good example of a push query is subscribing to a +particular user’s credit score. The query requests the value of the credit +score. Because it's a push query, any change to the credit score is "pushed" +to the client as soon as it occurs over a long-lived connection. This is useful +for building programmatically controlled microservices, real-time apps, or any +sort of asynchronous control flow. + +Push queries are expressed using a SQL-like language. They can be used to query +either streams or tables for a particular key. They’re executed by running over +the ksqlDB REST API. The result of a push query isn't persisted to a backing +{{ site.ak }} topic. + +Push queries enable you to query a materialized view with a subscription to +the results. Push queries emit refinements to materialized views, which enable +reacting to new information in real-time. They’re a good fit for asynchronous +application flows. For request/response flows, see +[Pull Query](pull.md). + +Execute a push query by sending an HTTP request to the ksqlDB REST API, and +the API sends back a chunked response of indefinite length. + +Page last revised on: {{ git_revision_date }} \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 4018c27edd7a..762f6a64e1a8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -38,28 +38,23 @@ nav: - Architecture: concepts/ksql-architecture.md - Events: concepts/events.md - Collections: - - Overview: concepts/collections/overview.md + - Collections Overview: concepts/collections/overview.md - Streams: concepts/collections/streams.md - Tables: concepts/collections/tables.md - Inserting events: concepts/collections/inserting-events.md - Stream Processing: concepts/stream-processing.md - Materialized Views: concepts/materialized-views.md - Queries: - - Overview: concepts/queries/index.md - - Push queries: concepts/queries/push.md - - Pull queries: concepts/queries/pull.md + - Queries Overview: concepts/queries/index.md + - Push Queries: concepts/queries/push.md + - Pull Queries: concepts/queries/pull.md # - Processing Guarantees (TODO): concepts/ksqldb-processing-guarantees.md # future - #- Push and Pull Queries (TODO): concepts/push-and-pull-queries.md # - Schemas (TODO): concepts/ksqldb-schemas.md # remove - Connectors (TODO): concepts/connectors.md #- Functions: # Suggest flattening - Functions: concepts/functions/udfs.md # Unless there's more abstraction to discuss, move from dev guide node and add a bit more on intrinsic functions, with links - Time and Windows: concepts/time-and-windows-in-ksql-queries.md - Serialization: developer-guide/serialization.md - # - Deploying: : # Suggest renaming the Operate node to "Operate and Deploy" - # - Operations: operations.md - # - Capacity planning: capacity-planning.md - #- How it works: # Suggest flattening - Relationship to Kafka Streams: concepts/ksql-and-kafka-streams.md - Developer Guide: # Renamed from "How To" - Develop ksqlDB Applications: developer-guide/index.md # index topic @@ -155,6 +150,8 @@ markdown_extensions: - toc: permalink:  - admonition + - codehilite: + guess_lang: false - def_list - mdx_gh_links: user: JimGalasyn