From a0cdf21feeedfcb232539ac68f2db735fb9cd32a Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 4 Dec 2023 15:44:01 -0700 Subject: [PATCH 01/13] Add kv processor content gap doc Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 44 +++++++++++++++++++++++ _ingest-pipelines/processors/lowercase.md | 2 +- 2 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 _ingest-pipelines/processors/kv.md diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md new file mode 100644 index 0000000000..d273e6b563 --- /dev/null +++ b/_ingest-pipelines/processors/kv.md @@ -0,0 +1,44 @@ +--- +layout: default +title: KV +parent: Ingest processors +nav_order: 200 +redirect_from: + - /api-reference/ingest-apis/processors/lowercase/ +--- + +# KV + +The `kv` processor automatically extracts specific event fields or messages that are `key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. + +## Example +The following is the syntax for the `lowercase` processor: + +```json +{ + "kv": { + "field": "message", + "field_split": " ", + "value_split": " " + } +} +``` +{% include copy-curl.html %} + +#### Configuration parameters + +The following table lists the required and optional parameters for the `lowercase` processor. + +| Name | Required/Optional | Description | +|---|---|---| +`field` | Required | The name of the field that contains the data to be parsed. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | +`field_split` | Required | The regex pattern for key-value pair splitting. | +`value_split` | Required | The regex pattern for splitting the key from the value within a key-value pair, for example, equal sign `=` or colon `:`. +`include_keys` | Optional | The keys for filtering and inserting. Default is to include all keys. +`description` | Optional | A brief description of the processor. | +`if` | Optional | A condition for running this processor. | +`ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | +`on_failure` | Optional | A list of processors to run if the processor fails. | +`ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not have the specified field. Default is `false`. | +`tag` | Optional | An identifier tag for the processor. Useful for debugging to distinguish between processors of the same type. | +`target_field` | Optional | The name of the field in which to insert the extracted keys. Default is `null`. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | \ No newline at end of file diff --git a/_ingest-pipelines/processors/lowercase.md b/_ingest-pipelines/processors/lowercase.md index 938cd52ace..7450b55a27 100644 --- a/_ingest-pipelines/processors/lowercase.md +++ b/_ingest-pipelines/processors/lowercase.md @@ -31,7 +31,7 @@ The following table lists the required and optional parameters for the `lowercas | Name | Required | Description | |---|---|---| -`field` | Required | The name of the field that contains the data to be converted. Supports template snippets. | +`field` | Required | The name of the field that contains the data to be converted. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | `description` | Optional | A brief description of the processor. | `if` | Optional | A condition for running this processor. | `ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | From 388727fb5aac5321a10328bad67292e998e6239b Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 5 Dec 2023 09:10:00 -0700 Subject: [PATCH 02/13] Add parameters section Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 78 +++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index d273e6b563..2f894f870e 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -34,11 +34,85 @@ The following table lists the required and optional parameters for the `lowercas `field` | Required | The name of the field that contains the data to be parsed. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | `field_split` | Required | The regex pattern for key-value pair splitting. | `value_split` | Required | The regex pattern for splitting the key from the value within a key-value pair, for example, equal sign `=` or colon `:`. -`include_keys` | Optional | The keys for filtering and inserting. Default is to include all keys. +`exclude_keys` | Optional | The keys to exclude from the document. Default is `null`. | +`include_keys` | Optional | The keys for filtering and inserting. Default is to include all keys. | +`prefix` | Optional | The prefix to add to the extracted keys. Default is `null`. | +`strip_brackets` | Optional | If set to `true`, strips brackets `()`, `<>,` or `[]` and quotes `'` or `"` from extracted values. Default is `false`. +`trim_key` | Optional | String of characters to trim from extracted keys. | +`trim value` | Optional | String of characters to trim from extracted values. | `description` | Optional | A brief description of the processor. | `if` | Optional | A condition for running this processor. | `ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | `on_failure` | Optional | A list of processors to run if the processor fails. | `ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not have the specified field. Default is `false`. | `tag` | Optional | An identifier tag for the processor. Useful for debugging to distinguish between processors of the same type. | -`target_field` | Optional | The name of the field in which to insert the extracted keys. Default is `null`. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | \ No newline at end of file +`target_field` | Optional | The name of the field in which to insert the extracted keys. Default is `null`. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | + +## Using the processor + +Follow these steps to use the processor in a pipeline. + +**Step 1: Create a pipeline.** + +The following query creates a pipeline, named `kv-pipeline`, that uses the `kv` processor to extract the `message` field of a document: + +```json +PUT _ingest/pipeline/kv-pipeline +{ + "description" : "Pipeline that extracts user profile data", + "processors" : [ + { + "kv" : { + "field" : "message", + "field_split": "", + "value_split": "=" + } + } + ] +} +``` +{% include copy-curl.html %} + +**Step 2 (Optional): Test the pipeline.** + +It is recommended that you test your pipeline before you ingest documents. +{: .tip} + +To test the pipeline, run the following query: + +```json +POST _ingest/pipeline/kv-pipeline/_simulate + + + +``` +{% include copy-curl.html %} + +#### Response + +The following example response confirms that the pipeline is working as expected: + +```json + + + +``` + +**Step 3: Ingest a document.** + +The following query ingests a document into an index named `testindex1`: + +```json +PUT testindex1/_doc/1?pipeline=kv-pipeline + +``` +{% include copy-curl.html %} + +**Step 4 (Optional): Retrieve the document.** + +To retrieve the document, run the following query: + +```json +GET testindex1/_doc/1 +``` +{% include copy-curl.html %} From eb203df0f7c9c5816e05fe2845614ac64c3c64e2 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 5 Dec 2023 10:24:47 -0700 Subject: [PATCH 03/13] Add parameters section Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index 2f894f870e..5ffe4fbf4d 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -7,7 +7,7 @@ redirect_from: - /api-reference/ingest-apis/processors/lowercase/ --- -# KV +# KV processor The `kv` processor automatically extracts specific event fields or messages that are `key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. From 87a09364434c73f265203fe81c5920727036a599 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Fri, 8 Dec 2023 11:23:18 -0700 Subject: [PATCH 04/13] Update _ingest-pipelines/processors/kv.md Co-authored-by: Andriy Redko Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index 5ffe4fbf4d..ceec9646a0 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -9,7 +9,7 @@ redirect_from: # KV processor -The `kv` processor automatically extracts specific event fields or messages that are `key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. +The `kv` processor automatically extracts specific event fields or messages that are in a`key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. ## Example The following is the syntax for the `lowercase` processor: From 5037f3771bfc30e1fc696b8d7dcff6e7884e8ba5 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Fri, 8 Dec 2023 14:06:27 -0700 Subject: [PATCH 05/13] Address tech review feedback Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 47 ++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 9 deletions(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index ceec9646a0..2d7efce6e8 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -9,10 +9,10 @@ redirect_from: # KV processor -The `kv` processor automatically extracts specific event fields or messages that are in a`key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. +The `kv` processor automatically extracts specific event fields or messages that are in a`key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. ## Example -The following is the syntax for the `lowercase` processor: +The following is the syntax for the `kv` processor: ```json { @@ -64,7 +64,7 @@ PUT _ingest/pipeline/kv-pipeline { "kv" : { "field" : "message", - "field_split": "", + "field_split": " ", "value_split": "=" } } @@ -83,8 +83,18 @@ To test the pipeline, run the following query: ```json POST _ingest/pipeline/kv-pipeline/_simulate - - +```json +{ + "docs": [ + { + "_index": "testindex1", + "_id": "1", + "_source":{ + "message": "goodbye=everybody hello=world" + } + } + ] +} ``` {% include copy-curl.html %} @@ -93,9 +103,24 @@ POST _ingest/pipeline/kv-pipeline/_simulate The following example response confirms that the pipeline is working as expected: ```json - - - +{ + "docs": [ + { + "doc": { + "_index": "testindex1", + "_id": "1", + "_source": { + "hello": "world", + "message": "goodbye=everybody hello=world", + "goodbye": "everybody" + }, + "_ingest": { + "timestamp": "2023-12-06T09:59:21.823292Z" + } + } + } + ] +} ``` **Step 3: Ingest a document.** @@ -104,7 +129,11 @@ The following query ingests a document into an index named `testindex1`: ```json PUT testindex1/_doc/1?pipeline=kv-pipeline - + +```json +{ + "message": "goodbye=everybody hello=world" +} ``` {% include copy-curl.html %} From f46eb5a6da7d0e972ab14e5a00cc28b1d2c3b008 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 11 Dec 2023 16:09:10 -0700 Subject: [PATCH 06/13] Update _ingest-pipelines/processors/kv.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index 2d7efce6e8..28d87191f5 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -25,7 +25,7 @@ The following is the syntax for the `kv` processor: ``` {% include copy-curl.html %} -#### Configuration parameters +## Configuration parameters The following table lists the required and optional parameters for the `lowercase` processor. From e4a8c691569464a9259e0d0f3df2580219acb0e3 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 11 Dec 2023 16:09:25 -0700 Subject: [PATCH 07/13] Update _ingest-pipelines/processors/kv.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index 28d87191f5..c2e0e88465 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -39,7 +39,7 @@ The following table lists the required and optional parameters for the `lowercas `prefix` | Optional | The prefix to add to the extracted keys. Default is `null`. | `strip_brackets` | Optional | If set to `true`, strips brackets `()`, `<>,` or `[]` and quotes `'` or `"` from extracted values. Default is `false`. `trim_key` | Optional | String of characters to trim from extracted keys. | -`trim value` | Optional | String of characters to trim from extracted values. | +`trim value` | Optional | The string of characters to trim from the extracted values. | `description` | Optional | A brief description of the processor. | `if` | Optional | A condition for running this processor. | `ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | From 6c69453313b65844437351c3d9f6224ab6f0023a Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 11 Dec 2023 16:09:32 -0700 Subject: [PATCH 08/13] Update _ingest-pipelines/processors/lowercase.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/lowercase.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/lowercase.md b/_ingest-pipelines/processors/lowercase.md index 6a190418a1..84208fc1e9 100644 --- a/_ingest-pipelines/processors/lowercase.md +++ b/_ingest-pipelines/processors/lowercase.md @@ -29,7 +29,7 @@ The following is the syntax for the `lowercase` processor: The following table lists the required and optional parameters for the `lowercase` processor. -| Name | Required | Description | +| Parameter | Required | Description | |---|---|---| `field` | Required | The name of the field that contains the data to be converted. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | `description` | Optional | A brief description of the processor. | From 74a52228f08bccbede7ae4a4cd8d8f362313bd1e Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 11 Dec 2023 16:09:47 -0700 Subject: [PATCH 09/13] Update _ingest-pipelines/processors/kv.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index c2e0e88465..184a950652 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -38,7 +38,7 @@ The following table lists the required and optional parameters for the `lowercas `include_keys` | Optional | The keys for filtering and inserting. Default is to include all keys. | `prefix` | Optional | The prefix to add to the extracted keys. Default is `null`. | `strip_brackets` | Optional | If set to `true`, strips brackets `()`, `<>,` or `[]` and quotes `'` or `"` from extracted values. Default is `false`. -`trim_key` | Optional | String of characters to trim from extracted keys. | +`trim_key` | Optional | The string of characters to trim from the extracted keys. | `trim value` | Optional | The string of characters to trim from the extracted values. | `description` | Optional | A brief description of the processor. | `if` | Optional | A condition for running this processor. | From 3639d820bff36e5905d5f4bd53a5adcd29f2204f Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 11 Dec 2023 16:16:42 -0700 Subject: [PATCH 10/13] Copy edits Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index 184a950652..601ff6331d 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -27,9 +27,9 @@ The following is the syntax for the `kv` processor: ## Configuration parameters -The following table lists the required and optional parameters for the `lowercase` processor. +The following table lists the required and optional parameters for the `kv` processor. -| Name | Required/Optional | Description | +| Parameter | Required/Optional | Description | |---|---|---| `field` | Required | The name of the field that contains the data to be parsed. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | `field_split` | Required | The regex pattern for key-value pair splitting. | From 6cb60fa1aa6be0ac3fd4a0928788fa62f882c23c Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 12 Dec 2023 10:54:49 -0700 Subject: [PATCH 11/13] Update _ingest-pipelines/processors/kv.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index 601ff6331d..e1fa199d90 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -9,7 +9,7 @@ redirect_from: # KV processor -The `kv` processor automatically extracts specific event fields or messages that are in a`key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. +The `kv` processor automatically extracts specific event fields or messages that are in a `key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. ## Example The following is the syntax for the `kv` processor: From e3e8fe06a774af5110b69e56b6c5845026beb19f Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 12 Dec 2023 11:14:26 -0700 Subject: [PATCH 12/13] Update _ingest-pipelines/processors/lowercase.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/lowercase.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ingest-pipelines/processors/lowercase.md b/_ingest-pipelines/processors/lowercase.md index 84208fc1e9..f9dfbc1e6c 100644 --- a/_ingest-pipelines/processors/lowercase.md +++ b/_ingest-pipelines/processors/lowercase.md @@ -31,7 +31,7 @@ The following table lists the required and optional parameters for the `lowercas | Parameter | Required | Description | |---|---|---| -`field` | Required | The name of the field that contains the data to be converted. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | +`field` | Required | The name of the field containing the data to be converted. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | `description` | Optional | A brief description of the processor. | `if` | Optional | A condition for running this processor. | `ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | From 37082cd57f03a25f0442dc55490f1a2e4d2f0353 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 12 Dec 2023 11:17:33 -0700 Subject: [PATCH 13/13] Address editorial feedback Signed-off-by: Melissa Vagi --- _ingest-pipelines/processors/kv.md | 26 +++++++++++------------ _ingest-pipelines/processors/lowercase.md | 2 +- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/_ingest-pipelines/processors/kv.md b/_ingest-pipelines/processors/kv.md index e1fa199d90..c81025360f 100644 --- a/_ingest-pipelines/processors/kv.md +++ b/_ingest-pipelines/processors/kv.md @@ -9,9 +9,10 @@ redirect_from: # KV processor -The `kv` processor automatically extracts specific event fields or messages that are in a `key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data such as user behavior analytics, performance optimizations, or security investigations. +The `kv` processor automatically extracts specific event fields or messages that are in a `key=value` format. This structured format organizes your data by grouping it together based on keys and values. It's helpful for analyzing, visualizing, and using data, such as user behavior analytics, performance optimizations, or security investigations. ## Example + The following is the syntax for the `kv` processor: ```json @@ -30,31 +31,30 @@ The following is the syntax for the `kv` processor: The following table lists the required and optional parameters for the `kv` processor. | Parameter | Required/Optional | Description | -|---|---|---| -`field` | Required | The name of the field that contains the data to be parsed. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | +`field` | Required | The name of the field containing the data to be parsed. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | `field_split` | Required | The regex pattern for key-value pair splitting. | `value_split` | Required | The regex pattern for splitting the key from the value within a key-value pair, for example, equal sign `=` or colon `:`. `exclude_keys` | Optional | The keys to exclude from the document. Default is `null`. | `include_keys` | Optional | The keys for filtering and inserting. Default is to include all keys. | `prefix` | Optional | The prefix to add to the extracted keys. Default is `null`. | -`strip_brackets` | Optional | If set to `true`, strips brackets `()`, `<>,` or `[]` and quotes `'` or `"` from extracted values. Default is `false`. +`strip_brackets` | Optional | If set to `true`, strips brackets (`()`, `<>,` or `[]`) and quotes (`'` or `"`) from extracted values. Default is `false`. `trim_key` | Optional | The string of characters to trim from the extracted keys. | `trim value` | Optional | The string of characters to trim from the extracted values. | `description` | Optional | A brief description of the processor. | -`if` | Optional | A condition for running this processor. | +`if` | Optional | A condition for running the processor. | `ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | `on_failure` | Optional | A list of processors to run if the processor fails. | -`ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not have the specified field. Default is `false`. | -`tag` | Optional | An identifier tag for the processor. Useful for debugging to distinguish between processors of the same type. | +`ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not contain the specified field. Default is `false`. | +`tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. | `target_field` | Optional | The name of the field in which to insert the extracted keys. Default is `null`. Supports [template snippets]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/#template-snippets). | ## Using the processor Follow these steps to use the processor in a pipeline. -**Step 1: Create a pipeline.** +**Step 1: Create a pipeline** -The following query creates a pipeline, named `kv-pipeline`, that uses the `kv` processor to extract the `message` field of a document: +The following query creates a pipeline, named `kv-pipeline`, that uses the `kv` processor to extract the `message` field of a document: ```json PUT _ingest/pipeline/kv-pipeline @@ -73,7 +73,7 @@ PUT _ingest/pipeline/kv-pipeline ``` {% include copy-curl.html %} -**Step 2 (Optional): Test the pipeline.** +**Step 2 (Optional): Test the pipeline** It is recommended that you test your pipeline before you ingest documents. {: .tip} @@ -98,7 +98,7 @@ POST _ingest/pipeline/kv-pipeline/_simulate ``` {% include copy-curl.html %} -#### Response +**Response** The following example response confirms that the pipeline is working as expected: @@ -123,7 +123,7 @@ The following example response confirms that the pipeline is working as expected } ``` -**Step 3: Ingest a document.** +**Step 3: Ingest a document** The following query ingests a document into an index named `testindex1`: @@ -137,7 +137,7 @@ PUT testindex1/_doc/1?pipeline=kv-pipeline ``` {% include copy-curl.html %} -**Step 4 (Optional): Retrieve the document.** +**Step 4 (Optional): Retrieve the document** To retrieve the document, run the following query: diff --git a/_ingest-pipelines/processors/lowercase.md b/_ingest-pipelines/processors/lowercase.md index f9dfbc1e6c..706899ed6f 100644 --- a/_ingest-pipelines/processors/lowercase.md +++ b/_ingest-pipelines/processors/lowercase.md @@ -37,7 +37,7 @@ The following table lists the required and optional parameters for the `lowercas `ignore_failure` | Optional | If set to `true`, failures are ignored. Default is `false`. | `on_failure` | Optional | A list of processors to run if the processor fails. | `ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not have the specified field. Default is `false`. | -`tag` | Optional | An identifier tag for the processor. Useful for debugging to distinguish between processors of the same type. | +`tag` | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. | `target_field` | Optional | The name of the field in which to store the parsed data. Default is `field`. By default, `field` is updated in place. | ## Using the processor