Add support for marking component templates as deprecated #101148

felixbarny · 2023-10-19T15:05:15Z

Deprecated component templates don't appear when using a wildcard search by default.

This also needs to be extended to composable index templates, ILM policies, and ingest pipelines.

At this point this is just a POC and means as a conversation starter on whether it's worth implementing this.

closes #100992

They'll also not appear when using a wildcard search by default

server/src/main/java/org/elasticsearch/cluster/metadata/ComponentTemplate.java

…entTemplate.java Co-authored-by: eyalkoren <41850454+eyalkoren@users.noreply.github.com>

also add docs

…templates

…ort version is sufficient

…templates

elasticsearchmachine · 2023-10-25T15:29:10Z

Hi @felixbarny, I've created a changelog YAML for you.

dakrone

this looks like a good start to me, I don't have anything other than minor comments so far

server/src/main/java/org/elasticsearch/cluster/metadata/ComponentTemplate.java

server/src/test/java/org/elasticsearch/cluster/metadata/ComponentTemplateTests.java

server/src/test/java/org/elasticsearch/cluster/metadata/ComposableIndexTemplateTests.java

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/LifecyclePolicyTests.java

server/src/main/java/org/elasticsearch/cluster/metadata/MetadataIndexTemplateService.java

…templates

elasticsearchmachine · 2023-10-27T07:56:05Z

Pinging @elastic/es-data-management (Team:Data Management)

…templates

dakrone

This LGTM. I left some minor comments but nothing major. I think this will require merging master and adjusting the TransportVersions before merging though.

server/src/main/java/org/elasticsearch/cluster/metadata/ComponentTemplate.java

server/src/main/java/org/elasticsearch/cluster/metadata/ComposableIndexTemplate.java

server/src/main/java/org/elasticsearch/ingest/Pipeline.java

dakrone · 2023-10-30T22:12:27Z

test/framework/src/main/java/org/elasticsearch/test/ESTestCase.java

@@ -809,6 +809,10 @@ public static boolean randomBoolean() {
        return random().nextBoolean();
    }

+    public static Boolean randomOptionalBoolean() {
+        return randomBoolean() ? Boolean.TRUE : randomFrom(Boolean.FALSE, null);


This changes the statistics of the responses somewhat, what about

Suggested change

return randomBoolean() ? Boolean.TRUE : randomFrom(Boolean.FALSE, null);

return randomFrom(Boolean.TRUE, Boolean.FALSE, null);

This was on purpose because otherwise, "falsey" return values (Boolean.FALSE, null) would have a higher probability. As this is a new method, and mostly used for new code, it doesn't really alter existing behavior. But it does assume that null is equivalent to false, which may not always be true in all cases.

I'll leave this as-is for now but happy to change if you still think it would be better. But I'll not block merging.

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ilm/LifecyclePolicy.java

…templates

Partially addresses #170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when an ingest pipeline will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the ingest pipelines UI to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> Ingest pipelines * Verify that: * Deprecated pipelines are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated policies are highlighted and when editing, we also show a warning callout. ### Screenshot <img width="1386" alt="Screenshot 2024-02-05 at 16 02 18" src="https://github.com/elastic/kibana/assets/6585477/739d6fea-5f15-4da4-aee8-cae8f412a040"> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

…onent template UIs (#174450) Partially addresses #170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when a component template or index template will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the component template and index template UIs to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> index management (component_templates|index_templates) * Verify that: * Deprecated resources are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated resources are highlighted and when editing, we also show a warning callout. <details> <summary>Create deprecated component template</summary> ``` PUT _component_template/template_1 { "template": { "settings" : { "number_of_shards" : 1 } }, "deprecated": true } ``` </details> <details> <summary>Screenshots</summary> // index templates ![Screenshot 2024-01-10 at 15 25 26](https://github.com/elastic/kibana/assets/1191206/ff632464-c4ac-43a2-b89d-be423489a82a) ![Screenshot 2024-01-10 at 15 25 31](https://github.com/elastic/kibana/assets/1191206/bb231c7c-5ae3-466d-9411-3cca842124dc) ![Screenshot 2024-01-08 at 13 52 35](https://github.com/elastic/kibana/assets/1191206/c6724867-3a0e-4998-871b-35fd577689bc) ![Screenshot 2024-01-08 at 14 00 46](https://github.com/elastic/kibana/assets/1191206/0b1001d4-c0bb-49a8-862a-746754e34fe0) // component templates ![Screenshot 2024-01-10 at 14 45 10](https://github.com/elastic/kibana/assets/1191206/675519b8-80d7-466a-a4ad-185bb7f52651) ![Screenshot 2024-01-10 at 14 45 15](https://github.com/elastic/kibana/assets/1191206/0cbe5704-8549-4812-9202-e44b8f02ac95) ![Screenshot 2024-01-08 at 14 51 03](https://github.com/elastic/kibana/assets/1191206/0288ca12-c8f5-4ed5-ba4c-f77834702e68) ![Screenshot 2024-01-08 at 15 07 17](https://github.com/elastic/kibana/assets/1191206/97f72f52-6816-4bc9-910c-fbe88df50ad2) </details> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

Partially addresses elastic#170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when an ingest pipeline will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the ingest pipelines UI to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> Ingest pipelines * Verify that: * Deprecated pipelines are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated policies are highlighted and when editing, we also show a warning callout. ### Screenshot <img width="1386" alt="Screenshot 2024-02-05 at 16 02 18" src="https://github.com/elastic/kibana/assets/6585477/739d6fea-5f15-4da4-aee8-cae8f412a040"> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

…onent template UIs (elastic#174450) Partially addresses elastic#170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when a component template or index template will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the component template and index template UIs to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> index management (component_templates|index_templates) * Verify that: * Deprecated resources are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated resources are highlighted and when editing, we also show a warning callout. <details> <summary>Create deprecated component template</summary> ``` PUT _component_template/template_1 { "template": { "settings" : { "number_of_shards" : 1 } }, "deprecated": true } ``` </details> <details> <summary>Screenshots</summary> // index templates ![Screenshot 2024-01-10 at 15 25 26](https://github.com/elastic/kibana/assets/1191206/ff632464-c4ac-43a2-b89d-be423489a82a) ![Screenshot 2024-01-10 at 15 25 31](https://github.com/elastic/kibana/assets/1191206/bb231c7c-5ae3-466d-9411-3cca842124dc) ![Screenshot 2024-01-08 at 13 52 35](https://github.com/elastic/kibana/assets/1191206/c6724867-3a0e-4998-871b-35fd577689bc) ![Screenshot 2024-01-08 at 14 00 46](https://github.com/elastic/kibana/assets/1191206/0b1001d4-c0bb-49a8-862a-746754e34fe0) // component templates ![Screenshot 2024-01-10 at 14 45 10](https://github.com/elastic/kibana/assets/1191206/675519b8-80d7-466a-a4ad-185bb7f52651) ![Screenshot 2024-01-10 at 14 45 15](https://github.com/elastic/kibana/assets/1191206/0cbe5704-8549-4812-9202-e44b8f02ac95) ![Screenshot 2024-01-08 at 14 51 03](https://github.com/elastic/kibana/assets/1191206/0288ca12-c8f5-4ed5-ba4c-f77834702e68) ![Screenshot 2024-01-08 at 15 07 17](https://github.com/elastic/kibana/assets/1191206/97f72f52-6816-4bc9-910c-fbe88df50ad2) </details> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

Partially addresses elastic#170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when an ingest pipeline will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the ingest pipelines UI to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> Ingest pipelines * Verify that: * Deprecated pipelines are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated policies are highlighted and when editing, we also show a warning callout. ### Screenshot <img width="1386" alt="Screenshot 2024-02-05 at 16 02 18" src="https://github.com/elastic/kibana/assets/6585477/739d6fea-5f15-4da4-aee8-cae8f412a040"> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

…onent template UIs (elastic#174450) Partially addresses elastic#170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when a component template or index template will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the component template and index template UIs to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> index management (component_templates|index_templates) * Verify that: * Deprecated resources are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated resources are highlighted and when editing, we also show a warning callout. <details> <summary>Create deprecated component template</summary> ``` PUT _component_template/template_1 { "template": { "settings" : { "number_of_shards" : 1 } }, "deprecated": true } ``` </details> <details> <summary>Screenshots</summary> // index templates ![Screenshot 2024-01-10 at 15 25 26](https://github.com/elastic/kibana/assets/1191206/ff632464-c4ac-43a2-b89d-be423489a82a) ![Screenshot 2024-01-10 at 15 25 31](https://github.com/elastic/kibana/assets/1191206/bb231c7c-5ae3-466d-9411-3cca842124dc) ![Screenshot 2024-01-08 at 13 52 35](https://github.com/elastic/kibana/assets/1191206/c6724867-3a0e-4998-871b-35fd577689bc) ![Screenshot 2024-01-08 at 14 00 46](https://github.com/elastic/kibana/assets/1191206/0b1001d4-c0bb-49a8-862a-746754e34fe0) // component templates ![Screenshot 2024-01-10 at 14 45 10](https://github.com/elastic/kibana/assets/1191206/675519b8-80d7-466a-a4ad-185bb7f52651) ![Screenshot 2024-01-10 at 14 45 15](https://github.com/elastic/kibana/assets/1191206/0cbe5704-8549-4812-9202-e44b8f02ac95) ![Screenshot 2024-01-08 at 14 51 03](https://github.com/elastic/kibana/assets/1191206/0288ca12-c8f5-4ed5-ba4c-f77834702e68) ![Screenshot 2024-01-08 at 15 07 17](https://github.com/elastic/kibana/assets/1191206/97f72f52-6816-4bc9-910c-fbe88df50ad2) </details> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

Partially addresses elastic#170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when an ingest pipeline will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the ingest pipelines UI to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> Ingest pipelines * Verify that: * Deprecated pipelines are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated policies are highlighted and when editing, we also show a warning callout. ### Screenshot <img width="1386" alt="Screenshot 2024-02-05 at 16 02 18" src="https://github.com/elastic/kibana/assets/6585477/739d6fea-5f15-4da4-aee8-cae8f412a040"> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

…onent template UIs (elastic#174450) Partially addresses elastic#170805 ## Summary With elastic/elasticsearch#101148 we now can alert the users when a component template or index template will soon be deprecated and shouldnt be relied upon. This PR adds a badge and a callout to the component template and index template UIs to alert the users about this. ##### How to test * Start up kibana and es * Navigate to Stack management -> index management (component_templates|index_templates) * Verify that: * Deprecated resources are hidden by default * Upon enabling filtering to see them they should pop up on the table with a badge * Verify that deprecated resources are highlighted and when editing, we also show a warning callout. <details> <summary>Create deprecated component template</summary> ``` PUT _component_template/template_1 { "template": { "settings" : { "number_of_shards" : 1 } }, "deprecated": true } ``` </details> <details> <summary>Screenshots</summary> // index templates ![Screenshot 2024-01-10 at 15 25 26](https://github.com/elastic/kibana/assets/1191206/ff632464-c4ac-43a2-b89d-be423489a82a) ![Screenshot 2024-01-10 at 15 25 31](https://github.com/elastic/kibana/assets/1191206/bb231c7c-5ae3-466d-9411-3cca842124dc) ![Screenshot 2024-01-08 at 13 52 35](https://github.com/elastic/kibana/assets/1191206/c6724867-3a0e-4998-871b-35fd577689bc) ![Screenshot 2024-01-08 at 14 00 46](https://github.com/elastic/kibana/assets/1191206/0b1001d4-c0bb-49a8-862a-746754e34fe0) // component templates ![Screenshot 2024-01-10 at 14 45 10](https://github.com/elastic/kibana/assets/1191206/675519b8-80d7-466a-a4ad-185bb7f52651) ![Screenshot 2024-01-10 at 14 45 15](https://github.com/elastic/kibana/assets/1191206/0cbe5704-8549-4812-9202-e44b8f02ac95) ![Screenshot 2024-01-08 at 14 51 03](https://github.com/elastic/kibana/assets/1191206/0288ca12-c8f5-4ed5-ba4c-f77834702e68) ![Screenshot 2024-01-08 at 15 07 17](https://github.com/elastic/kibana/assets/1191206/97f72f52-6816-4bc9-910c-fbe88df50ad2) </details> --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com> Co-authored-by: Yulia Cech <yulia.cech@elastic.co>

Add support for marking component templates as deprecated

99d78c6

They'll also not appear when using a wildcard search by default

elasticsearchmachine added external-contributor Pull request authored by a developer outside the Elasticsearch team v8.12.0 labels Oct 19, 2023

felixbarny mentioned this pull request Oct 19, 2023

Adding the ability to mark templates and pipelines as deprecated #100992

Closed

eyalkoren reviewed Oct 19, 2023

View reviewed changes

server/src/main/java/org/elasticsearch/cluster/metadata/ComponentTemplate.java Outdated Show resolved Hide resolved

felixbarny and others added 13 commits October 20, 2023 08:18

Update server/src/main/java/org/elasticsearch/cluster/metadata/Compon…

dbc561a

…entTemplate.java Co-authored-by: eyalkoren <41850454+eyalkoren@users.noreply.github.com>

Always include deprecated component templates in get template response

45cd902

Check transport version in ComponentTemplate

3178f8f

also add docs

Remove unnecessary yaml rest test

d225979

Add deprecated flag to ComposableIndexTemplate

67538cb

Add deprecated flag to ingest pipelines

f1acb60

Use equals rather than identity check for Booleans

62311cc

Add deprecated flag to ILM policies

cc776cf

Merge remote-tracking branch 'origin/main' into deprecated-component-…

1cbce31

…templates

Apply spotless suggestions

bb92cc9

Read boolean instead of optional boolean when we know that the transp…

0002e2e

…ort version is sufficient

Fix test by fixing propagation of deprecated flag

6e2558a

Merge remote-tracking branch 'origin/main' into deprecated-component-…

bcfb189

…templates

felixbarny requested a review from dakrone October 25, 2023 15:28

felixbarny added >enhancement :Data Management/Indices APIs APIs to create and manage indices and templates labels Oct 25, 2023

Update docs/changelog/101148.yaml

59dcc45

dakrone reviewed Oct 25, 2023

View reviewed changes

Add deprecated flag in mutateInstance

732ee09

felixbarny commented Oct 26, 2023

View reviewed changes

server/src/main/java/org/elasticsearch/cluster/metadata/MetadataIndexTemplateService.java Outdated Show resolved Hide resolved

server/src/main/java/org/elasticsearch/cluster/metadata/MetadataIndexTemplateService.java Show resolved Hide resolved

felixbarny added 4 commits October 26, 2023 08:43

Merge remote-tracking branch 'origin/main' into deprecated-component-…

17fd399

…templates

Apply spotless suggestions

b0dadbf

Retain unset/true/false

e93ab49

Merge remote-tracking branch 'origin/main' into deprecated-component-…

a8580fb

…templates

felixbarny added 2 commits October 27, 2023 15:35

Attempt to fix tests

fdf10ea

Merge remote-tracking branch 'origin/main' into deprecated-component-…

9d898e0

…templates

felixbarny mentioned this pull request Oct 27, 2023

Mark legacy stack templates as deprecated #101476

Merged

dakrone approved these changes Oct 30, 2023

View reviewed changes

felixbarny added 3 commits November 2, 2023 10:54

Merge remote-tracking branch 'origin/main' into deprecated-component-…

93f3784

…templates

Implement suggestions from review

e0caf3f

Add another missing @nullable

35fc26b

ruflin assigned felixbarny Nov 2, 2023

felixbarny added 3 commits November 2, 2023 17:46

Expose Boolean deprecated instance variable to fix tests

7a3cc53

Merge remote-tracking branch 'origin/main' into deprecated-component-…

3b99dfe

…templates

Fix NPE in tests

28e57a2

felixbarny merged commit 978a546 into elastic:main Nov 2, 2023

felixbarny deleted the deprecated-component-templates branch November 2, 2023 18:28

dakrone mentioned this pull request Nov 7, 2023

Add support for the 'deprecated' flag to ILM policies, component templates, index templates, and pipelines elastic/kibana#170805

Closed

This was referenced Jan 3, 2024

[Index lifecycle management] Add deprecated policy warnings elastic/kibana#174150

Merged

[Index Management] Add deprecation warning to index template and component template UIs elastic/kibana#174450

Merged

felixbarny mentioned this pull request Jan 8, 2024

Document the deprecated flag for templates #104052

Closed

sabarasaba mentioned this pull request Jan 14, 2024

[Ingest Pipelines] Highlight deprecated pipelines elastic/kibana#174814

Merged

pquentin mentioned this pull request Jan 27, 2025

Fix cluster.get_component_template API elastic/elasticsearch-specification#3617

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for marking component templates as deprecated #101148

Add support for marking component templates as deprecated #101148

felixbarny commented Oct 19, 2023

elasticsearchmachine commented Oct 25, 2023

dakrone left a comment

elasticsearchmachine commented Oct 27, 2023

dakrone left a comment

dakrone Oct 30, 2023

felixbarny Nov 2, 2023

	return randomBoolean() ? Boolean.TRUE : randomFrom(Boolean.FALSE, null);
	return randomFrom(Boolean.TRUE, Boolean.FALSE, null);

Add support for marking component templates as deprecated #101148

Add support for marking component templates as deprecated #101148

Conversation

felixbarny commented Oct 19, 2023

elasticsearchmachine commented Oct 25, 2023

dakrone left a comment

Choose a reason for hiding this comment

elasticsearchmachine commented Oct 27, 2023

dakrone left a comment

Choose a reason for hiding this comment

dakrone Oct 30, 2023

Choose a reason for hiding this comment

felixbarny Nov 2, 2023

Choose a reason for hiding this comment