[Request] Prebuilt rule customization, upgrade, and export/import workflows

[jpdjere]

Epic: elastic/kibana#174168
Related to: #6238

Summary

Description

We are introducing the ability for users to customize prebuilt Elastic rules and adjusting the rule upgrade workflow to adapt to that change. This includes ability to:

• edit and customize prebuilt rules (modify almost all rule parameters, besides rule actions);
• export and import prebuilt rules, including customized ones;
• upgrade prebuilt rules while keeping the user customizations whenever possible.

See more details below.

Background & resources

• PRs: many, created between the 8.14 and 8.18 release cycles
• Issues/metas: [Security Solution] Users can Customize Prebuilt Detection Rules: Milestone 3 kibana#174168
• Points of contact: @banderror @xcrzx @maximpn @nikitaindik @dplumlee; please reach out via the team channel
• Test environments: document (internal)

Which documentation set does this change impact?

ESS and Serverless

ESS release

8.18.0

Serverless release

TBD, currently targeting first half of February

Feature differences

None.

API docs impact

No impact - most of the changes are made to the internal prebuilt rules API endpoints.

Prerequisites, privileges, feature flags

Feature flag: prebuiltRulesCustomizationEnabled

xpack.securitySolution.enableExperimental: ['prebuiltRulesCustomizationEnabled']

User stories

Source: elastic/kibana#174168

Prebuilt rule customization workflow

• User can edit a single prebuilt rule. User can click "edit" button for prebuilt rules and customize (almost) any field on the Rule Editing page, just like it's possible to do with custom rules.

Before	After

• User can't edit the Author and License fields.

Before	After

• User can bulk edit multiple prebuilt rules via bulk actions.

Before	After

• User can see if the rule is customized on the Rule Details page.
  Note: we do not yet show which fields were customised in UI ( the annotations below are for illustration), the rule customization is shown with the "Modified Elastic rule" badge.

• User can see which rules are customized on the Rule Management page.

Prebuilt rule upgrade workflow

• User can upgrade a single prebuilt rule to its latest version with previewing the incoming updates.
• User can preview updates from Elastic, for each rule field that has an update from Elastic.

Screen.Recording.2025-01-15.at.17.28.06.mov

• User can preview their customizations, for each rule field that was customized.

Screen.Recording.2025-01-15.at.17.40.23.mov

• User can compare their customizations with updates from Elastic and see if there are any conflicts between them, per each rule field.

Screen.Recording.2025-01-15.at.18.27.36.mov

Screen.Recording.2025-01-15.at.18.29.07.mov

• User can manually resolve conflicts between their customizations and updates from Elastic, per each rule field.

Screen.Recording.2025-01-15.at.18.41.03.mov

• User can edit the final field values before submitting the update.

Screen.Recording.2025-01-15.at.18.43.53.mov

• User can upgrade a rule if its type has been changed by Elastic in the latest version, but can only accept the incoming changes.

Screen.Recording.2025-01-15.at.18.48.23.mov

Screen.Recording.2025-01-15.at.18.49.40.mov

• User can upgrade a single prebuilt rule to its latest version without previewing the incoming updates. But only if this rule doesn't contain conflicts (a rule with conflicts must be updated with previewing it in the flyout).

Screen.Recording.2025-01-15.at.18.57.54.mov

• User can bulk upgrade multiple prebuilt rules to their latest versions. But only those which don't contain conflicts (rules with conflicts must be updated one-by-one with previewing them in the flyout).

Screen.Recording.2025-01-15.at.19.15.53.mov

• We've added a copy for a case when Building Block property is disabled. Copy: Will not mark alerts as "building block" alerts. Please check if this needs to be reworded. Context: PR comment.

Schermopname.2025-01-18.om.11.53.24.mov

Prebuilt rule export/import workflow

• User can export a single prebuilt rule.
  • Pages: Rule Details, Rule Management
  • It can be a prebuilt non-customized or prebuilt customized rule

Screen.Recording.2025-01-15.at.19.26.48.mov

• User can bulk export multiple prebuilt rules via bulk actions.
  • Pages: Rule Management
  • We support exporting prebuilt non-customized, prebuilt customized, and custom rules in any combination

Screen.Recording.2025-01-15.at.19.34.00.mov

• User can bulk import multiple prebuilt rules.
  • Pages: Rule Management
  • We support importing prebuilt non-customized, prebuilt customized, and custom rules - in any combination

Screen.Recording.2025-01-15.at.19.38.37.mov

Licensing restrictions

TBD. Details will be added by @xcrzx.

[approksiu]

Good to add a note on upgrading prebuilt rules to the newest versions before customizing them - to avoid having to manually solve a lot of rule fields update conflicts.

[nastasha-solomon]

Earlier today, I met with a few Rules Management folks to discuss the following UI and copy changes that'll be introduced in the coming weeks:

• The upgrade flow for prebuilt rules without base versions is changing. See [Security Solution] Relax the rules of handling missing base versions of prebuilt rules kibana#210358 for background and more info.
• The license reqs for importing/exporting rules in ESS and Serverless are becoming more granular. See https://github.com/elastic/security-team/issues/11502 for updated reqs.

For now, I plan to tackle docs for customizing prebuilt rules and importing/exporting modified prebuilt rules first. I'll also start docs for the unaffected rule upgrade flows since those are finalized. The docs for updating prebuilt rules without base versions will come last and will likely be completed after the functionality has been fully added.

cc: @banderror @dplumlee @nikitaindik