[CloudSecurity][Fleet] Add CloudFormation install method to CSPM #159994

opauloh · 2023-06-20T11:35:07Z

Summary

it closes #157761

This PR adds the CloudFormation install method to the CSPM Cloud Security integration. The following changes were made:

Cloud Security changes

Added option to select between CloudFormation or Manual setup format.
Converted EuiRadioGroup to Select for the Preferred manual method
Fixed a bug where the Confirmation Modal was dismissing after saving the integration.

Updates

Added aws.credentials.type : cloud_formation for better backwards compatibility
Fixed integration upgrade issues
Fixed integration editing issues
Moved CSPM AWS and KSPM EKS into their own separate files
Handling CloudFormation not supported use cases
Fixed broken links

Fleet changes

Added CloudFormation post-install modal
Added SUBMITTED_CLOUD_FORMATION to the integration install form to show the CloudFormation post-install modal for CloudFormation integrations.
Reorganized CloudFormation methods and added useCreateCloudFormationUrl hook.

Screenshots

elasticmachine · 2023-06-20T11:35:11Z

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

elasticmachine · 2023-06-20T11:35:14Z

Pinging @elastic/fleet (Team:Fleet)

x-pack/plugins/fleet/public/services/get_cloud_formation_template_url_from_package_policy.ts

x-pack/plugins/fleet/public/components/cloud_formation_guide.tsx

...ate_package_policy_page/single_page_layout/components/post_install_cloud_formation_modal.tsx

juliaElastic

LGTM, added a few suggestions.

JordanSh

left some suggestions mainly around simplifying readability

JordanSh · 2023-06-20T13:10:38Z

...k/plugins/cloud_security_posture/public/components/fleet_extensions/aws_credentials_form.tsx

+    const checkCurrentTemplate = newPolicy?.inputs?.find((i: any) => i.type === CLOUDBEAT_AWS)
+      ?.config?.cloud_formation_template_url?.value;
+
+    if (setupFormat !== 'cloudFormation') {
+      if (checkCurrentTemplate !== null) {
+        updateCloudFormationPolicyTemplate(newPolicy, updatePolicy, null);
+      }
+      return;
+    }


This can return undefined

const checkCurrentTemplate = newPolicy?.inputs?.find((i: any) => i.type === CLOUDBEAT_AWS) ?.config?.cloud_formation_template_url?.value;

so this:

if (checkCurrentTemplate !== null) { updateCloudFormationPolicyTemplate(newPolicy, updatePolicy, null); }

essentially checks for undefined !== null which will result in true, and updateCloudFormationPolicyTemplate will get called.

JordanSh · 2023-06-20T13:32:28Z

x-pack/plugins/cloud_security_posture/public/components/fleet_extensions/mocks.ts

+        ],
+      },
+    ],
+  } as PackageInfo;


might be better to type the return, with as we might make mistakes that won't get noticed.

x-pack/plugins/cloud_security_posture/public/common/api/use_package_policy_list.ts

x-pack/plugins/cloud_security_posture/public/components/fleet_extensions/utils.ts

JordanSh · 2023-06-22T22:28:52Z

@opauloh another issue is that we show CloudFormation option when the installed integration does not support it (1.3.0)

Screen.Recording.2023-06-22.at.21.14.16.mov

This one is important, happened to me and it's very confusing

kibana-ci · 2023-06-26T15:56:36Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`cloudSecurityPosture`	201	204	+3
`fleet`	819	823	+4
total			+7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`cloudSecurityPosture`	218.9KB	229.0KB	+10.0KB
`fleet`	974.1KB	977.5KB	+3.3KB
total			+13.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`fleet`	130.9KB	132.0KB	+1.1KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`cloudSecurityPosture`	14	16	+2
`enterpriseSearch`	14	16	+2
`securitySolution`	416	420	+4
total			+8

Total ESLint disabled count

id	before	after	diff
`cloudSecurityPosture`	15	17	+2
`enterpriseSearch`	15	17	+2
`securitySolution`	497	501	+4
total			+8

History

💔 Build #137961 failed e879af2
💔 Build #137949 failed 2b27573
💛 Build #136492 was flaky fc91a40
💔 Build #136471 failed e83f5d3

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

JordanSh

left some comments, i pulled the or and the overall flow seemed ok, i think its safe to merge so we can start testing it.

also, I don't think it's considered a blocker but I didn't have a fleet on my local env so I installed one locally. the installation was successful and I could follow it up by adding an agent to my integration (directly from fleet installation flyout). but when i clicked on add agent, I got the wrong form:

After that, closing and reopening the flyout did open the correct form with the cloudformation option. so it seems it's only a problem when coming from the fleet flyout, and we can push a separate fix for it.
cc: @kfirpeled @nick-alayil

JordanSh · 2023-06-26T11:09:51Z

...ity_posture/public/components/fleet_extensions/aws_credentials_form/aws_credentials_form.tsx

+import { SetupFormat, useAwsCredentialsForm } from './hooks';
+
+interface AWSSetupInfoContentProps {
+  integrationLink: string;


can we rename this? the current name suggest its a link to an integration

JordanSh · 2023-06-26T14:26:43Z

...ity_posture/public/components/fleet_extensions/aws_credentials_form/aws_credentials_form.tsx

+  input: Extract<NewPackagePolicyPostureInput, { type: 'cloudbeat/cis_aws' }>;
+  updatePolicy(updatedPolicy: NewPackagePolicy): void;
+  packageInfo: PackageInfo;
+  onChange: any;


missing type

JordanSh · 2023-06-26T14:29:29Z

x-pack/plugins/cloud_security_posture/public/components/fleet_extensions/utils.test.ts

@@ -15,7 +15,7 @@ describe('getPosturePolicy', () => {
    ['cloudbeat/cis_k8s', getMockPolicyK8s, null],
  ] as const) {
    it(`updates package policy with hidden vars for ${name}`, () => {
-      const inputVars = getPostureInputHiddenVars(name);
+      const inputVars = getPostureInputHiddenVars(name, {} as any);


as PackageInfo is preferable

JordanSh · 2023-06-26T16:03:20Z

...gins/cloud_security_posture/public/components/fleet_extensions/aws_credentials_form/hooks.ts

+  const template: string | undefined = newPolicy?.inputs?.find((i) => i.type === CLOUDBEAT_AWS)
+    ?.config?.cloud_formation_template_url?.value;
+
+  return template || undefined;


if template can either be string or undefined, || undefined is redundant

JordanSh · 2023-06-26T16:03:37Z

...gins/cloud_security_posture/public/components/fleet_extensions/aws_credentials_form/hooks.ts

+  newPolicy: NewPackagePolicy;
+  input: Extract<NewPackagePolicyPostureInput, { type: 'cloudbeat/cis_aws' }>;
+  packageInfo: PackageInfo;
+  onChange: (opts: any) => void;


missing type

kfirpeled

LGTM

…stic#159994) (cherry picked from commit 5728bfa)

kibanamachine · 2023-06-26T18:30:55Z

💚 All backports created successfully

Status	Branch	Result
✅	8.9

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

#159994) (#160559) # Backport This will backport the following commits from `main` to `8.9`: - [[CloudSecurity][Fleet] Add CloudFormation install method to CSPM (#159994)](#159994)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Paulo Henrique <paulo.henrique@elastic.co>

opauloh added 12 commits June 20, 2023 02:16

fix modal dismissing bug

cb31457

cloud formation for cspm

22f4cb6

fixing modal dismissing bug

5933fc0

adding packageinfo

4a99c08

getCspmCloudFormationDefaultValue

8d4e276

post install cloud formation modal

4966ac5

added SUBMITTED_CLOUD_FORMATION to integrations form

3425405

add reusable cloud formation guide

7c4ff20

updating get cloud formation methods

5643ebb

add useCreateCloudFormationUrl hook

bf9accd

Merge branch 'main' into cspm/add-cloud-formation

9d01907

updating tests

e83f5d3

opauloh added release_note:feature Makes this part of the condensed release notes Team:Cloud Security Cloud Security team related v8.9.0 labels Jun 20, 2023

opauloh requested review from a team as code owners June 20, 2023 11:35

botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Jun 20, 2023

opauloh requested a review from JordanSh June 20, 2023 11:36

juliaElastic reviewed Jun 20, 2023

View reviewed changes

x-pack/plugins/fleet/public/services/get_cloud_formation_template_url_from_package_policy.ts Show resolved Hide resolved

juliaElastic reviewed Jun 20, 2023

View reviewed changes

x-pack/plugins/fleet/public/components/cloud_formation_guide.tsx Outdated Show resolved Hide resolved

juliaElastic reviewed Jun 20, 2023

View reviewed changes

...ate_package_policy_page/single_page_layout/components/post_install_cloud_formation_modal.tsx Outdated Show resolved Hide resolved

juliaElastic approved these changes Jun 20, 2023

View reviewed changes

opauloh added 4 commits June 20, 2023 05:11

fixing translations

615e504

fixing condition

372729c

fixing typo

16e56e9

adding unit tests

fc91a40

JordanSh previously approved these changes Jun 20, 2023

View reviewed changes

opauloh added the ci:cloud-deploy Create or update a Cloud deployment label Jun 22, 2023

opauloh added 10 commits June 24, 2023 16:21

move eks to it own form

8f8aa46

update hook parameters

a2c7e66

updating launch cloud formation texts

91b25d5

updating utils

fcdf7c3

adding eks form to template selector

d17aef4

handling validation state in form

edaebe2

splitting files

6bb7d8e

updating tests

2acb170

Merge branch 'main' into cspm/add-cloud-formation

2b27573

CI fix: types

e879af2

opauloh requested review from JordanSh and kfirpeled June 26, 2023 07:43

Ci fix: exporting type

394e932

JordanSh approved these changes Jun 26, 2023

View reviewed changes

kfirpeled approved these changes Jun 26, 2023

View reviewed changes

opauloh added the backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) label Jun 26, 2023

opauloh merged commit 5728bfa into elastic:main Jun 26, 2023

kibanamachine added the v8.10.0 label Jun 26, 2023

kibanamachine mentioned this pull request Jun 26, 2023

[8.9] [CloudSecurity][Fleet] Add CloudFormation install method to CSPM (#159994) #160559

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jun 26, 2023

[CloudSecurity][Fleet] Add CloudFormation install method to CSPM (ela…

547cac3

…stic#159994) (cherry picked from commit 5728bfa)

opauloh linked an issue Jun 26, 2023 that may be closed by this pull request

[Cloud Security] [BUG] [CNVM] Edit an existing CSPM integration after migrating to CSPM 1.4.* - shows current mode is CloudFormation #160143

Closed

JordanSh mentioned this pull request Jul 6, 2023

[Cloud Security] Add AWS Organization form #161345

Closed

JordanSh mentioned this pull request Jul 26, 2023

[Cloud Security] AWS Organization form #162571

Merged

9 tasks

JordanSh mentioned this pull request Sep 12, 2023

[Cloud Security][CIS Azure] Azure integration form - ARM Template flow #165580

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CloudSecurity][Fleet] Add CloudFormation install method to CSPM #159994

[CloudSecurity][Fleet] Add CloudFormation install method to CSPM #159994

opauloh commented Jun 20, 2023 •

edited by kibanamachine

Loading

elasticmachine commented Jun 20, 2023

elasticmachine commented Jun 20, 2023

juliaElastic left a comment

JordanSh left a comment

JordanSh Jun 20, 2023

JordanSh Jun 20, 2023

JordanSh commented Jun 22, 2023

kibana-ci commented Jun 26, 2023 •

edited

Loading

ESLint disabled line counts

Total ESLint disabled count

JordanSh left a comment

JordanSh Jun 26, 2023

JordanSh Jun 26, 2023

JordanSh Jun 26, 2023

JordanSh Jun 26, 2023

JordanSh Jun 26, 2023

kfirpeled left a comment

kibanamachine commented Jun 26, 2023

[CloudSecurity][Fleet] Add CloudFormation install method to CSPM #159994

[CloudSecurity][Fleet] Add CloudFormation install method to CSPM #159994

Conversation

opauloh commented Jun 20, 2023 • edited by kibanamachine Loading

Summary

Cloud Security changes

Fleet changes

Screenshots

elasticmachine commented Jun 20, 2023

elasticmachine commented Jun 20, 2023

juliaElastic left a comment

Choose a reason for hiding this comment

JordanSh left a comment

Choose a reason for hiding this comment

JordanSh Jun 20, 2023

Choose a reason for hiding this comment

JordanSh Jun 20, 2023

Choose a reason for hiding this comment

JordanSh commented Jun 22, 2023

kibana-ci commented Jun 26, 2023 • edited Loading

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

ESLint disabled line counts

Total ESLint disabled count

History

JordanSh left a comment

Choose a reason for hiding this comment

JordanSh Jun 26, 2023

Choose a reason for hiding this comment

JordanSh Jun 26, 2023

Choose a reason for hiding this comment

JordanSh Jun 26, 2023

Choose a reason for hiding this comment

JordanSh Jun 26, 2023

Choose a reason for hiding this comment

JordanSh Jun 26, 2023

Choose a reason for hiding this comment

kfirpeled left a comment

Choose a reason for hiding this comment

kibanamachine commented Jun 26, 2023

💚 All backports created successfully

Questions ?

opauloh commented Jun 20, 2023 •

edited by kibanamachine

Loading

kibana-ci commented Jun 26, 2023 •

edited

Loading