[Reporting/Cloud] Soft disable reporting functionality on 1GB Kibanas #130651

jloleysens · 2022-04-20T09:48:25Z

Summary

Initial work for soft-disabling rather than crashing Kibana when generating reports on Kibana instances with low available RAM. This work ensures that Kibana server responds with the appropriate messages via our reporting HTTP APIs when generating a report.

How to test

Start Kibana & ES
Enable a trial license
Either test on 1GB Cloud deployment (see this PR [Reporting] Soft disable cloud test - do not merge #130846) or, if testing locally replace if (this.systemHasInsufficientMemory()) { with if (true) { in screenshotting/screenshots/index.ts to trigger that code path.
Generate a PDF or PNG report
See the error message in the UI
See the error message in the report info panel flyout that contains a link

Screenshots

Implementation details and notes

The InsufficientMemoryAvailableOnCloudError will be thrown on Cloud when we are running on a Kibana with <2GB RAM. Specifically, the error is thrown when we call getScreenshots exposed by the screenshotting plugin. In this way we block all screenshots from running.
Reporting now knows about this error type and sets an appropriate message explaining what is wrong. Additionally, the UI shows a message with a link, guiding users to how the can get around the issue
For ReportingError classes it is convenient to read this.code and ClassName.code I've opted to make code both an instance and static property name
In collaboration with Cloud stakeholders, we have decided that reading os.totalmem is sufficient for informing the soft disable decision
No new methods were added to screenhotting's API, although we may have to consider this if we want to pre-emptively disable the "generate report" button

Related to

#129989

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Does not stop from generating reports on Cloud	Low	High	Must be tested manually on Cloud

For maintainers

This was checked for breaking API changes and was labeled appropriately

…nstead of sync

…disable-server-side * 'main' of github.com:elastic/kibana: (35 commits) [Uptime] remove latency limit warnings when using monitor management (elastic#129597) [Security Solution] [ReponseOps] Executes Cases Cypress test when there is a change on cases plugin (elastic#129992) Paramaterized Discover tests (elastic#129684) [Security Solution][Investigations] - Minor bug fixes (elastic#130054) [DOCS} Adds technical preview to Lens annotations (elastic#130058) [Security solution] [Endpoint] Revisit blocklist wrong labels (elastic#128773) [Security Solutions] Adds API docs for value lists (elastic#129962) [CI] Move jest tests to spot instances, and fix spot retries in PRs (elastic#130045) chore(NA): upgrades rules_node_js to v5.4.0 (elastic#130051) [SecuritySolution] Remove the cell hovers actions for agent status (elastic#130042) Upgrade RxJS to 7 (elastic#129087) [SecuritySolution] Clean up CaseContext (elastic#130036) Revert "chore(NA): upgrades rules_node_js to v5.4.0 (elastic#130021)" Use RuleDataReader to query for threshold signal history (elastic#129763) Remove securityRulesCancelEnabled setting and set shorter default timeouts (elastic#129769) Upgrade EUI to v54.0.0 (elastic#129653) [Security Solution] More Ransomware exceptionable fields (elastic#130039) Add e2e for the apm integration policy form (elastic#129860) chore(NA): upgrades rules_node_js to v5.4.0 (elastic#130021) [ML] Fix Single Metric Viewer chart failing to load if no points during calendar event (elastic#130000) ... # Conflicts: # x-pack/plugins/screenshotting/server/screenshots/index.test.ts

…disable-server-side * 'main' of github.com:elastic/kibana: (103 commits) [Osquery] Update eslint config (elastic#129637) [Uptime] Update monitor saved object mappings (elastic#130433) Add links to metricbeat module docs (elastic#130519) Add link to troubleshooting guide in confirm data copy (elastic#130420) [Step 3] Cleanup charts plugin (elastic#130132) [Visualize] Adds a deprecation warning to the pie app (elastic#130447) [Maps] fix vector tile load errors not displayed in legend (elastic#130395) [CI] Split alerting-api-integration tests into separate cigroups (elastic#130414) [CI] Use spot instances for default cigroups in PR CI (elastic#130476) [functional-tests] TimePicker optimizations (elastic#130200) [kbn/pm] use stable module ids in dist (elastic#130497) [8.2.1][Security Solution][Session view] fix full screen session view margin (elastic#130496) Fix wrong config in comments (elastic#130378) Add deprecated telemetry (elastic#130458) Add eslint rule to support breaking up packages (elastic#130483) [Security Solution][Endpoint] Fix test stability and un-skip flaky tests (elastic#130176) Update object types for SharePoint Online external connector (elastic#130478) [Workplace Search] Fix broken feedback link (elastic#130475) Rename the term "execution" in config to "run" (elastic#130172) [Cloud Posture] use index with keyword mapping (elastic#130456) ... # Conflicts: # docs/user/reporting/index.asciidoc # x-pack/plugins/reporting/public/types.ts # x-pack/plugins/screenshotting/server/screenshots/index.test.ts # x-pack/plugins/screenshotting/server/screenshots/index.ts

…-fix'

jloleysens · 2022-04-25T13:30:33Z

@elasticmachine merge upstream

tsullivan · 2022-04-26T21:05:26Z

x-pack/plugins/reporting/public/management/components/report_info_flyout_content.tsx

+     * should only occur on cloud.
+     */
+    info.error_code === VisualReportingSoftDisabledError.code
+      ? sharedI18nTexts.cloud.insufficientMemoryError(


Any reason to not make this message part of the Job.getError implementation?

That's a good point... But I'd prefer to not raise this logic to a more general level yet. Because the i18n contains JSX we would no longer guarantee that Job.getError will only return a undefined | string.

I prefer having this code implemented in a more "dumb" way because it remains a bit easier to delete (something may still want to do one day).

tsullivan

LGTM! I just left one comment/suggestion.

tsullivan · 2022-04-26T21:25:01Z

x-pack/plugins/reporting/public/notifier/job_failure.tsx

-        >
-          {errorText}
+        <EuiCallOut size="m" color="danger" data-test-errorText={errorText}>
+          {job.errorCode === errors.VisualReportingSoftDisabledError.code


Is it possible to move this logic to ReportingAPIClient.getError, where the passed-in errorText comes from?

gchaps

doclink LGTM

One small nit on the message. I think we can remove the word "minimum" in this sentence, so it reads:

Check the RAM requirements.

jloleysens · 2022-04-28T07:24:13Z

@elasticmachine merge upstream

kibana-ci · 2022-04-28T08:56:12Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`reporting`	83	85	+2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/doc-links`	64	65	+1
`cloud`	23	24	+1
total			+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`reporting`	57.9KB	58.1KB	+220.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	288.6KB	288.7KB	+115.0B
`reporting`	38.8KB	42.0KB	+3.3KB
`screenshotting`	7.7KB	7.7KB	+6.0B
total			+3.4KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/doc-links`	64	65	+1
`cloud`	28	29	+1
total			+2

History

💚 Build #40208 succeeded 60d09e7
💚 Build #40097 succeeded acf924c
💚 Build #40041 succeeded 42e48f2
💔 Build #40034 failed fe35f0c
💔 Build #40030 failed 866baa6
💔 Build #40021 failed ca7d907

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2022-04-28T11:53:37Z

💔 All backports failed

Status	Branch	Result
❌	8.2	Backport failed because of merge conflicts You might need to backport the following PRs to 8.2: - [Reporting] Fix copy issue in report notifications (#123607) - [Reporting] Reporting info panel touch ups (#120617)

Manual backport

To create the backport manually run:

node scripts/backport --pr 130651

Questions ?

Please refer to the Backport tool documentation

kibanamachine · 2022-05-02T11:56:18Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 130651 or prevent reminders by adding the backport:skip label.

kibanamachine · 2022-05-03T12:06:18Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 130651 or prevent reminders by adding the backport:skip label.

…elastic#130651) * added callout about reporting on cloud * slight rewording * added functionality to disbale screenshotting on <2GB cloud instances * added test and fixed public facing API to throw an observable error instead of sync * update reporting integration with new screenshotting error * fix typo * added specific error code to telemetry * added positive test case * updated mappings * update jest test snapshots * update snapshot * fix use of deprecated RxJS * [revert this] added some logs for cloud * also check for deploymentId * remove logs * removed logger being passed to system check function * remove unused import * slight update to test * added cloud min requirements link * add link to UI * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * added todo comment * read mem limit from cgroup files * update jest * tidy up and simplify some of the logic * [revert this] test with 1024 * [revert this] added some logging again * actually pass the cloud plugin to Screenshots 🤦 * [revert this] try and read instance data file instead * Revert "[revert this] try and read instance data file instead" This reverts commit ebdea21. * Revert "[revert this] test with 1024" This reverts commit 7ae3e39. * Revert "[revert this] added some logging again" This reverts commit d21e808. * use injected environment variable to read instance size * implement copy feedback * fix variable rename * fix test * fixed some copy * fix test code * remove unused i18n * update snapshots * minor tidying up * tighten up the copy Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

jloleysens and others added 30 commits April 12, 2022 11:22

added callout about reporting on cloud

a07bee0

slight rewording

e9a4c0c

added functionality to disbale screenshotting on <2GB cloud instances

9eb4995

added test and fixed public facing API to throw an observable error i…

de83c1f

…nstead of sync

update reporting integration with new screenshotting error

8b738e6

fix typo

d8cbaf1

added specific error code to telemetry

43e19dc

added positive test case

1e04b41

updated mappings

3a5e1fe

update jest test snapshots

0ca062b

update snapshot

a0cfd75

fix use of deprecated RxJS

1027ed1

[revert this] added some logs for cloud

3878865

also check for deploymentId

67f3a91

remove logs

9f5f60c

Merge branch 'main' into reporting/soft-disable-server-side

f6f5d46

removed logger being passed to system check function

03dbd84

remove unused import

43cdce7

slight update to test

83ebe1a

added cloud min requirements link

1b7c5fb

add link to UI

cd13819

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

c642918

…-fix'

added todo comment

a9f37f3

Merge branch 'main' into reporting/soft-disable-server-side

ed5edf4

read mem limit from cgroup files

294873c

Merge branch 'main' into reporting/soft-disable-server-side

53e4214

update jest

06f68a5

tidy up and simplify some of the logic

94ae13e

kibanamachine and others added 2 commits April 25, 2022 03:05

Merge branch 'main' into reportings/soft-disable

d45a03e

minor tidying up

acf924c

Merge branch 'main' into reportings/soft-disable

60d09e7

tsullivan reviewed Apr 26, 2022

View reviewed changes

tsullivan approved these changes Apr 26, 2022

View reviewed changes

tsullivan reviewed Apr 26, 2022

View reviewed changes

gchaps approved these changes Apr 26, 2022

View reviewed changes

kibanamachine and others added 2 commits April 28, 2022 02:24

Merge branch 'main' into reportings/soft-disable

d206f67

tighten up the copy

5eb59e1

jloleysens added the auto-backport Deprecated - use backport:version if exact versions are needed label Apr 28, 2022

jloleysens merged commit 0c5dbca into elastic:main Apr 28, 2022

jloleysens deleted the reportings/soft-disable branch April 28, 2022 15:24

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 2, 2022

jloleysens added backport:skip This commit does not require backporting and removed backport missing Added to PRs automatically when the are determined to be missing a backport. auto-backport Deprecated - use backport:version if exact versions are needed v8.2.1 labels May 4, 2022

tsullivan mentioned this pull request Jun 8, 2022

[Reporting] Can't create report from Dashboard #133876

Closed

tylersmalley added ci:cloud-deploy Create or update a Cloud deployment and removed ci:deploy-cloud labels Aug 17, 2022

tsullivan mentioned this pull request Jun 15, 2023

[Serverless] Will the cloud plugin be configured? #159390

Closed

Dosant mentioned this pull request Mar 27, 2024

[Security Solution]Share PDF report for dashboard not generating #177856

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reporting/Cloud] Soft disable reporting functionality on 1GB Kibanas #130651

[Reporting/Cloud] Soft disable reporting functionality on 1GB Kibanas #130651

jloleysens commented Apr 20, 2022 •

edited

Loading

jloleysens commented Apr 25, 2022

tsullivan Apr 26, 2022

jloleysens Apr 28, 2022

tsullivan left a comment

tsullivan Apr 26, 2022 •

edited

Loading

gchaps left a comment

jloleysens commented Apr 28, 2022

kibana-ci commented Apr 28, 2022

API count

kibanamachine commented Apr 28, 2022

kibanamachine commented May 2, 2022

kibanamachine commented May 3, 2022

[Reporting/Cloud] Soft disable reporting functionality on 1GB Kibanas #130651

[Reporting/Cloud] Soft disable reporting functionality on 1GB Kibanas #130651

Conversation

jloleysens commented Apr 20, 2022 • edited Loading

Summary

How to test

Screenshots

Implementation details and notes

Related to

Checklist

Risk Matrix

For maintainers

jloleysens commented Apr 25, 2022

tsullivan Apr 26, 2022

Choose a reason for hiding this comment

jloleysens Apr 28, 2022

Choose a reason for hiding this comment

tsullivan left a comment

Choose a reason for hiding this comment

tsullivan Apr 26, 2022 • edited Loading

Choose a reason for hiding this comment

gchaps left a comment

Choose a reason for hiding this comment

jloleysens commented Apr 28, 2022

kibana-ci commented Apr 28, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

History

kibanamachine commented Apr 28, 2022

💔 All backports failed

Manual backport

Questions ?

kibanamachine commented May 2, 2022

kibanamachine commented May 3, 2022

jloleysens commented Apr 20, 2022 •

edited

Loading

tsullivan Apr 26, 2022 •

edited

Loading