Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[App Search] Allow for query parameter to indicate ingestion mechanism for new engines #115188

Merged
merged 3 commits into from
Oct 18, 2021
Merged

[App Search] Allow for query parameter to indicate ingestion mechanism for new engines #115188

merged 3 commits into from
Oct 18, 2021

Conversation

orhantoy
Copy link
Contributor

Summary

This PR adds support for a URL param (?method="<method>") to pass along the user's intent once an engine is created and named.

  • Web crawler: /app/enterprise_search/app_search/engines/new?method=crawler
    Redirects the user to the web crawler view

  • Upload JSON: /app/enterprise_search/app_search/engines/new?method=json
    Opens the upload JSON file flyout

  • Use API: /app/enterprise_search/app_search/engines/new?method=api
    Opens the API flyout

After the user makes it past the engine naming phase, we then show the correct ingestion method based on the passed-in URL param.

Checklist

Delete any items that are not applicable to this PR.

@orhantoy orhantoy added Feature:Plugins release_note:skip Skip the PR/issue when compiling release notes auto-backport Deprecated - use backport:version if exact versions are needed v7.16.0 labels Oct 15, 2021
@@ -39,8 +43,16 @@ import {
import { EngineCreationLogic } from './engine_creation_logic';

export const EngineCreation: React.FC = () => {
const { search } = useLocation() as Location;
const { method } = parseQueryParams(search) as unknown as { method: string };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be const { method } = parseQueryParams(search) as { method?: string }; since we don't know if method will be there or not

@orhantoy orhantoy marked this pull request as ready for review October 15, 2021 16:09
@orhantoy orhantoy requested a review from a team October 15, 2021 16:09
@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/ml/jobs/categorization_field_examples·ts.apis Machine Learning jobs Categorization example endpoint - partially valid, more than 75% are null

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]     │
[00:00:00]       └-: apis
[00:00:00]         └-> "before all" hook in "apis"
[00:10:44]         └-: Machine Learning
[00:10:44]           └-> "before all" hook in "Machine Learning"
[00:10:44]           └-> "before all" hook in "Machine Learning"
[00:10:44]             │ debg creating role ft_ml_source
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_source]
[00:10:44]             │ debg creating role ft_ml_source_readonly
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_source_readonly]
[00:10:44]             │ debg creating role ft_ml_dest
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_dest]
[00:10:44]             │ debg creating role ft_ml_dest_readonly
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_dest_readonly]
[00:10:44]             │ debg creating role ft_ml_ui_extras
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_ml_ui_extras]
[00:10:44]             │ debg creating role ft_default_space_ml_all
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space_ml_all]
[00:10:44]             │ debg creating role ft_default_space1_ml_all
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space1_ml_all]
[00:10:44]             │ debg creating role ft_all_spaces_ml_all
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_all_spaces_ml_all]
[00:10:44]             │ debg creating role ft_default_space_ml_read
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space_ml_read]
[00:10:44]             │ debg creating role ft_default_space1_ml_read
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space1_ml_read]
[00:10:44]             │ debg creating role ft_all_spaces_ml_read
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_all_spaces_ml_read]
[00:10:44]             │ debg creating role ft_default_space_ml_none
[00:10:44]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [ft_default_space_ml_none]
[00:10:44]             │ debg creating user ft_ml_poweruser
[00:10:44]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser]
[00:10:44]             │ debg created user ft_ml_poweruser
[00:10:44]             │ debg creating user ft_ml_poweruser_spaces
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser_spaces]
[00:10:45]             │ debg created user ft_ml_poweruser_spaces
[00:10:45]             │ debg creating user ft_ml_poweruser_space1
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser_space1]
[00:10:45]             │ debg created user ft_ml_poweruser_space1
[00:10:45]             │ debg creating user ft_ml_poweruser_all_spaces
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_poweruser_all_spaces]
[00:10:45]             │ debg created user ft_ml_poweruser_all_spaces
[00:10:45]             │ debg creating user ft_ml_viewer
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer]
[00:10:45]             │ debg created user ft_ml_viewer
[00:10:45]             │ debg creating user ft_ml_viewer_spaces
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer_spaces]
[00:10:45]             │ debg created user ft_ml_viewer_spaces
[00:10:45]             │ debg creating user ft_ml_viewer_space1
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer_space1]
[00:10:45]             │ debg created user ft_ml_viewer_space1
[00:10:45]             │ debg creating user ft_ml_viewer_all_spaces
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_viewer_all_spaces]
[00:10:45]             │ debg created user ft_ml_viewer_all_spaces
[00:10:45]             │ debg creating user ft_ml_unauthorized
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_unauthorized]
[00:10:45]             │ debg created user ft_ml_unauthorized
[00:10:45]             │ debg creating user ft_ml_unauthorized_spaces
[00:10:45]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [ft_ml_unauthorized_spaces]
[00:10:45]             │ debg created user ft_ml_unauthorized_spaces
[00:15:19]           └-: jobs
[00:15:19]             └-> "before all" hook in "jobs"
[00:15:19]             └-: Categorization example endpoint - 
[00:15:19]               └-> "before all" hook for "valid with good number of tokens"
[00:15:19]               └-> "before all" hook for "valid with good number of tokens"
[00:15:19]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Loading "mappings.json"
[00:15:19]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Loading "data.json.gz"
[00:15:19]                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [ft_categorization] creating index, cause [api], templates [], shards [1]/[0]
[00:15:19]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Created index "ft_categorization"
[00:15:19]                 │ debg [x-pack/test/functional/es_archives/ml/categorization] "ft_categorization" settings {"index":{"number_of_replicas":"0","number_of_shards":"1"}}
[00:15:20]                 │ info [x-pack/test/functional/es_archives/ml/categorization] Indexed 1501 docs into "ft_categorization"
[00:15:20]                 │ debg applying update to kibana config: {"dateFormat:tz":"UTC"}
[00:15:20]               └-> valid with good number of tokens
[00:15:20]                 └-> "before each" hook: global before each for "valid with good number of tokens"
[00:15:21]                 └- ✓ pass  (125ms)
[00:15:21]               └-> invalid, too many tokens.
[00:15:21]                 └-> "before each" hook: global before each for "invalid, too many tokens."
[00:15:21]                 │ info [r.suppressed] [node-01] path: /_analyze, params: {}
[00:15:21]                 │      org.elasticsearch.transport.RemoteTransportException: [node-01][127.0.0.1:63231][indices:admin/analyze[s]]
[00:15:21]                 │      Caused by: java.lang.IllegalStateException: The number of tokens produced by calling _analyze has exceeded the allowed maximum of [10000]. This limit can be set by changing the [index.analyze.max_token_count] index level setting.
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction$TokenCounter.increment(TransportAnalyzeAction.java:397) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.simpleAnalyze(TransportAnalyzeAction.java:229) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.analyze(TransportAnalyzeAction.java:204) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.analyze(TransportAnalyzeAction.java:122) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.shardOperation(TransportAnalyzeAction.java:110) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.shardOperation(TransportAnalyzeAction.java:62) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.lambda$asyncShardOperation$0(TransportSingleShardAction.java:99) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:47) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
[00:15:21]                 │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
[00:15:21]                 │      	at java.lang.Thread.run(Thread.java:833) [?:?]
[00:15:21]                 │ info [r.suppressed] [node-01] path: /_analyze, params: {}
[00:15:21]                 │      org.elasticsearch.transport.RemoteTransportException: [node-01][127.0.0.1:63231][indices:admin/analyze[s]]
[00:15:21]                 │      Caused by: java.lang.IllegalStateException: The number of tokens produced by calling _analyze has exceeded the allowed maximum of [10000]. This limit can be set by changing the [index.analyze.max_token_count] index level setting.
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction$TokenCounter.increment(TransportAnalyzeAction.java:397) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.simpleAnalyze(TransportAnalyzeAction.java:229) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.analyze(TransportAnalyzeAction.java:204) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.analyze(TransportAnalyzeAction.java:122) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.shardOperation(TransportAnalyzeAction.java:110) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.admin.indices.analyze.TransportAnalyzeAction.shardOperation(TransportAnalyzeAction.java:62) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.lambda$asyncShardOperation$0(TransportSingleShardAction.java:99) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:47) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
[00:15:21]                 │      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
[00:15:21]                 │      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
[00:15:21]                 │      	at java.lang.Thread.run(Thread.java:833) [?:?]
[00:15:21]                 └- ✓ pass  (358ms)
[00:15:21]               └-> partially valid, more than 75% are null
[00:15:21]                 └-> "before each" hook: global before each for "partially valid, more than 75% are null"
[00:15:21]                 └- ✖ fail: apis Machine Learning jobs Categorization example endpoint -  partially valid, more than 75% are null
[00:15:21]                 │       Error: expected 249 to sort of equal 250
[00:15:21]                 │       + expected - actual
[00:15:21]                 │ 
[00:15:21]                 │       -249
[00:15:21]                 │       +250
[00:15:21]                 │       
[00:15:21]                 │       at Assertion.assert (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:15:21]                 │       at Assertion.eql (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:244:8)
[00:15:21]                 │       at Context.<anonymous> (test/api_integration/apis/ml/jobs/categorization_field_examples.ts:303:36)
[00:15:21]                 │       at runMicrotasks (<anonymous>)
[00:15:21]                 │       at processTicksAndRejections (node:internal/process/task_queues:96:5)
[00:15:21]                 │       at Object.apply (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:15:21]                 │ 
[00:15:21]                 │ 

Stack Trace

Error: expected 249 to sort of equal 250
    at Assertion.assert (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at Context.<anonymous> (test/api_integration/apis/ml/jobs/categorization_field_examples.ts:303:36)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/dev/shm/workspace/parallel/23/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16) {
  actual: '249',
  expected: '250',
  showDiff: true
}

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
enterpriseSearch 1546 1547 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
enterpriseSearch 1.4MB 1.4MB +758.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@orhantoy orhantoy merged commit 672b592 into elastic:master Oct 18, 2021
@orhantoy orhantoy deleted the app-search-direct-ingestion-method-navigation branch October 18, 2021 07:02
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 18, 2021
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Oct 18, 2021
…m for new engines (#115188) (#115322)

Co-authored-by: Orhan Toy <toyorhan@gmail.com>
jloleysens added a commit to jloleysens/kibana that referenced this pull request Oct 18, 2021
…-migrate-away-from-injected-css-js

* 'master' of github.com:elastic/kibana: (237 commits)
  [Uptime] Added uptime query inspector panel (elastic#115170)
  [Osquery] Add packs (elastic#107345)
  [App Search] Allow for query parameter to indicate ingestion mechanism for new engines (elastic#115188)
  [Alerting] Active alerts do not recover after re-enabling a rule (elastic#111671)
  skip flaky tests.  elastic#115308, elastic#115313
  [Breaking] Remove deprecated `enabled` settings from plugins. (elastic#113495)
  skip flaky suite.  elastic#107057
  skip flaky tests. elastic#89052, elastic#113418, elastic#115304
  skip flaky test. elastic#113892
  Bump node to 16.11.1 (elastic#110684)
  [Security Solution] Restores Alerts table local storage persistence and the Remove Column action (elastic#114742)
  skip flaky suite.  elastic#115130
  one line remove assert (elastic#115127)
  Fixes migration bug where I was deleting attributes (elastic#115098)
  [Security Solutions] Fixes the newer notification system throttle resets and enabling immediate execution on first detection of a signal  (elastic#114214)
  [build] Dockerfile update (elastic#115237)
  Fixes Cypress flake cypress test (elastic#115270)
  Disable APM e2e tests
  log an invalid type for SO (elastic#115175)
  [Fleet] Don't auto upgrade policies for AUTO_UPDATE packages (elastic#115199)
  ...

# Conflicts:
#	src/plugins/dashboard/public/application/dashboard_app.tsx
#	src/plugins/dashboard/public/types.ts
#	x-pack/plugins/reporting/server/lib/layouts/print_layout.ts
jloleysens added a commit to jloleysens/kibana that referenced this pull request Oct 18, 2021
…-link-to-kibana-app

* 'master' of github.com:elastic/kibana: (287 commits)
  [Security Solution][Endpoint] Change `trustedAppByPolicyEnabled` flag to `true` by default (elastic#115264)
  [APM] generator: support error events and application metrics (elastic#115311)
  [kibanaUtils] Don't import full `semver` client side (elastic#114986)
  [RAC] Link inventory alerts to the right inventory view (elastic#113553)
  [Uptime] Added uptime query inspector panel (elastic#115170)
  [Osquery] Add packs (elastic#107345)
  [App Search] Allow for query parameter to indicate ingestion mechanism for new engines (elastic#115188)
  [Alerting] Active alerts do not recover after re-enabling a rule (elastic#111671)
  skip flaky tests.  elastic#115308, elastic#115313
  [Breaking] Remove deprecated `enabled` settings from plugins. (elastic#113495)
  skip flaky suite.  elastic#107057
  skip flaky tests. elastic#89052, elastic#113418, elastic#115304
  skip flaky test. elastic#113892
  Bump node to 16.11.1 (elastic#110684)
  [Security Solution] Restores Alerts table local storage persistence and the Remove Column action (elastic#114742)
  skip flaky suite.  elastic#115130
  one line remove assert (elastic#115127)
  Fixes migration bug where I was deleting attributes (elastic#115098)
  [Security Solutions] Fixes the newer notification system throttle resets and enabling immediate execution on first detection of a signal  (elastic#114214)
  [build] Dockerfile update (elastic#115237)
  ...

# Conflicts:
#	x-pack/plugins/reporting/public/management/__snapshots__/report_listing.test.tsx.snap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed Feature:Plugins release_note:skip Skip the PR/issue when compiling release notes v7.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants