[Logs Explorer] Support logs data views

[tonyghiani]

📓 Summary

Closes #175767

🧪 You can access a live deployment of this PR here.

There has been a lot of talking around supporting selection for data views and staying on the Logs Explorer for those concerning logs data streams.

This work supports selecting and exploring Discover data views on Logs Explorer. This is currently limited to logs data views.

Screen.Recording.2024-02-01.at.16.19.03.mov

Next steps

We had already an offline conversation with the team about how naming the selector, selection modes and related entities is becoming inconsistent and more difficult to maintain.
To keep this PR narrowed to the data views support, an upcoming PR will focus on renaming according to the new selector's responsibilities.

Core changes

DataViewDescriptor

The DataViewDescriptor instance is a way to describe a data view in the context of Logs Explorer. This does not represent a DataView object complete of fields and all the details provided with a DataView instance, but it instead encapsulates the logic around identifying what data type a data view is about, as well as defining the logic to use it with the new dataView selection mode.

It creates a new instance starting from a DataViewListItem object, which is a minimal object provided by the dataViews service to list existing data views.

LogExplorerController

The LogExplorerController state machine now handles the selected entry depending on its type, triggering different flows. There are 3 different journeys depending on the selected entry:

• For a data view entry which is not about logs, the page redirects to Discover with the data view selected
• For a data view entry about logs, the data loads in the Logs Explorer, switching to the persisted DataView.
• For a dataset entry, an ad-hoc data view loads in the Logs Explorer.

To avoid updating twice the data view (once during initialization, and immediately after during selection validation), the validation flow has been anticipated and restructured to follow different flows, depending on the selection type.

Dataset selector

The selector state machine unifies the selection handler and expands the selection modes, adding a new dataView mode which handles logs data view selections.

[apmmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• /oblt-deploy-serverless : Deploy a serverless Kibana instance using the Observability test environments.
• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[tonyghiani]

note: this property will be useful to differentiate between different data views and group them by its value.

[tonyghiani]

note: Starting from a single base pattern such as logs, metrics, etc.., this function allows to create a regular expression to test an index pattern for:

• word boundaries and subsequent paths (logs-, logs-system.syslog-default, etc...)
• local and remote clusters (logs-, remote_cluster:logs-*, etc...)
• multiple patterns, supporting above features (logs-*,remote_cluster:logs-*, etc...)

[tonyghiani]

note: this action fetches and sets the complete data view object by its ID using the Discover pre-existing logic.

[tonyghiani]

/oblt-deploy

[weltenwort]

Very solid work overall, I had a good time reading it! 👏 The changes to the state machines make them more consistent. It's also good to see how the discover plugin was removed as a dependency throughout the selector hierarchy. This will make its re-use in other locations easier.

I left just a few questions and suggestions below.

And thanks for updating the storybook, which was very helpful in testing the loading and error states.

[weltenwort]

The way the action is implemented (and how the selector acted before) would cause the page-wide side-effect of navigating to a different URL. Is that desirable when the logs explorer is used in an embedded fashion? Should we delegate that to the consumer?

If so, how do we do that? 🤔 Two options come to mind:

• a customization that customizes the action that happens here (like the one used for the ai assistant)
• emitting an event on the (yet unused) event$ observable that the consumer can react to

What do you think?

[tonyghiani]

I see your point here and both the proposed solutions LGTM.
On the other hand, what do you think should be the fallback behaviour when the consumer doesn't apply the customization or doesn't react to the event?

My concern is we might end up with a broken UX when the user clicks on a non-logs data view if the consumer embedding the LogsExplorer doesn't react to the emitted event in $event (assuming we go with the second option).

One option to keep the default behaviour without forcing the user to react to the emitted event, but still provide the flexibility to perform custom logic and prevent the default behaviour if necessary could be to provide something like a preventDefault() method as part of the emitted event, in a browser-fashion way.

What I mean is, in case we'll extend the set of events emitted by the controller, to have something like this:

export type LogsExplorerPublicEvent =
  | {
      type: 'REDIRECT_TO_DISCOVER';
      data: { context, eventPayload };
      preventDefault: () => void;
    }
  | {
      type: 'ANOTHER_EVENT';
      data: { ... };
      preventDefault: () => void;
    }
  | ...

This is just one possible approach, but it might be more convoluted than other options, what do you think?
Or did I misunderstand your suggestion and you meant something different? 😅

[tonyghiani]

Mmm I've been thinking more about the approach I proposed for preventing the default behaviour as we might have more subscribers on the same emitted event, which makes it impractical to decide whether the default behaviour should be skipped or not...
Adding a customization point would work, but it still leaves the discussion open about what happens if the consumer doesn't register the customization.

[tonyghiani]

Based on a offline conversation with Felix, we'll implement a customization point to delegate the redirect to the consumer in a shorted upcoming pull request.

[weltenwort]

Similarly, should this side-effect happen here? Or should it be delegated to the consumer too?

[tonyghiani]

The same work mentioned here will apply to this part.

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 4adaf96
• Storybooks Preview

Failed CI Steps

• Jest Integration Tests #6

Test Failures

• [job] [logs] Jest Integration Tests #6 / fake elasticsearch should fail to start Kibana because of the Product Check Error
• [job] [logs] Jest Integration Tests #6 / fake elasticsearch should return unknown product when it cannot perform the Product check (503 response)

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
logsExplorer	672	677	+5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
logsExplorer	1002.9KB	1005.8KB	+2.9KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
logsExplorer	19	20	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
logsExplorer	49.0KB	51.8KB	+2.8KB

History

• 💚 Build #192855 succeeded f8d903f
• 💔 Build #192852 failed ded1e99
• 💚 Build #192791 succeeded 11332ee
• 💔 Build #192521 failed cf5d8e9
• 💔 Build #192329 failed e90e1f0
• 💔 Build #192310 failed af509fc

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[achyutjhunjhunwala]

Really loved the explanatory way of creating the regex 💯