[Dataset quality] Filters for timeRange, integrations and datasetQuery

[yngrdyn]

Relates to #170242

📝 Summary

This PR introduces a set of filters for the table inside the dataset quality page. With these changes users now will be able to filter by timeRange, integrations and dataset name.
As part of this work, we had implemented the URL schema in the consumer observability log explorer, so anyone could share the current state of the page with others.

💡For Reviewers

State Machine

The changes is the state machine includes new events to handle the update of the selected integrations, selected query, selected timeRange and refreshing the data when changing the timeRange (or using the autorefresh in the timerange component).

@kbn/timerange

This package was added to handle timerange utilities, in particular it exposes two functions getDateRange and getISODateRange that helps converting from timeranges like { from: 'now-24h', to: 'now'} into their respective timestamps or ISO string dates.

✅ Testing

1. Navigate to /app/observability-logs-explorer/dataset-quality
2. The filters are displayed above the datasets tables

🎥 Demos

• Normal Scenario

Screen.Recording.2024-02-19.at.16.01.39.mov

• Auto Refresh

Screen.Recording.2024-02-19.at.16.36.45.mov

• Sharing URL

Screen.Recording.2024-02-19.at.16.39.37.mov

[apmmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• /oblt-deploy-serverless : Deploy a serverless Kibana instance using the Observability test environments.
• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mohamedhamed-ahmed]

I played around with the PR and got some questions:

• Should the time filter also apply to the Size column in the table, so that I see the size in the specified time range?
• Should we maybe allow the search bar to filter more than just the dataset name? I see the AC says searches the dataset name by default but what if I want to filter all dataset with size>20mb for example
• I am unable to select 2 datasets from different integrations together using the search bar, is there a way we can achieve this?
• Should we maybe rename the None to Uncategorized?
• Filtering and removing the filters, rerequests the table data again. Do we really need to do this? especially if the user has so many datasets this will take a long time as they search and remove the search.

[mohamedhamed-ahmed]

The page contents shift a bit when the table content changes, can we keep this fixed as it feels a bit annoying

TableColumnsResizing.mov

[yngrdyn]

Should we maybe allow the search bar to filter more than just the dataset name? I see the AC says searches the dataset name by default but what if I want to filter all dataset with size>20mb for example

I am unable to select 2 datasets from different integrations together using the search bar, is there a way we can achieve this?

We are quite limited regarding the search bar, mainly because of the DataStreams API, there we are only able to filter using a pattern

Should we maybe rename the None to Uncategorized?

This was already discussed in the figma file.

Filtering and removing the filters, rerequests the table data again. Do we really need to do this? especially if the user has so many datasets this will take a long time as they search and remove the search.

This is good point! I'll take a look on this one

[yngrdyn]

As we discussed offline, I will switch to UI filtering instead of going to our endpoint and filter using the dataStreams API, filtering in the UI will give more flexibility than just having a pattern. Users for example would be able to filter two datasets that comes from different integrations with a multi select (as they can do for example for integrations)

[mohamedhamed-ahmed]

LGTM!! Great work 🚀

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: f45313b

Failed CI Steps

• FTR Configs #62

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
datasetQuality	156	167	+11

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/timerange	-	8	+8

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
datasetQuality	118.8KB	136.3KB	+17.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
datasetQuality	29.9KB	31.4KB	+1.5KB
observabilityLogsExplorer	12.8KB	12.9KB	+151.0B
total			+1.7KB

Unknown metric groups

API count

id	before	after	diff
@kbn/timerange	-	8	+8

async chunk count

id	before	after	diff
datasetQuality	8	9	+1

ESLint disabled line counts

id	before	after	diff
datasetQuality	9	10	+1

Total ESLint disabled count

id	before	after	diff
datasetQuality	11	12	+1

History

• 💚 Build #194140 succeeded d5cb1be
• 💛 Build #193856 was flaky 6e696ae
• 💔 Build #193624 failed 8fd4bdb
• 💚 Build #193594 succeeded 66feb03
• 💔 Build #193564 failed 000986c
• 💔 Build #193528 failed 40cd1e8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @yngrdyn

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)