Look into rewriting system Dashboards

[fearful-symmetry]

So, I recently fixed an issue where the CPU usage Gauges in the system dashboards were using painless to normalize CPU usage metrics, as opposed to using norm.pct, presumably because the visualization predated the normalized metrics. In addition, both myself and @ruflin have run into issues with the TSVB visualizations zeroing out if you set the time interval too low.

I'm starting to wonder if we should just rewrite the system dashboards entirely. They're clearly very old, and I'm willing to bet there's a variety of weird behavior lurking in other visualizations that are part of the dashboards. This would also give us a chance to re-write the visualizations in lens. What do people think? Is it worth re-writing them entirely?

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[ruflin]

++ on rewriting the dashboards. It is not just about using Lens but also using new linking features in Kibana and more. The new system dashboards can serve as a model on how future dashboards should be done.

[exekias]

+1 to this, @fearful-symmetry are you looking into to also change the data that this is shown or this is mostly about trying to refactor it into lens, while trying to keep the same graphs?

also cc @sorantis

[sorantis]

+1 on giving the system dashboards a refresh. While they show relevant information they're lacking prebuilt filters (e.g. host selection). Also, I think it'll be a good opportunity to combine logs and metrics dashboards, at the very least by linking them together. We've compiled the dashboard creation guidelines that could aid this work.

[fearful-symmetry]

are you looking into to also change the data that this is shown or this is mostly about trying to refactor it into lens, while trying to keep the same graphs?

Considering that the old dashboards were deriving their own norm.pct values, I'm guessing there's other instances of data sources we'd rather change, or other things we should add. @exekias

[philippkahr]

+1 for refactoring the dashboards. Maybe we should look into rewriting all dashboards? E.g. the NetFlow ones are broken.

[fearful-symmetry]

Yah, I get the feeling that there's a lot of dashboard neglect going on, since it's kind of an easy area to neglect.

[fearful-symmetry]

@belimawr If you're still looking to ramp up work on system stuff, this would be a really good area, since it should be any easy win.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[ruflin]

Currently the team around @flash1293 is also looking into the system dashboard in context of moving to Lens. Worth syncing up with them in case this is touched.

[philippkahr]

Also please set the correct filter for the data_stream.dataset. Sometimes the system dashboards just crawl over the entire old metricbeat, filebeat indices and that can lead to really slow load times and not a pleasant UX.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!