Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Lens Migration] Migrate visualizations to lens for ActiveMQ, Spring Boot and WebSphere Application Server #5250

Conversation

rajvi-patel-22
Copy link
Contributor

@rajvi-patel-22 rajvi-patel-22 commented Feb 13, 2023

  • Enhancement

What does this PR do?

  • Migrate visualizations to lens for following integrations:
    • ActiveMQ
    • Spring Boot
    • WebSphere Application Server

Below are the statistics for ActiveMQ Lens migration:

ActiveMQ Dashboards Before Migration   After Migration  
  Lens Visualization Lens Visualization
[Logs ActiveMQ] Log 0 2 2 0
[Logs ActiveMQ] Audit 0 2 1 1
[Metrics ActiveMQ] Broker 0 6 6 0
[Metrics ActiveMQ] Topics 0 8 8 0
[Metrics ActiveMQ] Queues 0 8 8 0
  0 26 25 1

Note: One visualization is TagCloud that can not be migrated to Lens.

Below are the statistics for Spring Boot Lens migration:

Spring Boot Dashboards Before Migration   After Migration  
  Lens Visualization Lens Visualization
[Spring Boot] Audit Events 2 0 2 0
[Spring Boot] Overview Dashboard 6 3 9 0
[Spring Boot] Access logs 2 0 2 0
[Spring Boot] Memory and Threads 5 0 5 0
  15 3 18 0

Below are the statistics for WebSphere Application Server Lens migration:

WebSphere Application Server Dashboards Before Migration   After Migration  
  Lens Visualization Lens Visualization
[Metrics WebSphere Application Server] Session Manager 4 1 4 0
[Metrics WebSphere Application Server] Servlet 3 0 3 0
[Metrics WebSphere Application Server] ThreadPool 6 1 6 0
[Metrics WebSphere Application Server] JDBC 4 1 4 0
  17 3 17 0

Note: Three visualizations are migrated to new input controls.

Checklist

  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that panels are populated with data.
  • I have verified that panels are not distorted after being migrated to lens.
  • I have updated screenshots of dashboard.
  • I have verified that data count are matching and panel aggregations are same as before.

Author's checklist

  • I have added an entry to my package's changelog.yml file.
  • I have verified an Kibana version is set to ^8.6.0.
  • Migrated panels should be removed from visualization folder.

Related issues

Known issues

  • Input Controls visualization is deprecated in Kibana and many of the dashboards contain these visualizations. Also, support for Input Controls has been stopped because of the new Input Controls. Hence, to replicate dashboard-level filters old Input Controls are replaced with new ones.
  • TagCloud visualizations are not supported in the lens. [GitHub issue]
  • Manually handle timestamp field in TSVB panels otherwise that panel will not be migrated using automation script.[GitHub issue]
  • TSVB visualization doesn’t show negative values in UI. But the lens shows both negative and positive values. [GitHub issue]
  • TSVB visualization doesn’t show values for the small intervals. But the lens provides data even for the small intervals. [GitHub issue]
  • Split chart functionality is not supported in Lens. [GitHub issue]
  • While migrating the visualizations from TSVB to the lens, the colors are getting default ones instead of the configured ones.
  • In the lens migrated metric visualizations, values are displayed in the bottom right corner. [GitHub issue]
  • Show toolbar option is not provided explicitly in Lens. Instead Download in CSV option is provided for the panels to download data. [GitHub issue]
  • Panel titles are not displayed in lens metric visualization for Kibana 8.6.x version. [GitHub issue]

@cla-checker-service
Copy link

cla-checker-service bot commented Feb 13, 2023

💚 CLA has been signed

@elasticmachine
Copy link

elasticmachine commented Feb 13, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-02-16T09:40:52.497+0000

  • Duration: 29 min 1 sec

Test stats 🧪

Test Results
Failed 0
Passed 96
Skipped 0
Total 96

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@kush-elastic
Copy link
Collaborator

/test

@elasticmachine
Copy link

elasticmachine commented Feb 14, 2023

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (8/8) 💚
Files 100.0% (8/8) 💚
Classes 100.0% (8/8) 💚
Methods 98.734% (78/79) 👎 -1.266
Lines 96.515% (720/746) 👍 2.657
Conditionals 100.0% (0/0) 💚

@rajvi-patel-22 rajvi-patel-22 marked this pull request as ready for review February 14, 2023 12:01
@rajvi-patel-22 rajvi-patel-22 requested a review from a team as a code owner February 14, 2023 12:02
@kush-elastic kush-elastic added the Team:Service-Integrations Label for the Service Integrations team label Feb 14, 2023
@lalit-satapathy
Copy link
Collaborator

Regarding: packages/activemq/img/metricbeat-activemq-queues.png dont see the colour combinations of dashboard before vs. after. Is it related to this comment. What can we do to fix it?

While migrating the visualizations from TSVB to the lens, the colors are getting default ones instead of the configured ones.

@@ -14,7 +14,7 @@ categories:
- message_queue
- infrastructure
conditions:
kibana.version: ^8.2.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the Lens Migration going to force a Kibana version upgrade of the packages. Why is this being done?

That may not be acceptable, if the packages are in the older releases.

@rajvi-patel-22
Copy link
Contributor Author

Yes @lalit-satapathy.

  • In TSVB below options are available in color palette:
    cp-tsvb-1
    cp-tsvb-2

  • In Lens below only options are available in color palette:
    cp-1
    cp-2

In gradient color palette, user can customize the color ranges. But the gradient color palette is not available in lens. Hence, the exact color palette can not be replicated in lens. However, if colors are customized based on particular use case (for example, red color for failures and green color for success) then we are not relying on default colors and configuring the color ranges manually.

@kush-elastic
Copy link
Collaborator

/test

@rajvi-patel-22 rajvi-patel-22 changed the title [Lens Migration] Migrate visualizations to lens [Lens Migration] Migrate visualizations to lens for ActiveMQ, Spring Boot and WebSphere Application Server Feb 23, 2023
@drewdaemon
Copy link
Contributor

drewdaemon commented Mar 2, 2023

I'm on the AppEx visualizations team. A few initial takes on the issue list:

Input Controls visualization is deprecated in Kibana and many of the dashboards contain these visualizations. Also, support for Input Controls has been stopped because of the new Input Controls. Hence, to replicate dashboard-level filters old Input Controls are replaced with new ones.

Perfect

TagCloud visualizations are not supported in the lens. [https://github.com/elastic/kibana/issues/95542]

Yes, we suggest horizontal bar chart as a current alternative.

Manually handle timestamp field in TSVB panels otherwise that panel will not be migrated using automation script.[https://github.com/elastic/kibana/issues/151109]

We had trouble reproducing this issue, but got a response and may be able to look more into this. Is this a blocker or more of a nice-to-have?

TSVB visualization doesn’t show negative values in UI. But the lens shows both negative and positive values. [https://github.com/elastic/elastic-charts/issues/135]

Looks like this will be fixed in 8.8.

TSVB visualization doesn’t show values for the small intervals. But the lens provides data even for the small intervals. [https://github.com/elastic/kibana/issues/87704]

Are you sure the linked issue is related?

Split chart functionality is not supported in Lens. [https://github.com/elastic/kibana/issues/82208]

If you mean splitting a chart into multiple smaller charts by a breakdown dimension, this is the issue to track: elastic/kibana#72063

While migrating the visualizations from TSVB to the lens, the colors are getting default ones instead of the configured ones.

We'd welcome an issue if you haven't already filed one!

In the lens migrated metric visualizations, values are displayed in the bottom right corner. [https://github.com/elastic/kibana/pull/124124#discussion_r804476235]

This according to the design of the new metric visualization. Metric always gets shown in bottom right to promote alignment between panels. See this discussion for more details.

Show toolbar option is not provided explicitly in Lens. Instead Download in CSV option is provided for the panels to download data. [https://github.com/elastic/kibana/pull/83430]

Is this a blocker/problem?

Panel titles are not displayed in lens metric visualization for Kibana 8.6.x version. [https://github.com/elastic/kibana/issues/148563]

True, this was by design. But they got added back in 8.7.

cc @stratoula

@stratoula
Copy link

Thanx @drewdaemon! One more question from my side. Can you give more details on

While migrating the visualizations from TSVB to the lens, the colors are getting default ones instead of the configured ones.

Do you mean with breakdown by Filters or another case? If yes, which?

@rajvi-patel-22
Copy link
Contributor Author

TagCloud visualizations are not supported in the lens. [https://github.com/elastic/kibana/issues/95542]

Yes, we suggest horizontal bar chart as a current alternative.

Make sense! We will migrate the TagCloud visualizations to horizontal bar chart.

Manually handle timestamp field in TSVB panels otherwise that panel will not be migrated using automation script.[https://github.com/elastic/kibana/issues/151109]

We had trouble reproducing this issue, but got a response and may be able to look more into this. Is this a blocker or more of a nice-to-have?

No. This is not a blocker for us. But it would be good if this issue gets resolved. It seems like in TSVB, if some fields are not configured (eg., time_field, data-view etc.) then it will show the chart using default values (eg., @timestamp, logs-*). On the other hand, to migrate those visualizations in lens, these fields must be configured otherwise user won't see Edit visualization in lens option.

TSVB visualization doesn’t show negative values in UI. But the lens shows both negative and positive values. [https://github.com/elastic/elastic-charts/issues/135]

Looks like this will be fixed in 8.8.

👍

TSVB visualization doesn’t show values for the small intervals. But the lens provides data even for the small intervals. [https://github.com/elastic/kibana/issues/87704]

Are you sure the linked issue is related?

The issue is related to the default interval. Lens and TSVB shows the different output for the same interval (when interval is small and configured to "auto") [GitHub Issue]

Split chart functionality is not supported in Lens. [https://github.com/elastic/kibana/issues/82208]

If you mean splitting a chart into multiple smaller charts by a breakdown dimension, this is the issue to track: elastic/kibana#72063

👍

While migrating the visualizations from TSVB to the lens, the colors are getting default ones instead of the configured ones.

We'd welcome an issue if you haven't already filed one!

I have created one GitHub issue for this.

In the lens migrated metric visualizations, values are displayed in the bottom right corner. [https://github.com/elastic/kibana/pull/124124#discussion_r804476235]

This according to the design of the new metric visualization. Metric always gets shown in bottom right to promote alignment between panels. See this discussion for more details.

👍

Show toolbar option is not provided explicitly in Lens. Instead Download in CSV option is provided for the panels to download data. [https://github.com/elastic/kibana/pull/83430]

Is this a blocker/problem?

No. This is not a blocker issue. Because user is still able to export data using Download as CSV option.

Panel titles are not displayed in lens metric visualization for Kibana 8.6.x version. [https://github.com/elastic/kibana/issues/148563]

True, this was by design. But they got added back in 8.7.

👍

@rajvi-patel-22
Copy link
Contributor Author

Thanx @drewdaemon! One more question from my side. Can you give more details on

While migrating the visualizations from TSVB to the lens, the colors are getting default ones instead of the configured ones.

Do you mean with breakdown by Filters or another case? If yes, which?

Yes, in breakdown functionality we don't have Gradient and Rainbow color palette. I have created the GitHub issue for the same.
image

@rajvi-patel-22
Copy link
Contributor Author

We still need to work on this PR. Hence putting this in the draft as of now.

@rajvi-patel-22 rajvi-patel-22 marked this pull request as draft March 6, 2023 11:21
@rajvi-patel-22
Copy link
Contributor Author

Raised individual PRs for integrations and achieve minimum Kibana version using manual migration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Service-Integrations Label for the Service Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants