Add functional test cypress workflow improvements and enable the workflow for in-house tests #6061

manasvinibs · 2024-03-07T05:48:17Z

Description

Addressing comments from this PR #5725

Update to this: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/.github/workflows/cypress_workflow.yml to execute the cypress tests within this repo.
This way we can get the release testing verification workflow triggered with a dispatch event with a specified build. While the existing cypress test workflow will address the code within opened pull requests and run the tests.
Build in the concept of CiGroups like the selenium tests have similar to FT repo: https://github.com/opensearch-project/opensearch-dashboards-functional-test/blob/main/package.json#L28. Define a folder structure in package json file and use it to parallel trigger workflows based on the ciGroups defined inside package json file.
Also addresses minor comments from the above PR.

Issues Resolved

Testing the changes

Look into CI workflow in the PR to verify the change in this PR: https://github.com/opensearch-project/OpenSearch-Dashboards/actions/runs/8184064533/job/22377905977?pr=6061

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

codecov · 2024-03-07T06:03:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.13%. Comparing base (144c022) to head (cd4e25f).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6061   +/-   ##
=======================================
  Coverage   67.13%   67.13%           
=======================================
  Files        3318     3318           
  Lines       64037    64037           
  Branches    10266    10266           
=======================================
+ Hits        42990    42992    +2     
+ Misses      18549    18548    -1     
+ Partials     2498     2497    -1

Flag	Coverage Δ
Linux_1	`31.63% <ø> (ø)`
Linux_2	`55.20% <ø> (ø)`
Linux_3	`44.60% <ø> (+<0.01%)`	⬆️
Linux_4	`35.14% <ø> (ø)`
Windows_1	`31.65% <ø> (ø)`
Windows_2	`55.17% <ø> (ø)`
Windows_3	`44.61% <ø> (ø)`
Windows_4	`35.14% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

manasvinibs · 2024-03-08T00:40:50Z

package.json

-
+    "cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
+    "osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
+    "osd:ciGroup11": "echo \"apps/vis_builder/*.js\""


Created an issue to address this concern #6083

Yea have we specified how we group the cypress tests within the dashboard repo into cigroups?

Currently in Dashboards repo we are grouping it similar to how we group it in FT repo using package json by defining folder structure in it. I'll have to look into better way of doing it or something similar to how selenium tests are doing it today by passing it as a command.

…flow for in-house tests Add cigroup matrix for parallel runs Signed-off-by: Manasvini B Suryanarayana <manasvis@amazon.com>

abbyhu2000

Synced up with @manasvinibs offline, verified the cypress workflow on dashboard cypress tests.

abbyhu2000 · 2024-03-08T01:07:10Z

.github/workflows/cypress_workflow.yml

+          echo "DASHBOARDS_SPEC=${DASHBOARDS_SPEC}" >> $GITHUB_ENV
+
+      - name: Run FT repo tests
+        if: ${{ matrix.group < 10 }}


Do we plan to migrate all cypress test from FT repo into dashboard repo in the future?

yes I think that will be ideal as we want feature and its tests to reside in the same place.

ashwin-pc

We should convert this to a reusable workflow and find a better way to update the CI groups. I'm okay merging this as is to unblock writing functional tests in OSD while we iterate and clean this up :)

opensearch-trigger-bot · 2024-03-08T02:11:33Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-6061-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 89fcf63ca0790858143a97c3bfb47460214afc7e
# Push it to GitHub
git push --set-upstream origin backport/backport-6061-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-6061-to-2.x.

…flow for in-house tests (opensearch-project#6061) Add cigroup matrix for parallel runs Signed-off-by: Manasvini B Suryanarayana <manasvis@amazon.com> (cherry picked from commit 89fcf63) Signed-off-by: Manasvini B Suryanarayana <manasvis@amazon.com>

manasvinibs · 2024-03-13T21:12:44Z

Manual backport #6139

manasvinibs requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, kristenTian, zhongnansu, ZilongX, Flyingliuhub, BSFishy, curq, bandinib-amzn, SuZhou-Joe, ruanyl and BionIT as code owners March 7, 2024 05:48

github-actions bot added the distinguished-contributor label Mar 7, 2024

manasvinibs changed the title ~~Add cigroup matrix for parallel runs~~ Add functional test cypress workflow improvements Mar 7, 2024

manasvinibs force-pushed the Issue-5892 branch from 0289055 to d1a3f56 Compare March 7, 2024 06:46

manasvinibs changed the title ~~Add functional test cypress workflow improvements~~ Add functional test cypress workflow improvements and enable the workflow for in-house tests Mar 7, 2024

ashwin-pc assigned ashwin-pc and abbyhu2000 Mar 7, 2024

manasvinibs commented Mar 8, 2024

View reviewed changes

Add functional test cypress workflow improvements and enable the work…

cd4e25f

…flow for in-house tests Add cigroup matrix for parallel runs Signed-off-by: Manasvini B Suryanarayana <manasvis@amazon.com>

manasvinibs force-pushed the Issue-5892 branch from d1a3f56 to cd4e25f Compare March 8, 2024 01:10

abbyhu2000 reviewed Mar 8, 2024

View reviewed changes

ashwin-pc approved these changes Mar 8, 2024

View reviewed changes

abbyhu2000 approved these changes Mar 8, 2024

View reviewed changes

ashwin-pc merged commit 89fcf63 into opensearch-project:main Mar 8, 2024
68 checks passed

ashwin-pc added the backport 2.x label Mar 8, 2024

opensearch-trigger-bot bot added the failed backport label Mar 8, 2024

manasvinibs added v2.13.0 failed backport and removed failed backport labels Mar 13, 2024

manasvinibs mentioned this pull request Mar 13, 2024

[Manual][Backport 2.x] Add functional test cypress workflow improvements and enable the workflow for in-house tests #6139

Open

7 tasks

manasvinibs added v2.14.0 and removed failed backport v2.13.0 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add functional test cypress workflow improvements and enable the workflow for in-house tests #6061

Add functional test cypress workflow improvements and enable the workflow for in-house tests #6061

manasvinibs commented Mar 7, 2024 •

edited

Loading

codecov bot commented Mar 7, 2024 •

edited

Loading

manasvinibs Mar 8, 2024

abbyhu2000 Mar 8, 2024

manasvinibs Mar 8, 2024

abbyhu2000 left a comment

abbyhu2000 Mar 8, 2024

manasvinibs Mar 8, 2024

ashwin-pc left a comment

opensearch-trigger-bot bot commented Mar 8, 2024

manasvinibs commented Mar 13, 2024

Add functional test cypress workflow improvements and enable the workflow for in-house tests #6061

Add functional test cypress workflow improvements and enable the workflow for in-house tests #6061

Conversation

manasvinibs commented Mar 7, 2024 • edited Loading

Description

Issues Resolved

Testing the changes

Check List

codecov bot commented Mar 7, 2024 • edited Loading

Codecov Report

manasvinibs Mar 8, 2024

Choose a reason for hiding this comment

abbyhu2000 Mar 8, 2024

Choose a reason for hiding this comment

manasvinibs Mar 8, 2024

Choose a reason for hiding this comment

abbyhu2000 left a comment

Choose a reason for hiding this comment

abbyhu2000 Mar 8, 2024

Choose a reason for hiding this comment

manasvinibs Mar 8, 2024

Choose a reason for hiding this comment

ashwin-pc left a comment

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Mar 8, 2024

manasvinibs commented Mar 13, 2024

manasvinibs commented Mar 7, 2024 •

edited

Loading

codecov bot commented Mar 7, 2024 •

edited

Loading