[Dashboards] integ-test in Jenkins #1653

kavilla · 2022-02-18T05:12:09Z

Description

Execute the integ-test workflow for OpenSearch Dashboards including
tests.

As opposed to OpenSearch, we need an image with the appropriate libraries
so we pass the image but it does not have the wget so the steps were broken
to access the build manifest.

Hardcoding the OpenSearch build id for now until this get resolves:
#1492

ARM tests will fail still until this is resolved:
#1381

Signed-off-by: Kawika Avilla kavilla414@gmail.com

Issues Partially Resolved

#704

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov-commenter · 2022-02-18T05:13:20Z

Codecov Report

Merging #1653 (c337d9d) into main (18c173c) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #1653      +/-   ##
============================================
+ Coverage     94.68%   94.75%   +0.06%     
  Complexity       14       14              
============================================
  Files           163      168       +5     
  Lines          3446     3506      +60     
  Branches         21       26       +5     
============================================
+ Hits           3263     3322      +59     
- Misses          180      181       +1     
  Partials          3        3

Impacted Files	Coverage Δ
tests/jenkins/jobs/RunIntegTestScript_Jenkinsfile	`100.00% <100.00%> (ø)`
...nIntegTestScript_OpenSearch_Dashboards_Jenkinsfile	`100.00% <100.00%> (ø)`
src/manifests/bundle_manifest.py	`97.43% <0.00%> (-2.57%)`	⬇️
src/assemble_workflow/dist.py	`95.08% <0.00%> (-0.58%)`	⬇️
src/manifests/build_manifest.py	`100.00% <0.00%> (ø)`
src/assemble_workflow/bundle_recorder.py	`100.00% <0.00%> (ø)`
deployment/lib/artifacts-public-access.ts	`100.00% <0.00%> (ø)`
src/build_workflow/builder_from_source.py	`100.00% <0.00%> (ø)`
...loyment/lambdas/cf-url-rewriter/cf-url-rewriter.ts	`100.00% <0.00%> (ø)`
deployment/lambdas/cf-url-rewriter/https-get.ts	`100.00% <0.00%> (ø)`
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 18c173c...c337d9d. Read the comment docs.

kavilla · 2022-02-18T05:22:53Z

Build 52 was for linux x64 while build 53 was for linux arm64.

tests/jenkins/TestRunIntegTestScript.groovy

abhinavGupta16 · 2022-02-18T07:43:51Z

jenkins/opensearch-dashboards/integ-test.jenkinsfile

@@ -0,0 +1,132 @@
+lib = library(identifier: "jenkins@20211118", retriever: legacySCM(scm))


We'll need to add this job to #1468 once PR is merged, so we can add test cases for this when libTesters are ready

jenkins/opensearch-dashboards/integ-test.jenkinsfile

jenkins/opensearch-dashboards/distribution-build.jenkinsfile

dblock · 2022-02-18T16:44:10Z

Would it be simpler and future-proof to read the agent from the test manifest just like we do from the build manifest?

https://github.com/opensearch-project/opensearch-build/blob/main/manifests/2.0.0/opensearch-2.0.0.yml#L3

kavilla · 2022-02-21T20:27:30Z

Would it be simpler and future-proof to read the agent from the test manifest just like we do from the build manifest?

https://github.com/opensearch-project/opensearch-build/blob/main/manifests/2.0.0/opensearch-2.0.0.yml#L3

Will it change often? Regardless, I believe it would look cleaner and enables forks to have their own agents defined.

kavilla · 2022-02-22T02:42:09Z

Thanks for your comments @abhinavGupta16, made the updates as suggested.

jenkins/opensearch-dashboards/integ-test.jenkinsfile

tests/jenkins/lib-testers/RunIntegTestScriptLibTest.groovy

jenkins/opensearch/integ-test.jenkinsfile

jenkins/opensearch-dashboards/integ-test.jenkinsfile

abhinavGupta16 · 2022-02-22T05:18:28Z

@kavilla Thank you for making the changes. I have added a few more suggestions.

abhinavGupta16 · 2022-02-22T05:19:41Z

@peterzhuamazon - We have a few stages that run without docker. Should we add docker to all the stages?

kavilla · 2022-02-23T21:57:31Z

@opensearch-project/engineering-effectiveness, anyway to get more reviews on this PR? It would be a big win to have this running integration tests running for OpenSearch Dashboards.

Thank you!

gaiksaya

Not sure why tests/jenkins/data/opensearch-1.3.0-build.yml file is being added.

jenkins/opensearch-dashboards/integ-test.jenkinsfile

jenkins/opensearch/integ-test.jenkinsfile

vars/runIntegTestScript.groovy

kavilla · 2022-02-23T23:10:38Z

Not sure why tests/jenkins/data/opensearch-1.3.0-build.yml file is being added.

For testing purposes, the original jenkins/data files were added in here 6c54533

But to add it a test for runIntegTestScript it needed a build manifest but for the purpose of the test we added a build ID to a regular manifest. So it gave it my room to test on how manifests that go into the groovy script look like. I understand that there is https://github.com/opensearch-project/opensearch-build/tree/main/tests/data but given that more recent hash is in jenkins/data and the tests in the previous commit used jenkins/data it made sense to me to add the new file along with OpenSearch Dashboards.

However, if asking how come OpenSearch and not just OpenSearch Dashboards this PR comes with inclusion of new tests.

Execute the integ-test workflow for OpenSearch Dashboards including tests. As opposed to OpenSearch, we need an image with the appropriate libraries so we pass the image but it does not have the wget so the steps were broken to access the build manifest. Hardcoding the OpenSearch build id for now until this get resolves: opensearch-project#1492 ARM tests will fail still until this is resolved: opensearch-project#1381 Issue partially resolved: opensearch-project#704 Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

abhinavGupta16 · 2022-02-24T03:56:15Z

jenkins/opensearch-dashboards/integ-test.jenkinsfile

+        }
+        stage('download-manifest') {
+            agent {
+                node {


We should run this in a docker container

Within OpenSearch integ-test, they are able to have a single step that downloads and runs the test. However, the image that is required to run the integration tests for OpenSearch Dashboards does not support wget which is what the downloadBuildManifest requires.

Would the suggestion be to hardcode this stage to utilize a docker image that would normally be used by OpenSearch?

Also for my own curiosity, what's the benefit of running within a docker container vs running within a node? Does this prevent potential conflicts?

Running within container ensures a fresh run every-time because the container will be destroyed after every run. Also, as you rightly mentioned, it also helps prevent conflicts. It would also ensure that the node is not affected by the job.

We can specify any image that would help support this operation.

Also, why are we taking image name as input from the user? Can these possibly change without changes to the job?

We can specify any image that would help support this operation.

Do you know of which I can use? Used a couple docker images and all fail with line 1: wget: command not found. It would appear that's why OpenSearch is doing the same for integ tests: https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/integ-test.jenkinsfile#L37

Also, why are we taking image name as input from the user?

When OpenSearch Dashboards builds it already detected the docker image and will require to use the appropriate image for example: https://github.com/opensearch-project/opensearch-build/blob/main/manifests/2.0.0/opensearch-dashboards-2.0.0.yml#L8. So when the build pipeline triggers the job it will pass the values it already has on hand.

Can these possibly change without changes to the job?

Apologies I'm not clear what you mean here could you clarify?

You can curl instead of wget I believe ...

You can curl instead of wget I believe ...

Secret sauce, was able to update.

Does this job pick the docker image from the manifest file? If yes, we should not have an input param, but assign the image variable locally using manifest.

As of right now, the actual build manifest doesn't include the CI image stuff. The manifests in manifests/x.y.z/ do. There could be potential to pass MANIFEST, TEST_MANIFEST, and BUILD_MANIFEST_URL. But if we are going that route I would rather we open an issue and address that in both OpenSearch and OpenSearch Dashboards.

Input manifest: #1687
OpenSearch Integ test not running in docker: #1688

@kavilla - In that case, can we code in the docker image rather than taking it as an input parameter? It would reduce one parameter for the user.
Once the manifest supports holding the image, the job would then parse the build manifest and get the image from there

We could utilize #1687, which would have the image available. But that wouldn't resolve the issue about the number of params. I wouldn't see any issue if we did include the CI information into the actual build manifest to reduce the amount of params.

I created an issue for that: #1694 since that will require more refactoring within the actual build repo.

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

kavilla · 2022-03-02T08:51:47Z

@opensearch-project/engineering-effectiveness

bbarani · 2022-03-02T18:37:13Z

@opensearch-project/engineering-effectiveness

We are looking in to it.

jenkins/opensearch-dashboards/integ-test.jenkinsfile

peterzhuamazon

I dont have too much issues with this PR except a few naming convention.

Thanks.

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

peterzhuamazon · 2022-03-02T21:48:12Z

Need @abhinavGupta16 to remove his block before we merge.
Thanks.

abhinavGupta16 · 2022-03-02T22:02:24Z

Need @abhinavGupta16 to remove his block before we merge. Thanks.

Added comments. I believe we can get rid of the image name parameter. One less thing for the user to input and worry about

abhinavGupta16 · 2022-03-02T22:57:44Z

I have approved it for now, however, we should look at resolving these new issues at the earliest.

@kavilla @bbarani @gaiksaya @peterzhuamazon

kavilla requested a review from a team as a code owner February 18, 2022 05:12

kavilla added the opensearch-dashboards label Feb 18, 2022

kavilla linked an issue Feb 18, 2022 that may be closed by this pull request

Automate integration tests for OpenSearch Dashboards #704

Closed

kavilla requested review from ohltyler, dblock, peternied, peterzhuamazon and tianleh February 18, 2022 05:26

abhinavGupta16 suggested changes Feb 18, 2022

View reviewed changes

tianleh mentioned this pull request Feb 19, 2022

Implement a basic Jenkins job for running OpenSearch Dashboards integ test #1368

Closed

1 task

kavilla force-pushed the avillk/osd-integ-test branch 4 times, most recently from 8950881 to cea6ac6 Compare February 22, 2022 02:37

kavilla requested a review from abhinavGupta16 February 22, 2022 02:41

abhinavGupta16 reviewed Feb 22, 2022

View reviewed changes

kavilla force-pushed the avillk/osd-integ-test branch from a56608b to 212330c Compare February 23, 2022 21:11

kavilla requested a review from abhinavGupta16 February 23, 2022 21:44

gaiksaya reviewed Feb 23, 2022

View reviewed changes

kavilla force-pushed the avillk/osd-integ-test branch from 212330c to 170d581 Compare February 24, 2022 02:34

abhinavGupta16 reviewed Feb 24, 2022

View reviewed changes

[Dashboards] copy artifact permission JOB_BASE_NAME

2ca2fdb

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

kavilla force-pushed the avillk/osd-integ-test branch from 58779b4 to 2ca2fdb Compare February 28, 2022 10:06

kavilla added 2 commits March 1, 2022 20:55

[Test] Use curl for runIntegTestScript

3b28aa4

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

[Dashboards] use latest for OpenSearch

f2619f7

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

This was referenced Mar 2, 2022

Build manifest to include JOB_NAME #1686

Closed

Add wget for fetching manifests #1685

Closed

Integ Test Jenkinsfiles Input Manifest Param #1687

Closed

kavilla requested review from abhinavGupta16 and gaiksaya March 2, 2022 08:51

peterzhuamazon reviewed Mar 2, 2022

View reviewed changes

jenkins/opensearch-dashboards/integ-test.jenkinsfile Outdated Show resolved Hide resolved

peterzhuamazon reviewed Mar 2, 2022

View reviewed changes

[Tests] update input params for test jobs

c337d9d

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>

kavilla requested a review from peterzhuamazon March 2, 2022 20:20

peterzhuamazon approved these changes Mar 2, 2022

View reviewed changes

gaiksaya approved these changes Mar 2, 2022

View reviewed changes

kavilla mentioned this pull request Mar 2, 2022

Build manifest to include more CI information #1694

Closed

abhinavGupta16 approved these changes Mar 2, 2022

View reviewed changes

gaiksaya merged commit 2993c66 into opensearch-project:main Mar 2, 2022

kavilla deleted the avillk/osd-integ-test branch March 2, 2022 23:00

kavilla linked an issue Mar 14, 2022 that may be closed by this pull request

[Bug]: copyArtifactPermission path requires current job without path #1566

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Dashboards] integ-test in Jenkins #1653

[Dashboards] integ-test in Jenkins #1653

kavilla commented Feb 18, 2022 •

edited

Loading

codecov-commenter commented Feb 18, 2022 •

edited

Loading

kavilla commented Feb 18, 2022

abhinavGupta16 Feb 18, 2022

dblock commented Feb 18, 2022

kavilla commented Feb 21, 2022

kavilla commented Feb 22, 2022

abhinavGupta16 commented Feb 22, 2022

abhinavGupta16 commented Feb 22, 2022

kavilla commented Feb 23, 2022 •

edited

Loading

gaiksaya left a comment

kavilla commented Feb 23, 2022

abhinavGupta16 Feb 24, 2022 •

edited

Loading

kavilla Feb 28, 2022

abhinavGupta16 Feb 28, 2022

kavilla Mar 1, 2022

dblock Mar 1, 2022

kavilla Mar 1, 2022

kavilla Mar 1, 2022 •

edited

Loading

kavilla Mar 2, 2022 •

edited

Loading

abhinavGupta16 Mar 2, 2022

kavilla Mar 2, 2022

kavilla commented Mar 2, 2022

bbarani commented Mar 2, 2022

peterzhuamazon left a comment

peterzhuamazon commented Mar 2, 2022

abhinavGupta16 commented Mar 2, 2022

abhinavGupta16 commented Mar 2, 2022

		@@ -0,0 +1,132 @@
		lib = library(identifier: "jenkins@20211118", retriever: legacySCM(scm))

[Dashboards] integ-test in Jenkins #1653

[Dashboards] integ-test in Jenkins #1653

Conversation

kavilla commented Feb 18, 2022 • edited Loading

Description

Issues Partially Resolved

Check List

codecov-commenter commented Feb 18, 2022 • edited Loading

Codecov Report

kavilla commented Feb 18, 2022

Choose a reason for hiding this comment

dblock commented Feb 18, 2022

kavilla commented Feb 21, 2022

kavilla commented Feb 22, 2022

abhinavGupta16 commented Feb 22, 2022

abhinavGupta16 commented Feb 22, 2022

kavilla commented Feb 23, 2022 • edited Loading

gaiksaya left a comment

Choose a reason for hiding this comment

kavilla commented Feb 23, 2022

abhinavGupta16 Feb 24, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kavilla Mar 1, 2022 • edited Loading

Choose a reason for hiding this comment

kavilla Mar 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kavilla commented Mar 2, 2022

bbarani commented Mar 2, 2022

peterzhuamazon left a comment

Choose a reason for hiding this comment

peterzhuamazon commented Mar 2, 2022

abhinavGupta16 commented Mar 2, 2022

abhinavGupta16 commented Mar 2, 2022

kavilla commented Feb 18, 2022 •

edited

Loading

codecov-commenter commented Feb 18, 2022 •

edited

Loading

kavilla commented Feb 23, 2022 •

edited

Loading

abhinavGupta16 Feb 24, 2022 •

edited

Loading

kavilla Mar 1, 2022 •

edited

Loading

kavilla Mar 2, 2022 •

edited

Loading