Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing Automation For Elyra Test in Dspv2 #1394

Merged
merged 5 commits into from
Apr 22, 2024
Merged

Fixing Automation For Elyra Test in Dspv2 #1394

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
95adecc
Fixing Automation For Elyra Test in Dspv2
Raghul-M Apr 18, 2024
7ac5dcb
removed comment
Raghul-M Apr 18, 2024
6ea8263
Refactor Elyra tests. Move one of them to Smoke. Fix bugs
jgarciao Apr 19, 2024
72333d5
Delete testing tag
jgarciao Apr 19, 2024
e95f2b7
Add keyword teardown to stop the workbench
jgarciao Apr 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 2 additions & 8 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataSciencePipelines.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,14 +166,8 @@ Open Pipeline Detail
Open Pipeline Run
[Documentation] Open the Pipeline Run detail.
[Arguments] ${pipeline_run_name}
${present}= Run Keyword And Return Status Element Should Be Visible xpath=//a[text()='Runs']
# We can run Open Pipeline Run step multiple times. If the menu is expanded, then it will colapse
IF ${present} == ${FALSE}
Wait Until Page Contains Element xpath=//button[text()='Data Science Pipelines']
Click Element xpath=//button[text()='Data Science Pipelines']
Wait Until Page Contains Element xpath=//a[text()='Runs']
END
Click Element xpath=//a[text()='Runs']
Navigate To Page Data Science Pipelines Runs
ODHDashboard.Maybe Wait For Dashboard Loading Spinner Page
Wait Until Page Contains Element xpath=//*[@data-testid="active-runs-tab"] timeout=30s
Click Element xpath=//*[@data-testid="active-runs-tab"]
Wait Until Page Contains Element xpath=//span[text()='${pipeline_run_name}']
Expand Down
4 changes: 2 additions & 2 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Pipelines.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -386,8 +386,8 @@ Verify Successful Pipeline Run Via Pipelines Runs UI
... View: Data Science Pipelines -> Runs, Triggered tab
[Arguments] ${pipeline_name}
Menu.Navigate To Page Data Science Pipelines Runs
Wait Until Page Contains Element xpath://span[text()='Triggered']
Click Element //span[text()='Triggered']
Wait Until Page Contains Element xpath://span[text()='Triggered'] timeout=10s
Click Element //span[text()='Active']
Wait Until Pipeline Run Is Started pipeline_name=${pipeline_name}
... timeout=10s
Wait Until Pipeline Run Is Finished pipeline_name=${pipeline_name}
Expand Down
2 changes: 1 addition & 1 deletion ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Workbenches.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ Access To Workbench
Wait Until Page Contains Element xpath://div[@class="monaco-dialog-box"] timeout=60s
Wait Until Page Contains Do you trust the authors of the files in this folder?
ELSE IF "${expected_ide}"=="JupyterLab"
Wait Until Page Contains Element xpath://div[@id="jp-top-panel"] timeout=60s
Wait Until Page Contains Element xpath://div[@id="jp-top-panel"] timeout=90s
Maybe Close Popup
ELSE
Fail msg=Unknown IDE typy given: '${expected_ide}'. Please check and fix or implement.
Expand Down
119 changes: 39 additions & 80 deletions ...ts/400__ods_dashboard/430__data_science_pipelines/435__data-science-pipelines-elyra.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@
Resource ../../../Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Storages.resource
Resource ../../../Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/DataConnections.resource
Resource ../../../Resources/Page/ODH/ODHDashboard/ODHDataScienceProject/Pipelines.resource
Resource ../../../Resources/Page/ODH/ODHDashboard/ODHDataSciencePipelines.resource
Library Screenshot
Library String
Library DebugLibrary
Library JupyterLibrary
Test Tags DataSciencePipelines
Expand All @@ -24,73 +26,45 @@
${SVG_INTERACTABLE} = /*[name()="g" and @class="d3-canvas-group"]
${SVG_PIPELINE_NODES} = /*[name()="g" and @class="d3-nodes-links-group"]
${SVG_SINGLE_NODE} = /*[name()="g" and contains(@class, "d3-draggable")]
${PRJ_TITLE} = elyra-test-project
${PRJ_TITLE} = elyra-test
${PRJ_DESCRIPTION} = testing Elyra pipeline functionality
${PV_NAME} = ods-ci-pv-elyra
${PV_DESCRIPTION} = ods-ci-pv-elyra is a PV created to test Elyra in workbenches
${PV_SIZE} = 2
${ENVS_LIST} = ${NONE}
${DC_NAME} = elyra-s3
@{IMAGE_LIST} PyTorch TensorFlow TrustyAI


*** Test Cases ***
Verify Pipeline Is Displayed Correctly In Standard Data Science Workbench
[Documentation] Loads an example Elyra pipeline and confirms the Elyra web UI displays it correctly
[Tags] Sanity Tier1
... ODS-2197
[Setup] Elyra Pipelines SDS Setup
Create Workbench workbench_title=elyra-sds workbench_description=Elyra test
... prj_title=${PRJ_TITLE} image_name=Standard Data Science deployment_size=Small
... storage=Persistent pv_existent=${FALSE}
... pv_name=${PV_NAME} pv_description=${PV_DESCRIPTION} pv_size=${PV_SIZE}
... envs=${ENVS_LIST}
Start Workbench workbench_title=elyra-sds timeout=300s
Launch And Access Workbench workbench_title=elyra-sds
Clone Git Repository And Open https://github.com/redhat-rhods-qe/ods-ci-notebooks-main
... ods-ci-notebooks-main/notebooks/500__jupyterhub/elyra/run-pipelines-on-data-science-pipelines/hello-generic-world.pipeline # robocop: disable
Verify Hello World Pipeline Elements

Verify Pipeline Can Be Submitted And Runs Correctly From Standard Data Science Workbench
[Documentation] Submits an example Elyra pipeline to be run by Data Science Pipelines and
... Confirms that it runs correctly
[Tags] Sanity Tier1
... ODS-2199
Set Runtime Image In All Nodes runtime_image=Datascience with Python 3.9 (UBI9)
Run Pipeline
Wait Until Page Contains Element xpath=//a[.="Run Details."] timeout=30s
${pipeline_run_name} = Get Pipeline Run Name
${handle} = Switch To Pipeline Execution Page

## TODO: modify it to use Pipeles > Runs
# Verify Successful Pipeline Run Via Project UI pipeline_run_name=${pipeline_run_name}
# ... pipeline_name=hello-generic-world project_name=${PRJ_TITLE}

Switch Window ${handle}
Click Element //button[.="OK"]
[Teardown] Elyra Pipelines SDS Teardown

Verify Elyra Pipelines In SDS-Based Images
[Documentation] Runs the same Elyra test of the first two test cases in the other images based on SDS
... (Tensorflow, Pytorch and TrustyAI)
Verify Pipelines Integration With Elyra When Using Standard Data Science Image
[Documentation] Verifies that a workbench using the Standard Data Science Image can be used to
... create and run a Data Science Pipeline
[Tags] Smoke Tier1
... ODS-2197 ODS-2199 RunThisTest
Verify Pipelines Integration With Elyra Running Hello World Pipeline Test Standard Data Science

Verify Pipelines Integration With Elyra When Using Standard Data Science Based Images
[Documentation] Verifies that a workbench using an image based on the Standard Data Science Image
... can be used to create and run a Data Science Pipeline
... Note: this a templated test case
... (more info at https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#test-templates)
[Template] Verify Pipelines Integration With Elyra Running Hello World Pipeline Test
[Tags] Sanity Tier1
... ODS-2271
[Setup] Elyra Pipelines SDS Setup
FOR ${img} IN @{IMAGE_LIST}
Run Elyra Hello World Pipeline Test ${img}
END
[Teardown] Elyra Pipelines SDS Teardown
PyTorch
TensorFlow
TrustyAI
HabanaAI


*** Keywords ***
Elyra Pipelines Suite Setup
Elyra Pipelines Suite Setup # robocop: off=too-many-calls-in-keyword
[Documentation] Suite Setup
Set Library Search Order SeleniumLibrary
RHOSi Setup

Elyra Pipelines SDS Setup
[Documentation] Suite Setup, creates DS Project and opens it
Launch Data Science Project Main Page
${project_suffix} = Generate Random String 3 [NUMBERS]
Set Suite Variable \${PRJ_TITLE} ${PRJ_TITLE}-${project_suffix}
Dismissed Show dismissed Hide dismissed
Create Data Science Project title=${PRJ_TITLE} description=${PRJ_DESCRIPTION}
${to_delete} = Create List ${PRJ_TITLE}
Set Suite Variable ${PROJECTS_TO_DELETE} ${to_delete}
Expand All @@ -100,41 +74,17 @@
Create Pipeline Server dc_name=${DC_NAME}
... project_title=${PRJ_TITLE}
Wait Until Pipeline Server Is Deployed project_title=${PRJ_TITLE}
Create Env Var List If RHODS Is Self-Managed

Elyra Pipelines SDS Teardown
[Documentation] Closes the browser and deletes the DS Project created
Close All Browsers
Delete Data Science Projects From CLI ocp_projects=${PROJECTS_TO_DELETE}
Sleep 15s reason=Wait until pipeline server is detected by dashboard

Elyra Pipelines Suite Teardown
[Documentation] Closes the browser and performs RHOSi Teardown
Delete Data Science Projects From CLI ocp_projects=${PROJECTS_TO_DELETE}
Close All Browsers
RHOSi Teardown

Verify Hello World Pipeline Elements
[Documentation] Verifies that the example pipeline is displayed correctly by Elyra
Wait Until Page Contains Element xpath=${SVG_CANVAS}
Maybe Migrate Pipeline
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Load weather data"] # robocop: disable
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Part 1 - Data Cleaning.ipynb"] # robocop: disable
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Part 2 - Data Analysis.ipynb"] # robocop: disable
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Part 3 - Time Series Forecasting.ipynb"] # robocop: disable

Create Env Var List If RHODS Is Self-Managed
[Documentation] If RHODS is running a self-managed environment, this keyword will create a dictionary containing
... The required environment variables for Elyra to trust the endpoint SSL connection.
${self_managed} = Is RHODS Self-Managed
IF ${self_managed}==${TRUE}
${env_vars_ssl} = Create Dictionary
... PIPELINES_SSL_SA_CERTS=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
... k8s_type=Config Map input_type=Key / value
${list} = Create List ${env_vars_ssl}
Set Suite Variable ${ENVS_LIST} ${list}
END

Run Elyra Hello World Pipeline Test # robocop: disable
[Documentation] Runs the same steps of the first two tests of this Suite, but on different images
Verify Pipelines Integration With Elyra Running Hello World Pipeline Test # robocop: off=too-many-calls-in-keyword
[Documentation] Creates and starts a workbench using ${img} and verifies that the Hello World sample pipeline
... runs successfully
[Arguments] ${img}
Create Workbench workbench_title=elyra_${img} workbench_description=Elyra test
... prj_title=${PRJ_TITLE} image_name=${img} deployment_size=Small
Expand All @@ -144,13 +94,22 @@
Start Workbench workbench_title=elyra_${img} timeout=300s
Launch And Access Workbench workbench_title=elyra_${img}
Clone Git Repository And Open https://github.com/redhat-rhods-qe/ods-ci-notebooks-main
... ods-ci-notebooks-main/notebooks/500__jupyterhub/elyra/run-pipelines-on-data-science-pipelines/hello-generic-world.pipeline # robocop: disable
... ods-ci-notebooks-main/notebooks/500__jupyterhub/pipelines/v2/elyra/run-pipelines-on-data-science-pipelines/hello-generic-world.pipeline # robocop: disable
Verify Hello World Pipeline Elements
Set Runtime Image In All Nodes runtime_image=Datascience with Python 3.9 (UBI9)
Run Pipeline pipeline_name=${img} Pipeline
Wait Until Page Contains Element xpath=//a[.="Run Details."] timeout=30s
${pipeline_run_name} = Get Pipeline Run Name
Switch To Pipeline Execution Page
Verify Successful Pipeline Run Via Pipelines Runs UI pipeline_name=${img} Pipeline
Verify Pipeline Run Is Completed ${pipeline_run_name} timeout=5m
Open Data Science Project Details Page project_title=${PRJ_TITLE} tab_id=workbenches
Stop Workbench workbench_title=elyra_${img}

Verify Hello World Pipeline Elements
[Documentation] Verifies that the example pipeline is displayed correctly by Elyra
Wait Until Page Contains Element xpath=${SVG_CANVAS} timeout=10s
Maybe Migrate Pipeline
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Load weather data"] # robocop: disable
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Part 1 - Data Cleaning.ipynb"] # robocop: disable
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Part 2 - Data Analysis.ipynb"] # robocop: disable
Page Should Contain Element xpath=${SVG_CANVAS}${SVG_INTERACTABLE}${SVG_PIPELINE_NODES}${SVG_SINGLE_NODE}//span[.="Part 3 - Time Series Forecasting.ipynb"] # robocop: disable
Loading